Verified AI answer drafting

What a verified draft gives you on a live deal

When a counterparty asks a diligence question, CogniSuite drafts the answer and links every factual claim to the exact passage in the exact file that supports it. Clicking a linked phrase opens that document in the viewer with the supporting quote highlighted. The verified source files also render as chips beside the draft, so a reviewer sees the whole evidence set without reading the draft twice.

The more important behaviour is what happens when the evidence is not there. If the model returns an affirmative answer and none of its citations survive checking, the answer text is discarded. The reviewer instead gets a line saying the assistant could not verify a specific answer and that the item has been routed for manual review, with the draft downgraded and flagged. A reviewer who gets no answer has to write one. A reviewer who gets a wrong answer may not notice, and the other side receives it.

Why telling a model to cite its sources is not a control

The legal profession is working through this problem in public. In Mata v. Avianca, Inc., Judge P. Kevin Castel of the Southern District of New York sanctioned two attorneys and their firm $5,000 on 22 June 2023 after they filed a brief citing six judicial decisions that did not exist. ChatGPT had produced them, and had confirmed they were real when asked. (opinion)

That was not a one-off. The AI Hallucination Cases database maintained by Damien Charlotin, which records only decisions where a court found or implied that a party relied on hallucinated content, had passed 1,600 tracked decisions by July 2026.

Retrieval on its own does not fix it. The Stanford RegLab study Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools tested purpose-built, retrieval-backed legal research products and found hallucination rates between 17% and 33% of queries, against 43% for GPT-4. Many of those errors were not invented case names. They were real citations attached to propositions the cited source did not support.

The structural reason is simple. When a model is told to cite its sources, the citation is generated text produced by the same process that produced the claim. Nothing has checked whether the quoted words exist in the cited document. The instruction makes the typical output better. It does not prevent the failure case, because nothing checks the citation.

M&A carries the same exposure in a different form. A diligence answer sent to the buy side can end up underpinning a rep, a disclosure schedule entry, or a price adjustment. An answer that cites the right file but quotes a figure that file does not contain is worse than no answer. The citation makes a reviewer less likely to check it.

How the draft is produced and then checked

The model is not asked for prose. It is required to emit a structured draft through a forced tool call with a fixed schema: an availability value of yes, partial or no; a short answer; a statement of what is missing; a list of citations, each carrying a quote, a document id and an anchor; and the list of source document ids. A forced tool call is used rather than a strict JSON response format, because the tool call preserves the model's reasoning while still constraining the shape of the output.

The application then verifies that output before a character reaches the screen. Three gates run over every citation.

The permission gate. The citation's document id must resolve to a document in the set that was actually passed to the model. Exact id first, then a one-based index in case the model referred to "document 3", then an exact title match. Anything that does not resolve is dropped. A citation to a document outside the permitted set cannot survive.

The faithfulness gate. The quote must appear in that document's extracted text as a substring after whitespace normalisation, with an alphanumeric-only fallback so soft hyphens, ligatures and punctuation artefacts from PDF and spreadsheet extraction do not discard a genuine quote. Quotes below a short minimum length are rejected, which removes empty and near-empty strings rather than applying any test of how much a quote actually proves.

The anchor gate. The anchor, meaning the span of the answer that the quote is supposed to support, must actually occur in the answer text. A citation that supports nothing the reader will see is dropped.

Then the fail-closed rule. If the draft claimed the answer was available and no citation survived all three gates, the answer is replaced with the manual-review line, the availability is downgraded to partial, and the draft is marked as unverified for want of a surviving citation.

Surviving drafts are scanned three more times. A word-boundary pattern scan looks for material non-public terms across the answer, the verified quotes and the missing field: reserve price, floor price, walk-away price, competing bid, and "bidder X". A figure check compares money-shaped numbers in the answer against the verified quotes, canonicalising first so that $45.2M, 45.20 and 45.2 million compare equal, and deliberately ignoring bare integers so years and day numbers do not generate noise. A polarity check flags cases where a verified quote carries a negation and the answer does not, or the reverse. All three raise a warning banner for the reviewer. None of them block.

Finally, the first occurrence of each distinct verified anchor is rewritten into a link to the document viewer with the supporting quote passed as a highlight target. Anchors past a certain length are left as plain prose, because linking a whole clause makes an answer unreadable, and the source chip still carries the click.

Both counterparty-facing draft features, the Q&A board and request threads, run through this same generation and verification function. There is one code path, so the gates cannot drift apart between the two surfaces. What does differ is what gets retrieved before the model writes. A request draft is searched against the request's own text, while a Q&A draft is searched against the thread's title alone, so the body of the question never reaches the search and the grounding is only as good as the title the other side gave the thread. Verification cannot compensate for a document that retrieval never returned.

What the model is allowed to see before it writes

Verification is the second layer. The first is scope. Counterparty-facing drafts are grounded in what the other side may read, not what the advisor may read.

For a request thread, the app resolves the parent request list, enumerates every counterparty organisation that can see that list, and admits a folder only if every one of them may read it. That is an intersection, not a union. If the owning organisation cannot be positively resolved, retrieval is denied rather than falling back to the advisor's own access. On the Q&A board, grounding is scoped to the thread creator's own access, and it admits nothing at all unless that creator resolves to a buy-side role. That default is deliberately blunt: a thread whose creator does not resolve cleanly grounds in no documents, and the draft comes back as unavailable rather than answered from the advisor's wider view of the room. Organisation-level checks ignore per-user overrides so an individual grant cannot widen the baseline. More on the permission model is on our security page.

Drafts are returned only to advisory roles. They are working content, generated inside a review workflow.

What server-side verification rules out

Four failure modes stop being probabilistic and become impossible.

• A citation to a document outside the permitted set. It is dropped whether the id is wrong, invented, or belongs to a document the counterparty may not read.
• A quote that does not appear in the document it is attributed to. Fabricated quotes, quotes moved from one document to another, and paraphrases presented as quotations all fail the substring test.
• An affirmative answer with no surviving evidence. It is replaced rather than shown.
• A citation attached to text the reader will never see, which is how a citation list ends up looking thorough while supporting nothing.

None of these depend on the model behaving well. They run on the server, over the stored document text, after generation and before rendering.

What it does not rule out

• Verification proves a quote exists. It does not prove the quote means what the answer says it means. A verbatim sentence can be lifted out of a context that qualifies it. The anchor link exists so a reviewer can check exactly that, quickly.
• The figure check is intra-answer, not cross-document. It compares numbers in the answer against that answer's own verified quotes. There is no engine reconciling a figure in one document against a conflicting figure in another. The chat prompt asks the model to surface disagreements it notices, and that is an instruction, not a mechanism.
• Verification runs against extracted text. A scanned PDF with no text layer gives a quote nothing to match, so such a document effectively cannot be cited. There is no OCR in the pipeline.
• Retrieval uses one vector per document, built from a summary plus the opening of the transcribed text. Long documents are represented by their beginning and there is no passage-level index, so a relevant clause deep inside a long agreement may not be retrieved at all. Verification cannot rescue a document that retrieval never surfaced.
• Enrichment runs in the background after upload and is best-effort. A document whose embedding failed is stored and viewable but is not retrievable by AI, and there is no indicator on the file. Recovery is an explicit re-run by the deal team.
• The risk flags are heuristics. The pattern scan will miss a disclosure phrased around its terms. The polarity check is crude by design. They direct attention. They do not certify an answer.
• Retrieval admits any folder the asker can read at all, including view-only and watermarked tiers. A verified quote can therefore reproduce text from a document that person may view but not download.
• Prompt-injection defence is a shared guardrail block plus input sanitisation patterns. That is a speed bump, not a boundary. The real boundaries are the per-deal database file, the folder permissions applied before retrieval, and the quote verification described above.
• Every draft is a draft. Nothing reaches a counterparty without a person sending it.

How this changes the review job

Reviewing an unverified AI answer means redoing the work: find the source, confirm the number, confirm the wording. Reviewing a verified draft means checking judgement: is this quote being read correctly, is this the right document to cite, should this be disclosed at all. The first job takes about as long as answering the request from scratch. The second one is what a reviewer is for, and it holds up across a few hundred items.