Private AI Document Review: Verify Before You Rely on It

Private AI document review needs a traceable comparison with the original. Fix the source version, check each material claim and restore missing conditions before using the analysis. Keep unresolved points visible and data-handling approval separate.

Fix the Source and the Question First

A document owner deciding how to review confidential documents with AI has two separate questions: whether the information may enter the chosen service, and whether the answer represents it accurately. This guide addresses the second after the first has been resolved for the actual workflow. A private label does not settle either question.

Keep the original available to the reviewer. Record its version and the sections supplied for analysis, including any schedules or definitions needed to understand them. If text was extracted from a scan, compare the relevant passage with the page image before relying on it. A missing line or unreadable table is a gap in the input, not evidence that the document contains no restriction.

Set a narrow review question, such as which conditions apply to payment. Ask for each conclusion with its section or page reference and any unresolved dependency. Do not request a verdict on the entire agreement when only a few pages are available. NIST's discussion of confabulation explains that generated answers can be confidently wrong and can include invented citations. A reference supplied by the model still needs checking.

Follow One Conclusion Back to Its Clause

Consider this entirely fictional example, created to demonstrate private AI contract review. It is not a customer document, an observed model result or legal advice. The invented source is a supplier agreement, version 2, page 4, clause 7.2:

"Payment is due within 30 calendar days after receipt of a valid invoice, provided the buyer has accepted the deliverables in writing. The buyer must report disputed charges within five business days of receipt."

The deliberately flawed draft summary says: "Payment is due within 30 days of invoice receipt." It preserves a familiar number while dropping the valid-invoice requirement, written acceptance and the separate dispute deadline. A reader could mistake that shortened sentence for the full payment rule.

Open clause 7.2 and compare the draft against each part:

A source-faithful revision could read: "Clause 7.2 sets payment within 30 calendar days after receipt of a valid invoice, subject to written acceptance of the deliverables. It also requires disputed charges to be reported within five business days of receipt."

That revision preserves the excerpt's conditions. It does not establish whether an invoice is valid, whether acceptance occurred or how another clause affects the obligation. Those questions need their own evidence and, where interpretation matters, the responsible specialist.

Check for Omissions as Well as Wrong Statements

In private AI document analysis, checking only the sentences that appear in the answer misses what was left out. Read the relevant source section independently, then ask which condition, exception or referenced definition could change the answer. A correct sentence can still provide an incomplete basis for a decision.

Microsoft's guidance on validating generated output recommends tracing important claims to sources and checking missing context. The guidance concerns Microsoft Copilot; it does not establish another assistant's capabilities. Here, apply that principle through a small review record containing:

• The conclusion being considered and its intended use.
• The exact source version and section or page locator.
• The supporting passage and any condition that limits it.
• The finding: supported, needs correction or unresolved.
• The reviewer responsible for the remaining question.

Keep this record within the approved document-handling environment. Where a locator is sufficient, avoid copying confidential passages into a widely shared summary. When a passage has to accompany the finding, preserve its access restrictions.

Check numbers, dates, units and defined terms against the source. If a conclusion depends on an absent schedule, mark it unresolved and request that schedule. Asking the model to review its answer can suggest gaps; its agreement with itself is not independent confirmation.

Close the Review with a Usable Decision

Teams considering private AI for internal policies can use the same method to check who a rule applies to, which exceptions remain and which version is current. A policy summary that removes an exception may change the instruction employees receive. Route ambiguity to the document owner rather than filling it with an assumed rule.

Keep source content separate from instructions governing the review. OWASP's document-related prompt injection guidance describes how external material can influence model behavior. Separating untrusted content can reduce its influence; it is not a guarantee. Unexpected instructions in a document are a reason to pause and inspect the affected analysis, not permission to follow them.

Retain the corrected answer with its source version and outstanding questions. If the document changes, reopen the affected comparisons. A completed check supports the stated review scope; it does not certify the whole document or replace legal, financial or security judgment.

If R2 Copilot is under evaluation, use a fictional excerpt with a known omitted condition to assess the proposed workflow. Compare the answer with the original and check data-handling permission separately; this is an evaluation task, not a claim about product functionality.

Choose one conclusion that will affect a decision today. Open its cited passage, restore every relevant qualifier and give any unresolved dependency to a named owner before using the summary.