Training Frontier Review Table Agents with Harvey

Applied Compute and Harvey partnered to train a model for Harvey’s Review Table product, achieving higher accuracy and lower latency at a fraction of the cost of frontier models.

When a workflow fans out across many model calls as in Harvey’s Review Table, small improvements in accuracy, latency, and cost can compound into meaningful product-level gains. Review Table is one of Harvey’s highest inference-volume workloads in production, so every basis point counts. We trained a specialized model for Harvey’s existing semantic-retrieval workflow, testing how far we could push the cost-quality frontier.

Harvey Review Table

Review Table is Harvey's large-scale document analysis tool. It allows lawyers to complete tasks that require reviewing enormous volumes of documents, such as litigation document review or the contract-analysis phase of due diligence, far faster than a human team alone.

A lawyer uploads up to 10,000 files, writes questions about each document in the corpus (these questions will become the column headers), and then Harvey returns a grid: one row per document group, one cell per question, where every cell contains an answer. Harvey also provides its reasoning, and citations to the source text.

Questions vary widely in what they demand of the model. Some ask to extract values that appear verbatim in the text. Others require dates, currencies, or durations to be identified and then parsed into a normalized format. Still others require information scattered across multiple sections to be synthesized into free-form analysis.

Every cell is its own model call against one or more documents. A single table – thousands of files by dozens of questions – can trigger hundreds of thousands of model calls. At that scale, small differences in per-call accuracy, latency, and cost compound quickly. That compounding is what makes Review Table a natural target for a specialized model.

Data

Harvey has strict data requirements, and no customer data is ever used for training purposes. To model document and query distributions that we expect to see in practice, we scraped open-source legal data, including filings, contracts, emails, and other document types, and turned it into an offline document corpus. This corpus became the foundation for a synthetic dataset designed to reflect how lawyers actually use Review Tables.

We built robust data pipelines using the Applied Compute Agent Cloud, or AC2. For every document, we created metadata and embeddings tracking provenance, content, filetype, length, and other attributes. These tags let us filter and deduplicate the corpus, preserving a broad mix of legal domains, document structures, and content before downselecting to approximate the Review Table distribution. To create tasks, we prompted frontier models to emulate representative user personas and generate diverse queries across the corpus. Next, oracle agents with full access to tools and the environment determined ground-truth answers for each query, including when the correct response is to abstain. Finally, several rounds of quality control with human experts compared answers across agents to ensure the training examples were high quality.

Reward design

Each cell in the Review Table needs to meet many intermediate requirements before providing a quality final answer for the user. The cell needs to identify the right part(s) of the document, correctly answer the question (or abstain from answering when not applicable), format the answer according to the user’s specifications, return the correct value, and cite the correct evidence. To meet these requirements, we designed a reward function that grades each of these dimensions separately.
For a standard answer, the base reward is:

The minimum acts as a gate such that both schema adherence and correctness of the final answer must be satisfied. Schema adherence, as well as correctness for verifiable response types like dates and numerics, can be computed deterministically. The correctness for extraction and free-response answers are evaluated by an LLM judge that determines whether the model’s response is semantically equivalent to a reference response.

Citations receive a separate sentence-level score. One challenge of explicitly rewarding cited evidence in RL training is preventing the model overzealously citing passages from the document. We found that adding a term to the reward consisting of a product between recall and precision was effective at optimizing citation quality, while penalizing citation spamming behavior more than other metrics like F1 score.

Recall measures how much of the reference evidence the model cited:

Precision measures how many cited sentences actually support the answer:

Each cited sentence is evaluated independently. Invalid citation indices remain in the denominator, preventing the model from receiving free credit for citations that cannot be resolved to the source document.

We combine precision and recall as:

The final reward folds citation quality into the base task score with a soft multiplicative term:

This construction keeps answer quality as the most important component of the model’s score. A wrong answer still receives zero regardless of its citations. A fully correct and well-cited answer receives full credit. A correct answer with missing or poorly selected evidence receives less than full credit, but is not treated as equivalent to a wholly incorrect answer. We empirically determined $\lambda=0.3$ to be a suitable weighting for this reward.

Specialized models for Review Table

Harvey’s Review Tables rely on substantial harness and prompt engineering for effective information retrieval. We trained a custom model in the production Review Table harness using Applied Compute’s AC2 platform to optimize for frontier model performance at a fraction of the cost.

A single-turn model optimized for semantic search extraction. The production Review Table harness uses a semantic search to retrieve snapshots of the relevant document corpus for each Review Table cell. This creates a large prefill designed to include information needed to answer the query. During model training, the policy learns to reason over the context, select the correct information, and output the exact schema and behavior that lawyers want.

Across our evaluation set, the custom Review Table model has improved answer quality while using fewer resources than the strongest general-purpose baselines. The largest gains were in correctness and citation quality, where the model achieved a 22.4% increase compared to citation quality over the base GLM 5.2 checkpoint. At the same time, average cost per cell fell by 54.8% compared to Claude Sonnet 5 on the same tasks.

Answer and citation quality

On answer quality, the AC Review Table model reached 0.903 on our benchmark, compared with 0.867 for Fable 5 and 0.857 for GPT-5.6-Sol. This represents an improvement of 5.8 percentage points over the base GLM5.2 checkpoint.

Citation quality followed a similar pattern, where the custom model achieved 84.1% citation precision and 91.93% citation recall. Relative to the GLM5.2 baseline, this represents an increase of 15.38 percentage points for the custom model’s citation score.

These results show that model specialization alone can materially improve the Review Table production frontier, without changing the retrieval architecture.

When agentic search helps

The results above show how post-training can improve Review Table performance within Harvey’s existing single-turn semantic-retrieval setup. Fixed retrieval works well when the initial semantic search surfaces the right evidence, but it can be inefficient when only a small portion of a document is relevant. The model is also unable to retrieve additional information if the initial context is incomplete. This motivated a separate experiment: could an agentic model retrieve evidence more selectively while maintaining comparable answer quality?

To isolate the effect of the retrieval harness from the main Review Table model results, we trained two Qwen3.6-35B-A3B models under the same answer score objective. One model was trained in the existing single-turn semantic-retrieval harness, which retrieves a broad set of document context before inference. The other was trained in a multi-turn agentic harness equipped with read- and grep-style tools, allowing it to search the document corpus iteratively as it reasons.

The two approaches have different computational profiles: single-turn retrieval pays for large context up front, while the agentic system instead trades that large prefill for several smaller model turns and targeted searches. This can be particularly useful for questions that can be narrowed to specific information; for example, an agent can search for a particular provision or email address directly rather than loading the broad context.

We compared the two Qwen3.6 models after training them to nearly identical answer scores. The agentic model reached 0.799, compared to 0.794 with the single-turn model and harness. At this matched level of answer quality, the agentic harness used 50.3% fewer input tokens and 28.6% fewer output tokens, trading a modest increase in response time for a large reduction in total token usage. These results suggest that retrieval architecture is an important optimization lever alongside model post-training: training a model to use a more targeted, agentic retrieval strategy can substantially improve token efficiency while maintaining high quality model responses.

Conclusion

High-volume document analysis makes small model-level improvements compound quickly. By training against Review Table’s data distribution and production requirements, we built a model that improves answer and citation quality while reducing the cost of serving each cell.

Separately, our agentic retrieval experiments showed that the way a model acquires context is another meaningful optimization lever that can substantially reduce token usage at matched answer quality. Harvey’s Review Table is an excellent example of where domain expertise can build upon open models to create capabilities that outperform more general-purpose models on specific, demanding workloads.