Advanced document review does not begin when reviewers open the first document. It begins much earlier, when legal and forensic teams decide what data to collect, how it will be processed, which information should enter the review population and what technology will be used to identify responsive material. A recent United States discovery decision provides a useful example.
In Schulte v LinkedIn Corporation, LinkedIn proposed using search terms to reduce its collected data before applying a generative AI review tool to make responsiveness decisions. The plaintiffs challenged parts of that process, but the court allowed the workflow to continue. The decision does not establish a universal approval for generative AI review. It does show that courts may focus less on whether advanced review technology is being used and more on whether the overall process is reasonable, proportionate and properly documented.
The Workflow Under Review
LinkedIn had 19 designated custodians whose information could potentially be relevant to the proceeding. The files of only two custodians totalled approximately 800 GB. Applying an advanced review tool across every item collected from all 19 custodians would have required multiple terabytes of data to be processed, hosted and reviewed.
LinkedIn instead proposed a staged approach:
- collect data from the agreed custodians and repositories;
- apply 25 search strings to create a smaller target population;
- load that population into Relativity aiR for Review;
- allow the tool to make initial responsiveness determinations; and
- conduct human quality-control review using samples from each responsiveness category.
The resulting target population contained 204,444 documents. The plaintiffs argued that using search terms before the AI review could exclude relevant information. They asked the court to require LinkedIn to apply the review tool across all custodial files and to disclose further performance information, including error rates and elusion estimates. The court denied those requests.
Processing Decisions Shape the Review
The decision highlights an important point: advanced review tools can only analyse the information provided to them. If relevant data is excluded during collection or early processing, a sophisticated review platform cannot recover it later. At the same time, loading every available file into review may create unnecessary cost, delay and duplication. A defensible processing workflow must balance completeness with proportionality.
Depending on the matter, early processing may include:
- removing system and application files;
- expanding container files and email archives;
- extracting text and metadata;
- applying optical character recognition to scanned material;
- identifying encrypted or unsupported files;
- deduplicating repeated documents;
- preserving email and attachment relationships;
- organising conversations and message threads; and
- applying agreed date, custodian or keyword criteria.
Each decision affects the population that eventually reaches reviewers and advanced analytics. The goal is not simply to make the dataset smaller. It is to reduce unnecessary material without losing information that could affect the matter.
Search Terms and AI Can Work Together
The court did not accept the argument that using search terms before advanced review was inherently inappropriate. The plaintiffs had not shown that LinkedIn’s 25 search strings were too narrow or otherwise deficient. The court found that using search terms to create a target population before applying technology-assisted review could satisfy the requirements of reasonableness and proportionality.
This is useful because search terms and AI review perform different functions. Search terms can quickly narrow obviously irrelevant material, particularly where the legal team understands the custodians, terminology and relevant period. Advanced review tools can then analyse the remaining population using broader concepts, context and patterns that simple keyword searching may not identify.
Neither method should be used without testing. Search terms may miss unusual language, abbreviations or coded references. AI tools may classify documents incorrectly or produce inconsistent results where the review criteria are unclear.
A combined workflow can be effective when each stage has a defined purpose and appropriate quality control.
Human Review Remains Part of the Process
LinkedIn advised the other parties that its AI tool would make final responsiveness calls, but that humans would review samples taken from each responsiveness category. That quality-control step was important.
Advanced review can reduce the number of documents that require manual examination, but the legal team remains responsible for the production. Human reviewers still need to test whether the system is applying the instructions correctly and whether potentially responsive documents are being excluded.
Quality control may include:
- reviewing samples of documents classified as responsive;
- reviewing samples classified as non-responsive;
- testing known important documents;
- comparing results across custodians and data sources;
- checking unusual file types and processing exceptions;
- examining documents near the decision boundary; and
- adjusting the review instructions where recurring errors are identified.
The correct method will depend on the size, risk and purpose of the review.
More Disclosure Is Not Always Required
LinkedIn disclosed that it was using Relativity aiR, that no seed or training set was required, that the tool would make responsiveness decisions and that human sampling would be used for quality control. The plaintiffs sought further information about the tool’s performance. The court declined to order that additional disclosure because the plaintiffs had not demonstrated a specific problem with LinkedIn’s production. The size of the review population alone was not enough to justify a deeper examination of the review process.
This does not mean legal teams can ignore validation metrics. It means they should understand and document their own methodology before it is challenged. Where a production gap or other deficiency emerges, the producing party may need to explain how the population was created, how the tool was instructed and how the results were checked.
What Legal Teams Should Do
Scope the data before processing
Identify the relevant custodians, systems, dates and communication channels before collecting everything available.
Record processing decisions
Document exclusions, deduplication settings, search criteria, processing exceptions and any files that could not be extracted or indexed.
Test early culling
Validate search terms and filters against known relevant documents and likely variations in language.
Define the AI review criteria
Clear, matter-specific instructions generally produce more useful results than broad requests to identify anything “relevant”.
Build in quality control
Use human review and sampling to test both included and excluded populations.
Preserve an audit trail
Retain the review instructions, search terms, population counts, processing reports and quality-control results.
Why This Matters in New Zealand
New Zealand litigation and investigation teams regularly face large collections from email, cloud storage, messaging platforms and business systems. The cost of a review is driven not only by the number of documents collected, but also by how effectively the data is processed and organised before review begins.
Advanced review tools can substantially reduce manual effort, but they work best within a structured process that combines forensic collection, defensible processing, targeted culling, AI or Technology-Assisted Review and human quality control. The technology is only one part of the workflow.
The Takeaway
The Schulte v LinkedIn decision provides a practical example of how modern eDiscovery workflows are being assessed.
The court did not require every collected file to be processed through the AI review tool. Nor did it require extensive disclosure of performance metrics without evidence of a deficiency.
The focus was on whether the workflow was reasonable and proportionate.
For legal teams, the key lesson is that defensibility depends on the complete process: what was collected, how it was processed, how the review population was created, how advanced tools were applied and how the results were checked.
How We Can Help
We help legal teams process, organise and review large volumes of electronic evidence.
This includes forensic data collection, eDiscovery processing, metadata extraction, deduplication, email threading, search-term testing, Technology-Assisted Review, AI-assisted review, review quality control and the preparation of defensible productions for litigation, investigations and regulatory matters.
Sources
Relativity, When the Court Doesn’t Blink: Schulte v LinkedIn on AI for Review
Relativity, aiR Assist for Early Case Insights: Faster, Grounded Answers
Relativity, Treating Linked Cloud Files Like Associated Files During eDiscovery
Forensic Tech 