Like many law firms and LegalTech businesses, since the explosion of large language models (LLMs) a few years ago, we’ve been working with generative AI (GenAI) to help lawyers produce a strong first draft faster – while maintaining the accuracy, consistency and rigorous quality control required in legal practice. Having recently integrated Clarilis AI Draft into our document automation suites, we wanted to share some practical lessons from the development process.
The three critical roles of a lawyer in developing an AI drafting tool
Despite the sophistication of LLMs, they can’t replace the nuanced thinking and professional judgment that lawyers bring to drafting. They suggest wording based on training data, not legal reasoning or an understanding of the client or wider commercial context. Consequently, GenAI tends to introduce the following risks in legal drafting (although expertly masked with a high degree of machine-generated confidence):
To address and mitigate these risks, we defined three essential tasks for legal experts in developing GenAI drafting tools.
Without expert legal input in these three key areas, AI drafting solutions (however advanced) risk becoming disconnected from the day-to-day challenges of producing legal documents.
1. Identifying the right problems to solve
Before rushing into empowering GenAI with any drafting responsibility, it’s critical to identify the precise input that’s needed and whether AI is the right tool for the job. For example, with AI Draft, our goal was to create an LLM-powered feature to help lawyers produce a review-ready first draft faster. At Clarilis, we already had rules-based automations that could do most of the heavy lifting when drafting the routine elements of legal documents. These are based on precedents crafted by experienced knowledge lawyers and offer a far superior starting point when compared to wording generated by an LLM. Using GenAI to produce this content would not only be unnecessary, it would also introduce inconsistency, error, and risk of hallucinated content.
However, where GenAI would be valuable is in offering a starting point for the elements of documents not covered by automated precedents. In other words, focusing the use of GenAI where there was no template or precedent for an automation to draw from.
2. Tailoring AI tools to specific areas of practice
When different practice areas ask for assistance from AI with producing a first draft, the format, tone, and content they expect in response can vary significantly.
In our exploration of the drafting potential of GenAI, we found it performed most effectively when provided with clear, high-quality instructions and examples of what ‘good’ looks like in a specific legal context. This relies on embedding practice-specific insight and testing into the development process – in other words, with legal experts working closely with product teams. Without this, even the most technically advanced GenAI tools will produce content that looks OK on the surface, but on closer inspection, lacks the precision and nuance required by the practice area in question.
3. Integrating AI tools into legal workflows
Even after taking the above precautions, legal content generated using GenAI must be reviewed and refined by a human lawyer for accuracy, quality, and client relevance. This is non-negotiable, no matter how impressive an AI-generated draft appears. However, if AI content is hard to distinguish from pre-approved content based on agreed precedents (automated or manually written), it can inadvertently make the review more cumbersome and time-consuming.
For example, one of the key benefits of Clarilis automations is that the consistency and reliability of the outputs significantly reduce review time. We were concerned that introducing AI-generated content could negate this advantage by creating uncertainty about which elements of a generated document need attention, and this uncertainty can materially increase the scope of review. To overcome this, we clearly mark where our AI Draft tool has contributed wording. This allows lawyers to target their attention on where professional judgment is needed rather than feeling compelled to recheck the entire document.
Conclusion: Human-guided parameters are critical to AI drafting
Our experience of developing AI Draft confirmed to us that effective use of GenAI in legal drafting doesn’t start and end with technology – it also relies on the insight of lawyers. As the legal profession continues to embrace the potential of LLM-generated content, we believe the most significant advances will come from combining technical innovation with deep legal expertise. To achieve this, we need to make the role of GenAI in drafting an evolving conversation – one we’re committed to continuing at Clarilis and are keen to invite you to contribute to.
Clarilis AI Draft is now available – visit the Clarilis website for more information.