UK top 20 law firm Ashurst today (10 June) publishes a behind-the-scenes look at its trials of three different generative AI platforms, focusing on the qualitative and quantitative data from its evaluations. The in-depth study, which does not reveal the specific technologies that Ashurst evaluated, takes a look at some of the benefits of gen AI and provides guidance on how best to support people in their future adoption.
The trials, which involved over 400 people spanning 23 offices and 14 countries, evaluated the use of a number of gen AI-powered tools, measuring their potential value and future impact in a legal context, as well as the best methods for navigating the exploration of gen AI.
The greatest initial value of gen AI for the legal sector was found to be helping lawyers to create drafts, where the firm measured average time savings of 45% on creating first draft legal briefings increasing to 80% to draft corporate filings requiring review and extraction of information from articles of association. Translated into typical time spent, participants saved 2.5 hours per briefing draft – tasks that would normally take anywhere between a few hours to a few days to prepare. Ashurst concludes that producing first drafts in this way not only creates the opportunity for people to focus on higher-value work, but may potentially also lead to an increase in the speed that legal services are delivered.
Ashurst conducted some interesting tests to see if an expert panel could tell if outputs where AI or human generated. In a blind study, the panel correctly identified all outputs made by the firm’s lawyers as human-generated. While half of the gen AI output was correctly identified as such, the other half was either mis-identified as human-produced or could not be determined by the panel.
Ashurst says that the value uncovered for gen AI during the trials was far broader than originally anticipated, as reflected by the 61% of post-trial survey respondents who felt that using gen AI would help them feel more supported in managing their workload. The research also showed that delivering and embedding a gen AI capability is about more than just saving time, it is also about preparing people to meet and stay ahead of market demands: rhe vast majority of respondents (88%) at the end of the biggest trial said that using gen AI technology helped them to feel more prepared for the future.
Tara Waters, Ashurst‘s chief digital officer, commented: “Generative AI offers huge potential gains, not only in terms of efficiency but also new value creation opportunities, for both law firms and our clients. Our experiments with gen AI-powered tools have helped us to assess the benefits and challenges that come with this technology and to plan how we can best use generative AI to deliver solutions for our clients.
“The legal industry is only at the start of its exploration journey in the adoption of advanced AI technologies, and this report has attempted to lift the lid on our own journey and findings so far. There are lots of possibilities this new technology offers and much more testing to be done. It is now vital that moving forward we embrace sharing and collaborating in this way, as we all navigate the noise around gen AI and promote meaningful, beneficial change in the legal industry.”
You can download the report, which includes a range of other findings, here in full: https://www.ashurst.com/en/insights/vox-populai-lessons-from-a-global-law-firms-exploration-of-generative-ai/