International law firm CMS announced yesterday (27 March) that it has selected generative AI vendor Harvey, as part of a portfolio approach that is seeing the UK top 50 law firm build out its experience of multiple gen AI systems.
Harvey, which is built on OpenAI’s GPT large language models, can analyse contracts, assist with due diligence, produce insights from various documents, assist in creating initial drafts, and create forecasts from large amounts of data.
CMS began a pilot programme with Harvey in the summer of 2023, involving a large number of CMS lawyers, tax advisors and notaries across several jurisdictions. The firm will now introduce Harvey in a phased approach across its member firms, starting in France, Germany, the Netherlands, Portugal and the UK, where it has the most licenses.
The project was run by Edinburgh-based director of innovation, John Craske; director of information technology & innovation at CMS Netherlands, Bert Vries; and London-based head of legal technology, Dominic Judge.
Speaking to Legal IT Insider, Craske explained that during the testing phase, the selection team asked CMS practice groups to think about how they might use Harvey. He said: “Broadly, Harvey gives you a document Q&A feature and that’s the strongest use case that we saw in testing: you’ve got a document you want to understand, that you want to review, and you can ask a whole load of questions or maybe specific questions around the likes of ‘when can I terminate this?’ So that’s really useful in Harvey and more and more often it gets it pretty much right.” He adds: “Because you’re giving it the document, it doesn’t hallucinate so much because it’s just the OpenAI GPT model underneath Harvey.”
Harvey has not been trained with legal data but the San Francisco-headquartered company has tuned it, and Craske says: “You get more legal sounding responses and longer answers. So it’s better.”
While Craske says you wouldn’t use Harvey to draft an entire agreement yet, it is used for drafting clauses and he adds: “We’ve seen good use cases around summarising stuff, whether that’s case judgments or agreements, and it’s customised for different practice groups.”
Harvey doesn’t yet integrate with core systems and so the process of using it is manual – you upload documents into Harvey – although Harvey will soon release a data room integration. Craske observes that using iManage Drive for review provides a front-end integration that avoids the security challenges around a back-end integration.
There remain inevitable questions around whether Harvey will use law firm data to train its model but Craske says: “They have a one of the best security information packs I think I’ve ever seen to pre-answer questions. You get these questionnaires with a bazillion questions that banks send to law firms and Harvey provides a huge pack where they have pre-answered those sorts of questions in there. So, of course, then you might go back and query a few things, but we got to those points quickly because the pack was pretty good.”
CMS is one of the firms that has trialled and is taking out 300 licenses with Microsoft Copilot and while there is some overlap in functionality such as summarisation with Harvey, Craske says they are very much not the same thing. The firm is, he says, taking a portfolio approach to gen AI products, observing: “I’ve been talking to the management team and our board and the view is that it is too soon to make one big bet on one big thing. Yes, you could ask Copilot to summarise some things but it’s mostly really good at helping you work across the Office 365 suite and navigate your day-to-day life, summarising meetings and finding out who sent what emails. But because of the way Microsoft has tuned it, it often gives you shorter answers than Harvey gives you. So what we want to do is try a load of different things. I don’t mean try as in five or 10 licenses, I mean properly try and measure and understand which roles, which tasks and which things the right tools are good at. Then we’re going to make some decisions about which ones to pick because there’s such a mosaic of things around.”
He adds: “We have Harvey and we have Copilot. We have already come out as saying that we are on the Lexis+ commercial preview. And then all of those other existing technology vendors are integrating gen AI into their solutions, not to mention the things you might build yourself. I can see that we are going to have a portfolio of three, four, five things and we’re going to try them and make a series of small bets rather than one big bet.”
One of the major enduring concerns for firms is the additional cost that obtaining new licenses represents. Copilot, for example, is $30 per use per month. Craske says that a portfolio approach make this more palatable, saying: “It also means that it gives you an opportunity to work out where the value is. If you want to expand or reduce the number of licenses, you can, and you can target the places where there’s the most value. There’s an importance of having the right sort of oversight and governance of what you’re doing and how you’re doing it, and the right buy in from the board, because you can’t just buy all the tools.”
CMS is aiming to take a very considered approach but by that Craske says: “I don’t mean slow, actually the opposite. Whilst we are convinced that all of the tools we are working with will prove benefits, we want to understand that in detail. That’s what we’re measuring as part of the process. For example, we’re getting people to do the same tasks with and without Copilot and measuring the difference.”
Other legal organisations that have partnered with Harvey include Allen & Overy and Macfarlanes, plus PwC Tax and Legal Services. To find out more or join the waitlist for early access see https://www.harvey.ai/