Osborne Clarke this week chucked the cat among the Gen AI pigeons by revealing the results of some testing the UK top 20 law firm has done in its own Azure environment of GPT-4o, saying that it has matched or surpassed Harvey’s outcomes.
AI engineer Christian Braun posted on LinkedIn that following Harvey’s publication of its BigLaw Bench – a framework for evaluating the performance of large language models on complex tasks – OC put it to the test using OC-GPT.
Braun says that:
– OC used Harvey’s published test cases.
– Each case was prompted with one-shot and with various prompt engineering techniques (e.g. Chain-of-Thought, Few-Shots).
– Hallucinations were excluded from the evaluation.
– The evaluation was based on LLM responses to Harvey’s test case questions.
– OC also considered the task weighting from Harvey’s BigLaw Bench publication.
“The results? Astonishing! Not only did we match Harvey’s outcomes, but in some cases, we even surpassed them,” said Braun. It may amuse you that one head of legal technology at a large law firm immediately messaged Legal IT Insider privately, commenting, “Pot stirrer!”
The point, Braun says, is that empowering all law firm staff to harness AI effectively is key. Whether it’s through well-crafted prompts tailored for specific use cases or by teaching prompt engineering skills, the potential is immense.
“Now, here’s a thought,” he said. “It would be fascinating to see if and how much better well-crafted prompts on Harvey perform compared to our results on plain GPT4o. Has anyone explored this yet?” Answers to newsroom@legaltechnology.com
Neil Cameron, a legal technology consultant and lead analyst for Legal IT Insider, commented on the post: “All good stuff; the more we mess about and share, the better. On balance though, I would love to have a full paper as well as be pointed to raw data – and this has prompted many of these questions below. I’m sure we’d all like to see the optimised prompts, as well as any intermediate ones.”
That post in full is here
Elsewhere this week international law and arbitration research platform Jus Mundi announced that it has secured €20m in Series B funding – one of the largest rounds of funding for a European legal tech company. The round was led by Germany-based Acton Capital and Singapore-based True Global Ventures, with participation from C4 Ventures in France and FJ Labs in the US. The funding will support R&D efforts to further develop Jus Mundi’s AI and expand its geographical presence.
Jus Mundi also announced that it will launch Jus AI, a research and productivity assistant for global legal professionals, at IBA Mexico on September 16. Jus AI has been piloted for several months, with more than 20 clients participating in a Jus AI Acceleration Program; they include DLA Piper, Reed Smith, Clyde & Co, Greenberg Traurig,Yulchon and Bredin Prat.
“Jus AI has significantly enhanced our efficiency, providing invaluable access to summaries of extensive decisions. It’s particularly useful when researching specific topics. By swiftly compiling relevant decisions, Jus AI has optimized our research, enabling us to focus more on strategic deliberation,” said Rebeca Mosquera, senior associate for international arbitration at Reed Smith.
For additional information about Jus AI, visithttps://jusmundi.com/en/jus-ai.
Industry hire
ContractPodAi has hired Tanja Podinic as SVP of AI Programs. Podinic’s previous roles include generative AI lead for global legal business solutions at PwC and global director of innovation programs at Dentons. This is the first vendor role for Podinic, who was previously a conflicts lawyer in Sydney.
Podinic said: “Throughout my career, I’ve had the opportunity to collaborate with several AI tech companies, and what stands out about ContractPodAI is its deep integration of generative AI at the core of its Leah platform. Unlike other solutions that tack on AI as an afterthought, Leah’s generative AI capabilities are embedded into the fabric of the product. In addition, the solution is intuitive and seamlessly sits alongside the daily workflows of legal professionals. It supports them to work smarter, faster and make decisions with greater confidence.”
ContractPodAi supports customers globally across North America, Europe and the APAC region, with a presence in each region. A top priority for ContractPodAi is furthering its expansion into the Middle East and APAC including Saudi Arabia, Egypt and UAE. Over the last year, the company has seen a 139% growth in the APAC region.
Product Launches
In the US this week data risk management and eDiscovery provider Exterro introduced a new request management module to help eDiscovery professionals automate critical process workflows.
Configurable templates and customized automation rules enable teams to track and execute on collection, matter intake, preservation requests, data processing, legal holds, and more. The new module includes real-time dashboards and advanced reporting options.
“Request Management gives legal teams a project management solution that was purpose-built for e-discovery,” said Brad Harris, vice president of eDiscovery solutions at Exterro. “E-Discovery is both detail-oriented and time-sensitive, so teams need trustworthy, efficient technology that supports them while they manage multiple, complex legal matters at the same time. Request Management helps ensure that projects stay on track and that teams don’t make mistakes that run afoul of courts’ requirements.”
Also in the US SimplyConvert has launched CaseHQ, a client portal and mobile app designed to centralise client-related communications, activities, and data management into one platform. From the initial intake to case resolution, it stores all information as structured data.
“CaseHQ is more than just a tool—it’s a transformative solution that redefines how law firms interact with their clients,” said Marc Stern, chief growth officer at SimplyConvert. “We’ve designed CaseHQ to address the real-world challenges that attorneys face every day, from managing client expectations to keeping cases on track. With CaseHQ, we’re helping law firms build stronger, more transparent relationships with their clients.”
For more information about CaseHQ visit https://simplyconvert.com/case-hq-client-portal.
And California-headquartered practice management system CARET Legal has launched CARET Analytics, a data analytics platform designed to provide law firms with actionable insights and transform complex data into user-friendly dashboards and reports.
“Law firms need a powerful tool to focus their energy on strategic planning, not spreadsheets,” said Jennifer Sherman, COO at CARET. “CARET Analytics provides comprehensive yet easy-to-understand actionable solutions for law firms to improve their efficiency and achieve sustainable growth.”
“As a CARET Legal customer, we are thrilled about the insights that CARET Analytics will surface to drive better decision-making at our firm,” shares Joshua R. Driskell, managing partner at Lagerlof, LLP. “Armed with this information, our management team will be able to provide greater transparency into firm operations and performance to enable our future success.”
For all our other news and hires make sure you are checking out our website daily and sign up for news to be delivered to your inbox HERE.