At a time when there is a great thirst for knowledge about real world law firm GenAI projects and use cases, knowledge management conference SKILLS last week stepped up to the plate with a series of presentations and discussions around successful collaborations. What was interesting was the emphasis not just on the nature of the projects, but the successful relationships and service delivery behind them.
Founded by longstanding legal chief knowledge and innovation officer Oz Benamram while at White & Case, SKILLS started out as an exclusive in-person New York City event, but the online version – such as the one that took place on 16 January – has enabled organisers to expand its global reach and audience.
Presentations this year appeared thanks to both an internal committee and public vote. The talks included a look at how Paul Weiss has embedded Harvey across their business and, more specifically, is training lawyers to use GenAI with Harvey’s help; the importance of teaching attorneys the power of prompt engineering at Ballard Spahr; a look at the work Weil Gotshal is doing on AI-powered email summarisation with Syntheia; how DLA Piper’s KM team has partnered with Centari to extract data for deal intelligence with GenAI; and how GenAI can revolutionise the processing and exploitation of KM in law firms – this from Legal IT Insider’s lead analyst and longstanding legaltech consultant Neil Cameron.
The second half of the day looked at how Salesforce’s law department is working with legal statute and caselaw database vLex; a study from Legaltech Hub and Vals AI comparing AI with human lawyers; how Dechert is using AI to benchmark suitable tools; Cleary Gottlieb’s AI and transparency awareness AI pilot tracker; the work Reed Smith is doing with Litera to centralise its data; and how Fasken achieved a breakthrough innovation working with Lega, which enables law firms and other organisations to safely explore GenAI technologies.
We’ve picked three presentations to share with you in some detail below, but you can watch each of them on YouTube, see the links at the bottom. The day also started with a higher-level opening panel on adopting AI in a law firm, led by Katie DeBord from Disco, and you can watch that here: https://www.youtube.com/watch?v=Lj5nA5-PVWI
Paul Weiss’s GenAI engagement program
When GenAI emerged in earnest two years ago, it disrupted Paul Weiss’s knowledge management strategy.
While there were big questions around the likes of use cases and risk, getting tools into the hands of lawyers at the Am Law top 30 law firm was a priority, in order to start educating them on new ways of working.
In a SKILLS presentation, Iris Skornicki, Elizabeth Wilkinson and Mike Ross from Paul Weiss described how the knowledge team initially relied on demos and developing power users to train users, but lawyers found it hard to apply the technology effectively afterwards, and used GenAI for tasks it wasn’t designed for, such as asking legal research or about current events.
It became clear the firm needed a scalable solution that actively engaged users in hands-on learning.
The turning point or epiphany came with a summer associate hackathon – by ‘doing’ rather than ‘watching’, participants internalise best practice. Paul Weiss took elements of their initial training and combined them with interactive elements of the hackathon.
The first step is an exercise with a group leader such as saving a prompt to your library, so you can reuse it. There is then an element of gamification and fun exercises followed by a mock assignment and discussion. Harvey has been working with an external provider to develop the training, with Ross observing that they have “gone above and beyond with us”, adding: “From a customer success standpoint all law firms need to push for this from our vendors, let them know it is a must. Vendors throwing features over the fence without any guidance on how to use them is really a recipe for stifling innovation in the long run.”
Paul Weiss says the early feedback has been “emphatic.” Attorneys are having fun while developing skills and learning far more than through dry material training. They are also learning more about the tools and their limitations.
Presenting to SKILLS from Harvey, Jake Weiner (pictured above) said: “These successful outcomes arose from several key principles that the Paul Weiss team discovered through developing the series and that we at Harvey have adopted broadly in our training and enablement process.
“First, we’ve learned that practical engagement is non-negotiable. So rather than walking through slides or watching demonstrations, our workshops immerse participants in hands-on exercises within the first few minutes.
“Second, we’ve embraced failure as an essential teacher. So by creating a safe space for experimentation, lawyers discover not just what AI can do, but also what it can’t. The strengths and weaknesses of AI tools are often surprising and non-intuitive. And this understanding is crucial for developing sound professional judgment about when and how to leverage AI tools effectively.
“Third, we’ve learned to proactively address the full spectrum of perspectives in our workshops, encouraging power users and skeptics to share the same training space. We’re not presenting AI tools like Harvey as panacea or complete solution. Instead, we frame them as the beginning of a journey. This transparency about both capabilities and limitations helps build trust with skeptics while keeping enthusiasm grounded in reality. It’s not about whether to use AI, but rather about understanding how to get the most value from these technologies as they currently exist. Finally, we’ve recognized that continuous learning is essential to reinforce skills and encourage sustained AI adoption. So our workshops aren’t one and done events, but rather they’re part of an ongoing conversation about integrating AI into day-to-day legal practice.”
Lega and Fasken – Presentation winner
The second collaboration we want to highlight won the vote for best presentation at SKILLS. Rachelle Thompson, senior director of practice and client innovation at Fasken (pictured below) talked about the work that the Canadian law firm is doing with Lega, founded by CEO Christian Lang, who presented alongside Thompson.
Much like other firms, Fasken had scrambled to find a secure alternative to using public GenAI tools and Thompson says that Lega was instrumental in helping them to implement a safe GenAI chat function for all users. “We didn’t have to build it ourselves which freed up our innovation team to focus on bespoke needs,” she says. “Additionally, Lega enabled us to leverage our Azure OpenAI license within our own Microsoft tenant and gave us the flexibility to use any other models we choose to in the future.”
As Fasken’s needs evolved, the firm began to expand the way they use Lega, for example, they have created a number of apps that perform specific tasks, such as document comparison. They also created a prompt library, which is the specific project at the centre of this presentation.
Thompson said: “While the level of prompting expertise varies across our users, they still expect consistent output, which we all know can be influenced by a single word that could be different than the prompt. Also, the individual approach could not meet all of our user needs and as we conducted outreach and gathered feedback on tools, one thing became clear: everyone wanted a prompt library. So, we returned to Lega to see if it would be possible to embed this approach right in the application.
“We explained the requirements for the prompt library and shared some initial ideas. They came back with an innovative solution that exceeded our expectations and an integrated system that not only provided a prompt library but also supported the creation of prompts with variables.”
Before Thompson handed over to Lang for a deeper dive from a vendor perspective, she shared her observation on partnering with vendors, commenting: “While it is worth investing in that exchange of ideas and expressing clearly our requirements [to vendors], keep in mind that we shouldn’t come to them with the final solution. We can make suggestions on the functionality and what we would like, but let the experts go away and then come back with the proposed solution. Because my lesson learned in this specific interaction was just that Lega came back with a bigger and better solution than I had envisioned.”
Lang (pictured below) echoed the message that collaboration is key, commenting: “We need to explore because we don’t know where we’re going yet; the models and capabilities are changing by the minute so we can’t just build by for a known set of cases – we need hands on experimentation at scale to learn and to test for value.”
The prompt library idea arose initially to fix a specific problem. Lang says: “While many users could just take the ball and run with it, others experienced – in a phrase coined by the Fasken that I absolutely love – ‘blank prompt syndrome’; they didn’t know what to do, and that led to an insight that many users need more help.
“As a starting point, as Rachelle shared, the Fasken team requested that we helped them save and share expertly refined prompts. We took that feature request from Fasken and had insight discovery conversations with our other customers to develop a holistic understanding of what our customers needed. We realised that not only could we give Fasken exactly what they were asking for, but with a little innovative product design, we could actually go one step further. Instead of just giving users static prompts that require constant manual tweaking when they’re used, we could empower users to build dynamic prompts that contain configurable parameters or variables to respond to context. In essence, that would give every user their own little innovation lab and what’s more, for those suffers of blank prompt syndrome, we realised that we could configure apps to automatically execute those dynamic prompts when the app is launched.”
Instead of users being met with a flashing cursor in a chat input field, Lang says: “We can present them with a simple form-based UI that asks them for a couple of inputs and lets them click a few buttons to get immediate value.” He added: “It’s this kind of collaborative innovation that ensures we’re not just keeping up with change, but we actively leading and shaping the future in ways that truly matter.”
Using GenAI to manage write offs
Unlike other talks that focus on existing projects, Neil Cameron’s talk centred on a concept that he has been working on for a decade, but that GenAI has potentially blown wide open: how to help lawyers create more accurate estimates of work so they don’t have to write off so much time on a matter. This is both a practice management issue but also KM as there is a heavy work categorisation element involved.
A few years ago, Cameron (pictured above) conducted a pilot with Ravn (as was then, now part of iManage) and a UK top 100 law firm. He says: “The original principle was that lawyers are bad at estimating how long work will take and what it will cost. According to regulatory frameworks in both the US and UK, lawyers have to give an idea not just of the basis of charges but how much work is likely to cost, but they almost always get it wrong and underestimate how long it will take.”
Cameron observes that if lawyers give a client an estimate of £1m but the work comes in at £1.3m, the lawyer is likely to either have an awkward conversation and get the client to pay £1.1m, or write it off, or both. This is why most law firms operate at between 78-84% realisation.
“Lawyers have been living with this for years and assume it’s as good as it gets. I happened upon research by Israeli psychologists Kahneman and Tversky, who identified the key conceptual biases that make it so difficult to gauge the cost and length of projects, whether that be the simplest domestic bathroom modelling job to the largest projects planned by the world’s leading experts, such as the F-35 fighter program, the Channel Tunnel, the Sydney Harbour Opera House and the infamous Boston Dig.
“After years of study, Kahneman and Tversky came up with a proposal which adopts the principle of of ‘reference class forecasting’. You simply analyse the behaviour on previous similar projects – the reference class – work out just how bad you are at planning as a proportion of the whole, and then add that proportion onto any future forecasts. It sounds easy. And it got them a Nobel Prize in 2002. The problem is finding the reference class.”
The issue at law firms is that even where lawyers are supposed to categorise a piece of work by a considered estimate of how much they expect it to cost at the outset, they typically don’t. In Cameron’s previous M&A pilot project, they were still able to identify a number of reference factors such as how many businesses you’re buying; how many lenders; how many employees; how many documents, etc and Cameron says: “It adds up to a fingerprint that you can run through with a client in order to agree the cost assumptions and then if it turns out there are more documents or lenders you can go back for another much more easy conversation.”
The previous project fell through, but with the advent of GenAI, Cameron says: “You don’t need the accurate lawyer-based data, you can send a GenAI crawler to look every data point, every narrative, every document and every invoice and it can tell you why this matter is more, or less, expensive and where it went wrong last time.”
Cameron is looking to continue the conversation, so if you’re interested message him here.
Conclusion
Summing up the day to Legal IT Insider, SKILLS organising committee member Nick Pryor (pictured below), who is director of knowledge and innovation at Freeths, said: “SKILLS has always been a community-led event; we are grateful for all the contributors who were willing to share their insight and perspective with their peers. It is particularly invaluable at a time when our industry is deep in the weeds exploring and learning how to harness Generative AI to drive KM goals and deliver business value. The legal industry has never been more open to change, and the results of the SKILLS survey (and presentations at the conference) provide compelling evidence of the momentum behind KM as an increasingly important component of a law firm’s value proposition.”
To watch any of the presentations from the day go to https://skills.law/2025