Springbok AI has launched a hallucination detection and referencing layer integrated into its no-code GenAI suite SpringLaw, we can reveal, in a major release for the UK legal product development boutique.
SpringLaw, released in June, is an LLM-agnostic solution that enables law firms and legal professionals to configure tools to their specific workflows and build out practice toolkits. The new release, SpringLaw Verify, now provides line-by-line referencing to source documents, identifying sentences that are not from the document and need to be revised, plus enabling users to automatically remove or redraft the sentence.
In a demo to Legal IT Insider, Springbok’s commercial director Edward Stevens uploaded a data processing agreement into the SpringLaw Chat interface, asking, ‘What are the obligations placed on the data processor regarding security measures?’ Where the output is green, you can hover over it, and it will show you the supporting references. You can click on those references, and it will take you to the document.
Stevens said: “The difference here is that there is a statement in the middle of all this which we have highlighted in yellow, and this statement has no reference within the document itself.”
In most cases, Stevens says, what’s included is information that is common knowledge or something in the training data for the LLM – in the example given, that processors must comply with Article 32 of GDPR. Leveraging their own knowledge, lawyers can leave the sentence in as something that is factually true, or, if the output must only come from the source document, remove it.
An ‘improve’ button in the top right enables the user to refresh the content and rerun just the highlighted statement, removing it if it does not come directly from the document.
There are occasional times when a statement is factually inaccurate and there is a contradictory reference in the document, in which case Verify highlights the sentence in red and flags the relevant contradictory reference for you to look at. In the example given by Stevens, a first pass reply from the LLM says, ‘Yes, the Supplier is held liable for security risks caused by its software.” Within the document, there is a paragraph to say the supplier shall not be liable. Stevens says: “I can click ‘improve’ and what it’s then able to do is use the new bits of information to redraft the sentence for me, to say ‘No the supplier is not held liable’ and now I can be confident in the answer.”
What is particularly significant is that within other tools you may have the original document source highlighted but if there is a sentence within a block of referenced text that is not supported, it can be missed.
Stevens said that in in side-by-side comparisons with earlier client-prepared documents, SpringLaw Verify has consistently outperformed, accurately identifying discrepancies 95% of the time.
This release will be interesting to firms that are concerned about the non-billable time that it is taking for associates to double check the output of LLM-generated content, and the lack of trust that fee-earners have in the output, which makes change management a more difficult process.
Steven says: “We’re getting closer to being able to hit the level of a human. I think anyone who says they’ll be 100% accurate is living in a different world but it will be, I believe, more than 95% of your output, maybe more will be accurate, and it will have been highlighted to you. And I think the main thing is not even about accuracy points at this level, it’s about showing the users what the output looks like, where something’s coming from an LLM, where something is coming from your document, getting that understanding as to how answers are being produced and then going, ah, okay, I can improve it, I can make the assessment, I’m in control now of this verifying process and I can do it in the platform.”
Over the coming weeks, select existing SpringLaw clients will be live with SpringLaw Verify, with a general release coming in January.
You can sign up for a free 14 day trial of SpringLaw at www.springbok.ai
Springbok works with firms including Linklaters and Charles Russell Speechlys, see below.