An academic paper published on 30 December called ‘Can Robots be Lawyers? Computers, Lawyers and the Practice of Law’ argues that the impact of automation on the demand for lawyers’ time is far less significant than suggested to date.
Taking on arguments put forward sometimes hysterically in the media and in a measured way by the likes of Richard and Daniel Susskind, Professor Dana Remus from the University of North Carolina School of Law and Professor Frank Levy from Massachusetts Institute of Technology examine in detail the suggestion that technology will soon replace much of the work currently done by lawyers, particularly junior ones.
While Professors Remus and Levy don’t deny that computers are changing the way law is practiced, in a 67-page paper they find significant weaknesses in the above argument and instead argue that computers are changing, not replacing the work of lawyers.
Skipping here an explanation of the basics of machine intelligence provided by Professors Remus and Levy (including a look at structured v. unstructured data and its potential for automation – it’s worth a look), the paper looks at the potential for current or near-term automation of six categories of lawyering tasks – document and case management; document review; document preparation; legal research and reasoning; interpersonal communication and interaction; and courtroom appearances. They then go on to consider potential employment effects, mapping automation against the tasks actually performed by lawyers, using data provided by Huron Legal’s consulting arm, Sky Analytics.
Document and case management
Document and case management has been successfully automated for years in terms of sorting and searching for files as well as automated templates, billing and other productivity tasks that used to be performed by clerical personnel. In contract management, Professors Remus and Levy highlight how new data-driven applications such as ContractAssistant are automating far less structured tasks, while KM Standards extracts common contract provisions and creates a basic template, going on to highlight discrepancies between the template and contracts proposed by other parties. In case management, structured tasks such as billing and docketing have been automated, while unstructured tasks, such as monitoring junior lawyers’ work and dealing with parties who fail to honour contractual obligations “require unstructured human interaction of a kind the computers cannot replace.”
Note – here Professors Remus and Levy seem to suggest that assigning work to lawyers is an unstructured task beyond the current capacity of computers, however Riverview Law’s CliXLEX platform, for one, manages activity including triage.
While most document review is too complex for box-ticking deductive reasoning, by feeding machines multiple samples of documents and outcomes it has been possible for programmers to give computers data driven instructions, leading to predictive coding, which can replace human review.
The software can quickly scan millions of documents and classify them as relevant or irrelevant by comparing them against sample documents, which can be expanded and altered until lawyers are satisfied with the results,
While the use of predictive coding is on the increase, Professors Remus and Levy highlight its limitations, including the fact that experienced lawyers must still classify the training sample of documents and train the system’s parameters, meaning sizeable up-front costs that will only likely be justified in large cases. Predictive coding is also less effective in contexts such as due diligence, where information is less structured, more diverse and may appear in limited volume.
Here the authors distinguish between legal writing and document drafting, the latter of which is more structured and is increasingly being automated, including by online providers such as LegalZoom and Rocket Lawyer.
“Although highly effective for basic and standardized legal documents, online document drafting is far more limited with respect to more complex and novel documents,” they point out, flagging the risk that programs will fail to anticipate a contingency and create an error, leading to legal conflict.
Legal writing, on the other hand, has largely resisted automation. While a computer can write about a structured game such as baseball (leading to the argument that this can be extended to legal writing) Professors Remus and Levy say: “..the articulation and explanation of an argument is the product of conceptual creativity and flexibility that computers cannot exhibit,” adding, “A single case can be used to support two opposing positions…”
Legal Research and Reasoning
Ever since IBM Watson won Jeopardy, commentators have argued that it will surely be able to answer legal questions (and it is in the process of developing Ross Intelligence to do just that). However, Professors Remus and Levy say these claims are premature.
While questions requiring a factoid often result in the right answer, more complex queries often (when put to the likes of Siri, for example) return a collection of various passages. Computers also struggle with concepts that rely on synonyms, hyponyms and analogies or other subtle uses of language. Furthermore, if a database is too broad, it will return irrelevant matters and too narrow, it will exclude potential answers.
For the foreseeable future, this approach will require significant human time and energy and will be expensive. (Here, at p.24/25 Professors Remus and Levy include an interesting summary of Watson’s answering architecture.) Designing the search parameters, interpreting the results and advising the clients will require significant human intervention.
Interpersonal communication and interaction
A fifth category of work encompasses interpersonal interactions including communication with the client/other side and factual investigation and negotiation.
While computers have made headway in prediction, clients are looking for more than statistical probabilities and lawyers are required to make assessments based on emotional intelligence and an understanding of a client’s situation, goals and interests.
While some parts of fact investigations can be automated (including pulling together information about a client or opponent), much of it relies on interviews where “significant amounts of information may be transmitted nonverbally, in ways a computer would have difficulty detecting.”
Negotiation, meanwhile, often requires personal interaction and the use of emotion. Here, however, Professors Remus and Levy flag the strides being made by the likes of online dispute resolution provider Modria, which gathers information, summarises areas of agreement and disagreement and makes suggestions for resolving the dispute, rendering negotiation unnecessary.
While Professors Remus and Levy say a little more than this, it’s fair to say even the most fervent technology advocates are not predicting near-term automation of courtroom advocacy.
Here the paper looks at data on lawyer time usage to test the claim that automation is displacing lawyer labour, particularly among junior lawyers.
The data is based on an aggregation and analysis of invoices billed by law firms, provided by Sky Analytics (the authors note the data provides no information on the work patterns of sole practitioners or contract lawyers, among other limitations.) The table of findings are set out below:
Table 1: Percent of Invoiced Hours Spent on Various Tasks, Grouped by Estimated Extent of Computer Penetration
|Task||Tier One Firms||Tiers Two – Five Firms|
|Strong Employment Effects||4.1%||3.6%|
|Moderate Employment Effects||39.7%||40.4%|
|Case Administration and Management||3.7%||5.6%|
|Legal Analysis and Strategy||28.5%||27.0%|
|Light Employment Effects||56.0%||55.7%|
|Court Appearances and Preparation||13.9%||14.5%|
** Percentages may not sum to 100% due to rounding.
Table 2: Distribution of Time on Tasks by Tenure in Tier One Firms
</= 2 Years
|All Partners||Tier One Total|
|Strong Employment Effects||8.5%||4.5%||1.1%||4.1%|
|Moderate Employment Effects||34.9%||38.5%||44.7%||39.0%|
|Case Administration and Management||3.4%||2.4%||6.0%||3.7%|
|Legal Analysis and Strategy||23.5%||28.7%||31.1%||28.5%|
|Light Employment Effects||56.3%||56.6%||54.2%||56.0%|
|Communications and Interactions||9.0%||11.1%||5.1%||8.8%|
|Court/Official Appearances and Preparation||12.0%||14.7%||13.8%||13.9%|
|Addendum: % of all Hours Billed by Tenure||18.0%||50.0%||32.0%||100.0%|
** Percentages may not sum to 100% due to rounding.
It is worth reading the paper for a more detailed explanation of these findings but Table 2 shows that the correlation between machine complexity and seniority refuses to fit neatly into a pattern and Professors Remus and Levy say: “The factor that undermines a simple relationship between machine complexity and role within a firm is unstructured human interaction, a skill that has so far resisted automation but that is a part of lawyering tasks at every level.”
Arguing for a shift rather than reduction of labour, Professors Remus and Levy also say: “By many estimates, 75% of civil legal need in the country goes unmet. The automation of lawyering tasks may address this latent market rather than replacing existing lawyer labor. Alternatively it may push lawyers to serve this latent market as a means of finding new work.”
In the final part of their paper, ‘Computers, Professionalism and the Rule of Law’, Professors Remus and Levy address how technology is changing rather than replacing the work of lawyers, how the profession should be addressing and regulating those changes, and what light those changes shed on the value of organising lawyers as a profession.
It’s outside of the direct impact of automation on employment but well worth a read. In conclusion, the paper finds: “Certainly automation is having a significant impact on the labor market for lawyers and that impact will increase over time, but predictions of imminent and widespread displacement of lawyers are premature. A careful look at existing and emerging technologies reveals that it is only relatively structured and repetitive tasks that can currently be automated. These tasks represent a relatively modest percentage of lawyers’ billable hours.
“We have also argued that the existing literature focusses too narrowly on employment impacts, ignoring an important set of broader questions. The broader inquiry starts with the ways in which computers approach particular task sets differently than humans, and then asks how those differences may change legal practice and through it, the law itself.”