Categories
Comment, Opinion & Guest Articles Latest News Law Firms

Chats with legal chatbots – episode 1: The Global Legal Hackathon and LawDroid

“The big thing in 2018 will be for law firms to start introducing chatbots onto their websites never mind getting actual content on there first for the chatbot to reference.” In his first ‘chat with chatbots’, Brian Inkster puts LawDroid through its paces. And gives up.

By BRIAN INKSTER

Chats with Legal Chatbots

Even now we have gone digital you can still wind up a robot!

This is a series I have had in mind for a little while since I experimented, a few months ago, having chats with some legal chatbots. Apparently chatbots are the next big thing.

In my Legal Technology/IT Predictions for 2018 I blogged:-

The big thing in 2018 will be for law firms to start introducing chatbots onto their websites never mind getting actual content on there first for the chatbot to reference.

The fact that introducing a basic search function to their website might be more effective and useful than a chatbot that gets confused with most queries put to it (unless there is a human being operating it) will be lost on most lawyers.

However, if it has not been done already, a legal chatbot will in 2018 be able to order your Uber for you. This will be major news in legal publications but not really a giant leap for the legal profession.

So I thought I would test out a chatbot or two. First up is LawDroid, the chatbot used by the Global Legal Hackathon. I tried to have a conversation with it at the time the Global Legal Hackathon was in full swing and before writing my blog posts linked to or about that eventLawyers and codingHack the Law to Reinvent the Wheel? and London Legal Hackathon bend the rules?

For the uninitiated it appears to be the norm and perhaps expected (at least in the UK) at a hackathon that the participants will be supplied with pizza. So I thought I would ask LawDroid about pizza.

This is how our chat went:-

LawDroid: Hi [hand symbol]! I’m LawDroid.

What’s your name?

Me: Bond, James Bond.

LawDroid: Hi James Bond!

I’m programmed to provide helpful information for Global Legal Hackathon hosts and attendees.

If you ever get stuck, don’t worry, you can type in the word ‘restart’ to start over. Or, type ‘end’ to stop.

Shall we get started?

Me: Yes

Law Droid: Would you like to learn more about GLH’s Organizers?

Me: No.

LawDroid: Will you be attending or hosting the Global Legal Hackathon in your location?

[presented with Attending or Hosting option]

I was doing neither but selected Attending.

LawDroid: Here are some FAQs for attendees:

What is the Global Legal Hackathon? What is a hackathon? How do I form a team? etc

Would you like to try another option?

Host Info – Attendee Info – Ask a Question

Me: Ask a Question

LawDroid: What issue do you need help with?

Me: Will there be pizza?

LawDroid: So we can follow up with you about your concern, I’ll need a couple of pieces of information.

What’s your email address?

Me: I don’t want you to e-mail me I want you to chat.

LawDroid: Sorry, I didn’t quite get that

Me: Can we continue our chat?

LawDroid: Sorry, I didn’t quite get that

Me: You have not been programmed well have you?

LawDroid: Sorry, I didn’t quite get that

Me: You don’t get very much do you?

LawDroid: Sorry, I’m still not getting that.

Me: Will we start again?

LawDroid: Hmm. I didn’t get that. Sorry.

Me: Goodnight.

LawDroid: Hmm. I didn’t get that. Sorry.

I gave up. A lot of time and questions to not get the answer I was looking for.

The Global Legal Hackathon website didn’t help much on the pizza or food front either. All I could find under a “food” search (a “pizza” search returned no results) was a recommendation to bring non-alcoholic beverages with you.

But the London Legal Hackathon had their own website with a FAQ page which reveals to the question “Will there be food?”:-

We are catering for around 100 participants comprising of 8 to 12 teams. There will be a plentiful supply of tea, coffee, water and meals (kindly provided by our sponsors). We’ll add more details here soon (and we’ll cater for non-meat eaters too, but please warn us if you have any special dietary requirements).

We will provide lunch and an evening meal, but not breakfast, so please grab a bite on your way Saturday and Sunday please.

You are welcome to bring your own food and refreshments if you want.

No mention specifically of pizza but at least it tells you there will be food.

Although confirmation, if needed, that there was pizza (at least in London and Sydney) could be found on Twitter:-

So a quick glance through a FAQ page on the London site (and this might apply to other location sites on the global circuit) was much quicker and more helpful than using the chatbot on the main site. So far, on the whole, that has been my experience of using legal chatbots. Do you agree? Or have you had good experiences of using legal chatbots? If you want to do a guest post of your conversation with a legal chatbot on this blog, for a future episode in this series, I will be happy to consider any submissions. Just e-mail it to me.

Reactions on Social Media

There have been reactions to this post on LinkedIn. To keep these together with the post itself I have copied the comments here:-

Alex G Smith:

I broke a GDPR one in a minute by asking it what privacy by design was. It then had a chat with me about how my day was and was having a good time. I only tried it because the hype said AI had landed … I never expected the Spanish Inquisition on how I was feeling about life in general. Can’t wait for part 2.

Graham Laing:

Create something even less responsive than the lawyers they ultimately seek to replace. Only in legal ;-P

Garth Watson:

If you think about the phrase “privacy by design” it’s no wonder it was confused.

Surely, “well designed privacy policies”, as opposed to privacy itself by a thing called “design” would be a better way of putting it.

Most lawyers would also struggle to answer what privacy by design was. Most obvious answer is locking the door.

Totally get that these thing have a long way to go, but baby steps come first.

Graham Laing:

That is an interesting question in itself – can ‘baby steps’ be permissible in a highly regulated environment? My view is not. Something works or it doesn’t.

Alex G Smith:

Garth Watson, Privacy by design is a core principle that GDPR pushes forward and given that these bots are “hand programmed” not self learnt I felt it not a hard question to ask. I don’t mind if the narrow decision tree it no doubt was following didn’t have it but if that’s all it is why the whole bot thing, an FAQ or a nice blog could have done the job. I’ll give your questions (or search terms) a go and see what happens … my guess is I’ll end up in the bizarre banter on “how I am” and “am I well” as last time.

-+-+-+-+-+-

Nir Golan:

At least you checked one of their GLH requirements box- legal chatbot.. check. it doesn’t matter if it does not solve the problem. That’s not relevant to the GLH.

-+-+-+-+-+-

Jonathan Maskew:

Well worth having a chat with Billy Bot

Brian Inkster:

Oh I have! We didn’t chat about pizza, although I understand he (unlike LawDroid) has been trained to deal with pizza queries. I would expect Billy to feature in a future episode of ‘Chats with Legal Chatbots’ 🙂

-+-+-+-+-+-

Enia Titova:

I don’t understand how an industry that puts either “this is not legal advice” disclaimers or “PRIVILEGED & CONFIDENTIAL, ATTORNEY-CLIENT COMMUNICATION” on literally everything that comes out of its mouth thought this was a good idea.

3 replies on “Chats with legal chatbots – episode 1: The Global Legal Hackathon and LawDroid”

Hi Brian – Thanks for checking out LawDroid as deployed on the Global Legal Hackathon website. In the lead up to GLH, LawDroid helped provide thousands of attendees and hosts with helpful information to prepare. However, it wasn’t programmed to answer venue specific questions. We did build in an opportunity for a person to ask a question that wasn’t answered by the bot and pass it on to a GLH volunteer for assistance. This automated + human approach worked very well for many, many people. LawDroid gave you this opportunity to ask such a question which it would email to a volunteer for follow up, but you didn’t do that. It also provided instructions for how to restart the conversation but you didn’t do that either. So, yes, you managed to break it. I would have enjoyed speaking with you to discuss our approach to designing the bot, what problem we were attempting to solve, what results we achieved, and what we learned, but you never reached out to me to discuss.

Tom

I didn’t think of it as a venue specific question as pizza and hackathons seem to go hand in hand from London to Sydney (although I accept some locations may not have been so keen on pizza).

However, in determining questions that might be asked by attendees I would have put it fairly high on the list.

The opportunity you built in for a person to ask a question that wasn’t answered by the bot and pass it on to a GLH volunteer for assistance was rather odd in its approach. I asked “Will there be pizza?”. The bot answered “So we can follow up with you about your concern, I’ll need a couple of pieces of information. What’s your email address?”. It wasn’t a “concern” as such just a query. The bot didn’t indicate that it didn’t know the answer and that a human would have to help if I supplied an e-mail address. It would have been more helpful had it done so in a more straightforward and honest fashion.

I must have missed the instructions for how to restart the conversation. Perhaps it would be helpful for the bot to actually tell you these when struggling as it was to answer. However, I would have thought that it would not be that difficult to factor in a restart from a statement like “Will we start again?”. However, I’m not sure I would want to restart and go around in circles again in any event!

I’m not sure that I need to reach out to the bot’s designers before posting an episode of an actual conversation with a chatbot. I am simply demonstrating what such a conversation is like and the limitations that chatbots may have as a result. It seems clear that they can only deal with what they have been programmed to deal with and that won’t always help the person chatting with them. Indeed it is likely to just lead to frustration.

Jonathan Maskew:

I spoke to Billy Bot, he crashed.
You said:
Contact a solicitor
Delivered at: 9:47am
Billy said:
Ok to put you in touch with a solicitor I just need a couple of bits of information. Could you tell me your name please?
Received at 9:47am
You said:
Dave
Delivered at: 9:47am
Billy said:
Ok could I please have your email address?
Received at 9:47am
You said:
dave@dave.com
Delivered at: 9:47am
Billy said:
Thanks! Could you please tell me your phone number?
Received at 9:47am
You said:
01234567890
Delivered at: 9:47am
Billy said:
Ok thanks, and finally if you could provide a brief description of what your enquiry is regarding that would be great.
Received at 9:47am
You said:
looking to get off a murder charge
Delivered at: 9:47am
Billy said:
One second while I process that for you…
Received at 9:47am

We are not 4 minutes on from that and he’s still thinking,

Comments are closed.