top of page
Search

Legal tech: Beyond the myths #5 - How do robots handle legal practice?

By Arnoud Engelfriet


Last time, we ended on a positive note: it’s not about the tool but about the humans using it. Which prompted several readers to write in and ask, how will the humans be using it? We know how to work with a new colleague, but what can you expect on a robot’s first day in the law department? More to the point, what can you do to get a robot to work as you expect? This gets to the issue of what legal tech aficionados call the playbook.

Numbers, numbers, numbers

Whenever we talk about artificial intelligence, the word algorithm makes an appearance. To programmers this is a bit strange, because the one thing that AI does not apply is algorithms. AI is essentially statistics-driven: does this phrase resemble this set of phrases or more that one, which of these twenty word groups is most similar to this word, and so on.


This is in stark contrast to what an algorithm is to programmers, namely a series of precisely-defined steps to take, one after the other. In an algorithmic approach, all possible choices would be enumerated in advance and the robot would simply check off the lists one by one. Of course, this is impossible, which is precisely why the statistics-driven approach of AI has proven so successful. With enough data, patterns can be discerned that allow for the derivation of bright lines (although quite complex lines) to help identify patterns.


However, this process of classification or recognition is merely one small part of the system that makes up a lawyerbot. Several other key pieces are needed to arrive at a fully functional system, one that can produce legal advice given a contract, case description, legal brief or other input document. And it is in these pieces that traditional algorithms play a large role.


Building the system

Let’s take the use case of contract review as an example. In other words: how can we turn a contract we received from a prospective partner into a redline that will further the negotiations? It’s good to keep perspective here. As noted earlier in the series, robots are good at standard work. So this review will be a check for standard issues. No one would use such a review for a high-stakes merger agreement or a complex bespoke services purchase. But such documents are rare: over 80% of legal documents reviewed by in-house lawyers or by attorneys-at-law are routine agreements.


Reviewing a standard agreement means checking for standard issues and making standard amendments to better reflect the client’s interests. A term should be a bit longer, a contractual fine should be gone for a service provider, liability should be capped at the level of the liability insurance, and so on. All perfectly suitable for a robot: spot the issue, grab the best alternative clause and make the amendmend.

Thanks to the above mentioned statistical approach, spotting the issue is not a problem for a well-trained machine learning system. But we have some steps to take before we get there. To start with: how do you turn a Word document into a set of clauses, from which issues can be spotted?


Reading like a robot

This may seem like a silly question: just open the document and read the text. But for a robot, this is a lot harder than it sounds. We humans see text, well-formatted, some bold, some italics, a list of definitions at the end, indented lists, and so on. For a computer, this is all a mass of letter groups held together with XML tags that may or may not provide a fixed structure to guide identification of what’s what.


A first approach could be to simply extract the paragraphs as Word provides them. Headings, normal text, tables and other elements have their own markup – technical codes that provide boundaries to separate text and to attach styles to. This markup can be recognized automatically, allowing e.g. a list of clauses to be built with the last-used heading preceding it. Then, each clause can be fed to the machine learning system to identify it and to determine what’s wrong with it.


There’s a little problem with that: we humans tend to be sloppy with our styles. In the tens of thousands of documents I have reviewed for our lawyerbots, I have yet to find the perfect document marked up as Microsoft had in mind when it created Word. People don’t apply styles but merely colour some text bold or enlargen it to indicate “heading”. Old-fashioned authors press enter when they think a line is done, as they did in the typewriter era. This looks the same, but for Word this means there are two paragraphs, two clauses – and to a robot, that means two incomplete clauses to review.


Fortunately, there are advanced text extraction libraries available, so part of the problem is solved. But once you add PDF support – a popular request – you get an entirely new set of problems, because PDF is designed to look good, not necessarily to automatically retrieve text from. For instance, if you extract text from a two-column PDF document you get garbage: every line from the right column is appended to the one appearing on that line in the left column, and you try making sense from that.


But even if you get the text out exactly as the writer intended it, you still have the issue of what goes together. Not every contract is of the one-clause-per-paragraph type. Consider for instance the numbered list, e.g. a list of representations and warranties that a seller provides. Or a list of undertakings by the recipient of confidential information. Or the security measures a data processor commits to in a GDPR data processing agreement. Is that one clause? Should it be split and evaluated separately, or considered one mega-clause? Both have advantages and disadvantages, but the quality of text extraction is key to a good identification.

Playing by the book

Having fed each clause to the machine learning system, we now know what each clause is about, as well as what’s going on within that topic. Here’s a liability clause, its cap is 2.5 million dollars. This venue clause picks Paris, France. The term is one year with silent renewal. This list of security measures is very long and its text does not conform to article 32 GDPR. Good to know, but is this bad?


Now we get to the heart of any robot lawyer: the playbook. The playbook is a set of boundaries if you were, a list of all possible outcomes that the machine learning can provide tied to a judgment. Typically each outcome is tied to a response message that can be included in the report generated by the lawyerbot, or coupled to an amendment or counterproposal that would overcome the issue that was found.


A simple example is a liability cap. The playbook can be configured to reject any caps higher than one million, or in the converse to insist on at least two point five million. Or whichever amount you want – this of course depends on the user’s perspective. Should the machine learning system then detect that a particular agreement has a cap that is three million, then the playbook will indicate this is a problem: it should be one million, no more.

Often, a legal position is not expressed in a number, but rather in some sort of criterion or position. A client may not be prepared to accept contractual fines, or doesn’t want a dispute venue outside the European Union. GDPR security obligations should always include two-factor authentication, compliance audits should be undertaken by independent auditors bound by professional secrecy and their bill paid by the other guy. Playbooks can configure this type of counter: if venue is not in this list, reject. If auditor is not 3rd party, reject. If auditor is 1st party or audit rights is not found, accept. And so on.


Configuring the playbook takes a surprisingly long time, even when experienced negotiators or lawyers are brought to the table. This is because many of them operate from intuition and rules of thumb. Perhaps that cap of one million is because of the insurance payout, but where did the two-factor authentication demand come from? Do we always object to fines or just if they are large, and if so what is “large” if we have to put a number on it?


Would we take a late delivery fine if payment is within 14 days? Getting these positions out of the human heads and into the computer table is more time-consuming than training the machine learning system.


(One startup in this field uses the clever approach of analyzing a customer’s own contracts to determine what should be acceptable. This works, although it is not uncommon for large companies to have purchasing terms that conflict utterly with their sales terms.)


Getting a response

The next, and ideally final step in the process is to produce a response. A simplistic approach is to just accompany each boundary in the playbook with a fixed sentence, such as “We cannot accept this high a liability cap, it must be lowered to one million.” Whenever a clause triggers this playbook boundary, this text is included in the output. The end result is a nice summary of issues with counterpositions, presumably drafted by a human lawyer.

Today, legal professionals expect more and lawyerbot systems should give them that. Some variation is nice: instead of the same text for the same issue, allow for multiple outputs given that issue and select one. One approach suggested in the literature uses a simple counter: the more objectionable issues were encountered, the more terse and blocking each subsequent response became. Just like a human lawyer or negotiator would get if a contract draft keeps asking and asking.


Very helpful is adding the outputs as comments in the Word or PDF document. This is part and parcel of human reviews, and lawyerbots should seek to emulate that. Comments can then immediately be tied to the relevant text, reviewed for correctness and responded to with a counter.


But what really takes the cake is an amendment in the text, of course as a redline (tracked changes) using Word’s facility for registering who changed what. This makes the counterproposal immediately clear, and allows the other party to respond focused on the content (or accept the proposal, since we are still talking about standard documents with standard counters).


This may seem hard. It’s true that AI can generate texts – the OpenAI generator called GPT-3 can produce near-perfect quality essays of any length in seconds, for instance. But coming up with a counter in a specific clause seems like a bit much. Fortunately there’s no need to introduce yet more artificial intelligence. A preconfigured set of counterproposals is more than enough. If you have twenty acceptable security clauses, one is likely to match the language of the original text closely and so can be used as a basis to draft an amendment. We are still talking about standard documents after all; in my experience there are rarely more than twenty variations of any clause in NDA’s or DPA’s, for instance.


In and out in minutes

Now let’s put things together. We can take Word documents and convert them to machine-readable clauses, with the right headings attached. We can take those clauses and assign meaning to them – strict security, 7-year term and so on. We can look up whether we like that – term no more than 2 years, strike that fine, no cap on liability. We can insert our objections into the document and even amend the clause proper.


One final issue remains: how do we get documents in and out? A web interface seems intuitive, but there’s an even more exciting option – just e-mail your document and get a reply back, just like you would work with a human lawyer. Except this one mails you back in five minutes. Does that sound like a myth to you?

 

About the Author

Arnoud Engelfriet is co-founder of the legal tech company JuriBlox, and creator of its AI contract review tool Lynn Legal.

Arnoud has been working as an IT lawyer since 1993. After a career at Royal Philips as IP counsel, he became partner at ICTRecht Legal Services, which has grown from a two-man firm in 2008 to a 80+ person legal consultancy firm.


bottom of page