• Legit
  • Posts
  • ⚖️ One paralegal's investigation into DoNotPay

⚖️ One paralegal's investigation into DoNotPay

The paralegal battling DoNotPay, generative AI, and more.

Hello! I'm super excited about the first article in today's newsletter.

In today's issue:

  • The paralegal taking on DoNotPay

  • Lawyers start to use generative AI

  • Legaltech startups to watch out for

EXPOSÉ

The paralegal taking on DoNotPay

Kathryn Tewson, a paralegal in Washington, is going head-to-head with DoNotPay, the self-proclaimed first robot lawyer, in a series of exposés and a fraud lawsuit.

If you read anything in today's newsletter, this should be it.

First, some context. When Joshua Browder burst on the legal tech scene in 2015, his story was compelling. An 18-year old student in the UK builds a tool to fight parking tickets, overturning 160,000 tickets in New York and saving consumers $4m within just one year. Then he expands to help the newly evicted and refugees seeking asylum. Then he builds out help for over 1000 areas of law and raises a $10m Series B from a16z, Felicis, Tribe Capital and prolific angel investors. These successes stack up and, across a slew of articles and podcast appearances, Browder is championed as the golden child of legal tech — a maverick who succeeded in interrupting a legal industry that's often pegged as archaic and slow-moving.

Around a month ago, Browder made headlines by tweeting that he would pay anyone with an upcoming Supreme Court case $1M to wear AirPods that let DoNotPay's GPT3 'robot lawyer' argue the case. When no one took the bait, Browder announced that DoNotPay would represent a client in court to fight a speeding ticket for the first time. The tweet was met with immediate backlash, and Browder quickly pulled the plug on the experiment after multiple state prosecutors warned him he could be prosecuted for unauthorised practice of law.

To Kathryn Tewson, there was always something off about the speeding ticket case. When Browder tweeted that he'd sent out a subpoena generated by AI for the traffic cop in the upcoming case, Tewson picked up on the fact that pretty much the last thing any defendant in a traffic court case would want to do is subpoena the cop. Also, subpoenas have to be signed by a party to the case or the attorney in order to be sent out. When Tewson asked Browder who signed the subpoena, he deleted his tweet and blocked her.

Browder's pattern of avoidance sparked Tewson's curiosity. She started digging into DoNotPay, signed up to the platform and started testing the products. Her findings are pretty wild. Here are the main points:

  • After typing in all the prompts to generate a defamation demand letter, DoNotPay said the letter would be ready in 1 hour. She did the same thing to generate a divorce settlement letter, and was told by the app that it would be ready in 8 hours. "I was like, that's weird. Because ChatGPT works instantly...one hour is not a computer-type timeline. And eight hours is really not a computer-type timeline. I began to get suspicious that what was doing the analysis was human beings."

  • When the one-hour for the defamation demand letter was up, DoNotPay told Tewson it needed more time and was working behind the scenes to 'make everything right.' "I was like, what behind the scene is there with AI?... At that point I began to become suspicious that it was not just humans, but like random freelancers, not even attorneys." After a day and half, there was still no defamation demand letter.

  • After Tewson complained to Browder, he blocked her account, discontinued those products and made it a general violation to "test" DoNotPay.

  • Tewson then launched an open investigation on Twitter.

  • She worked with Debbie Mia, a software engineer, to analyse publicly-available city parking ticket data. She found the total number of tickets successfully reversed on appeal in NYC for all of 2014, 2015 and 2016 was 18,889 — making Browder's previous claim that he helped overturn 160,000 tickets bogus.

  • She also found there was no spike in the number of successfully appealed tickets in either NYC or London over the period that DoNotPay broke onto the scene.

So, things aren't looking good for DoNotPay. Tewson has now filed a petition for pre-action discovery against DoNotPay in the hopes of getting more information to be able to peel back the layers of what she believes is a "fraud onion", and has compared Josh Browder to Theranos founder Elizabeth Holmes.

To make things even worse, techdirt came forward with an article they didn't run in 2017, where David Colarusso, a lawyer and data scientist, also noticed that nothing that DoNotPay claimed seemed to add up.

If you want to dig into the details, you can read Tewson's full interview here, or listen to her podcast with Bob Ambrogi here.

ARTIFICIAL INTELLIGENCE

Lawyers <3 generative AI

London-based Magic Circle firm Allen & Overy have partnered with Harvey, a law-based generative AI tool.

Background. A&O was first introduced to Harvey last September with the intention of running a small trial, where a few lawyers at the firm would use the system to ask legal questions, draft documents and write emails to clients. The trial ballooned, and it wasn't long before 3,500 lawyers across 43 offices had asked the tool 40,000 queries.

Now, A&O have entered into a formal partnership with Harvey. The value of the agreement is undisclosed, but Harvey says 25% of the firm's lawyers use the tool everyday, while 80% use it at least once a month.

The important stuff from Wired's coverage:

  • Lawyers are good customers for generative AI. Standardized documents are the cornerstone of the legal industry but labor-intensive to create, meaning Harvey can do the heavy lifting to create first drafts that can then be tailored towards clients.

  • A&O isn't feeding any client data into Harvey (because of the GDPR and crackdowns on generative AI using personal data), and wouldn't do so unless that data was completely ring-fenced.

  • All generative AI tools have a tendency to hallucinate (i.e. make things up). A&Os response to this is requiring that their lawyers "validate everything" Harvey produces. "We can't have hallucinations contaminating legal advice."

My (vanilla) take: I don't think it'll be long until tech-savvy clients (think startup founders) use tools like Harvey and ChatGPT to generate basic contracts on their own, cutting out or at least significantly reducing the involvement of their lawyers.

  • I'm curious to know whether lawyers agree or not. Feel free to respond to this email or message me your thoughts.

One liners

Legal tech to watch

Tome: venture deals simplified.

  • Blurb: Tome summarises contracts for angel investors and venture capitalists, starting with standard form investment documents (like the Y Combinator SAFE).

  • Location: San Francisco

  • Value prop: Tome analyses your investment documents and generates a report within 58 seconds that summarises the key terms and tells you if any term or clause seems unusual (compared to their bank of precedents). I have personally used Tome and think it's great

  • Stage: seed