Dutch lawyers increasingly have to convince clients that they can’t rely on AI-generated legal advice because chatbots are often inaccurate, the Financieele Dagblad (FD) found when speaking to several lawfirms. A recent survey by Deloitte showed that 60 percent of lawfirms see clients trying to perform simple legal tasks with AI tools, hoping to achieve a faster turnaround or lower fees.
I can’t stand AI, but the few times I’ve used it, I’ve used it as a starting point. Once it gives me advice, I then go and confirm that with other sources. But I don’t use AI much.
I had it cite a case which didn’t exist. It was perfect for what I was fighting (it tends to figure out what you want to hear then makes up stuff to satisfy you).
When I tried to search for a phrase from the case (hoping it just gave the wrong citation) it said there was no such case with that phrase.
I asked why it said there was such a case earlier. It confessed that AI sometimes hallucinates and promised to try better in future.
Let’s consider what you are doing on a purely abstract level.
We could also do the following:
That’s great when you do, and you usually do, but sometimes you don’t.
Case in point; A while back I was creating a 3D model for my 3D printer. It had a part that was essentially identical to a particular unusual pipe fitting that I have seen and knew existed, but didn’t know the name of (spoiler: I’m not a plumber), and I wanted to give the sketch in the modeling software a proper name for the thing.
Just trying keywords that sort of described it’s shape in search engines was useless. Search engines would focus more on the “pipe fitting” part of the keywords and just return links to articles about plumbing. Then I asked an LLM, and it responded with, “That sounds like X.” Then I checked that it wasn’t just making it up by searching for “X” and found online stores selling the very thing I was trying to figure out the name of.
Yes LLMs are good with finding terms and phrases that you can’t remember but are at the tip of your tongue