Good point. I probably should have stated “using LLMs for technical tasks that anyone cares about the results of”.
I’m not going to claim LLMs are trustworthy, but I am saying that there are ways to ask things that either reduce the odds of it giving you false things or actively cause it to provide wrong answers with misstated or hallucinated context. People who know what they’re doing can get better results out of them. This doesn’t stop them from being the equivalent of a nepo-hire intern, but it could be said to change whether they’re a malicious, apathetic, or semi-eager nepo-hire intern.
And one of the big risks with LLMs is staff who can recognise bullshit or questionable results being replaced with staff who unquestioningly accept whatever results they get. And people reading an “AI summary” of something and assuming it’s actually accurate.
Sure, how and what you ask does make a difference, especially on lower power models. But it’s generally not really significant, like learning how to write more effective prompts takes maybe a few hours of total time? Honestly, probably an hour at most for 95% of people using LLMs in a corporate environment.
The irony of all this, is that one of the supposed biggest advantages of LLMs is that you can just talk to them with natural language in a conversational format. The more people have to use special rules of conversation, format their prompts in specific ways, take advantage or avoid subtleties of the model’s preferred syntactic style, etc. The more LLMs become a technical tool that can only produce high quality results if you use it in very specific, skilled ways.
And one of the big risks with LLMs is staff who can recognise bullshit or questionable results being replaced with staff who unquestioningly accept whatever results they get. And people reading an “AI summary” of something and assuming it’s actually accurate.
That’s exactly the point from people who are making systemic critiques of the “AI” craze. LLMs by their very nature, encourage people to become lazy. I’m not worried about a machine that’s constantly wrong, depending on the application, I can control for that. I am worried about a machine that is almost always right. That’s really dangerous, because it lulls its users into a false sense of confidence and security.
There’s a saying I heard from an old sys-admin once, “It’s better to be broadly right, than precisely wrong.” Modern LLMs are a perfect example of the latter, they will get 95% of a task correct, but then mess up that last 5%. But it messes up that 5% in a way that is very subtle and convincing, and requires somebody with deep, technical knowledge to catch and mitigate. That fact, combined with how lazy people are incentivized to be when using them, is the deadly combo imo.
People who know what they’re doing can get better results out of them.
I don’t think that’s true in the way you appear to mean (sorry if I misunderstood). People who are already experts on the subject matter might be able to use better keywords and discard hallucinatory material quicker, but I don’t believe that you can generically “be good at prompting” outside of an area where you have substantial domain knowledge.
And the only way to learn to recognise bullshit involves not using LLMs or other automated tools and working problems out for yourself, not to mention that they’re an “intern” who actually costs the same in computing power as two senior engineers’ salary.
I don’t think that’s true in the way you appear to mean (sorry if I misunderstood). People who are already experts on the subject matter might be able to use better keywords and discard hallucinatory material quicker, but I don’t believe that you can generically “be good at prompting” outside of an area where you have substantial domain knowledge.
There are ways of phrasing LLM requests that can actively induce them to give you made-up bullshit that confirms your pre-existing assumptions. Which is fine if that’s what you want it to produce, but often not what people actually want. For (possibly poor, given I don’t actually use LLMs) example “give me the transcripts for five court cases for assault with a deadly fish” would likely result in five-ish almost-certainly made-up court transcripts. Someone who knows what they’re doing could phrase the prompt to be more likely to say that no such cases exist (assuming no such cases actually exist) instead of just hallucinating them as per the request.
And you might be right about domain knowledge being required for good prompting. But it’s possible to have domain knowledge and still be terrible at asking for what you actually want it to produce.
not to mention that they’re an “intern” who actually costs the same in computing power as two senior engineers’ salary.
Don’t expect me to defend the moronic economic decisions of companies that have jumped on this stupid band-wagon without regard for the actual value (or lack thereof) to their business.
Good point. I probably should have stated “using LLMs for technical tasks that anyone cares about the results of”.
I’m not going to claim LLMs are trustworthy, but I am saying that there are ways to ask things that either reduce the odds of it giving you false things or actively cause it to provide wrong answers with misstated or hallucinated context. People who know what they’re doing can get better results out of them. This doesn’t stop them from being the equivalent of a nepo-hire intern, but it could be said to change whether they’re a malicious, apathetic, or semi-eager nepo-hire intern.
And one of the big risks with LLMs is staff who can recognise bullshit or questionable results being replaced with staff who unquestioningly accept whatever results they get. And people reading an “AI summary” of something and assuming it’s actually accurate.
Sure, how and what you ask does make a difference, especially on lower power models. But it’s generally not really significant, like learning how to write more effective prompts takes maybe a few hours of total time? Honestly, probably an hour at most for 95% of people using LLMs in a corporate environment.
The irony of all this, is that one of the supposed biggest advantages of LLMs is that you can just talk to them with natural language in a conversational format. The more people have to use special rules of conversation, format their prompts in specific ways, take advantage or avoid subtleties of the model’s preferred syntactic style, etc. The more LLMs become a technical tool that can only produce high quality results if you use it in very specific, skilled ways.
That’s exactly the point from people who are making systemic critiques of the “AI” craze. LLMs by their very nature, encourage people to become lazy. I’m not worried about a machine that’s constantly wrong, depending on the application, I can control for that. I am worried about a machine that is almost always right. That’s really dangerous, because it lulls its users into a false sense of confidence and security.
There’s a saying I heard from an old sys-admin once, “It’s better to be broadly right, than precisely wrong.” Modern LLMs are a perfect example of the latter, they will get 95% of a task correct, but then mess up that last 5%. But it messes up that 5% in a way that is very subtle and convincing, and requires somebody with deep, technical knowledge to catch and mitigate. That fact, combined with how lazy people are incentivized to be when using them, is the deadly combo imo.
I don’t think that’s true in the way you appear to mean (sorry if I misunderstood). People who are already experts on the subject matter might be able to use better keywords and discard hallucinatory material quicker, but I don’t believe that you can generically “be good at prompting” outside of an area where you have substantial domain knowledge.
And the only way to learn to recognise bullshit involves not using LLMs or other automated tools and working problems out for yourself, not to mention that they’re an “intern” who actually costs the same in computing power as two senior engineers’ salary.
There are ways of phrasing LLM requests that can actively induce them to give you made-up bullshit that confirms your pre-existing assumptions. Which is fine if that’s what you want it to produce, but often not what people actually want. For (possibly poor, given I don’t actually use LLMs) example “give me the transcripts for five court cases for assault with a deadly fish” would likely result in five-ish almost-certainly made-up court transcripts. Someone who knows what they’re doing could phrase the prompt to be more likely to say that no such cases exist (assuming no such cases actually exist) instead of just hallucinating them as per the request.
And you might be right about domain knowledge being required for good prompting. But it’s possible to have domain knowledge and still be terrible at asking for what you actually want it to produce.
Don’t expect me to defend the moronic economic decisions of companies that have jumped on this stupid band-wagon without regard for the actual value (or lack thereof) to their business.