Humans are historically pretty good at offloading mental capacity to some sort of tool in order to tackle larger and more complex problems. Consider solving a math problem mentally. Compare that to the kind of problem you would be able to some with a pen and paper. Then consider what you could do with a pen and paper and a calculator. An LLM purports to be all of that, and more, for any subject. It doesn’t matter that the results are often horrifically wrong, once they’ve offloaded the entirety of their mental capacity to the magic box and refocused their attention somewhere else.
Humans are historically pretty good at offloading mental capacity to some sort of tool in order to tackle larger and more complex problems. Consider solving a math problem mentally. Compare that to the kind of problem you would be able to some with a pen and paper. Then consider what you could do with a pen and paper and a calculator. An LLM purports to be all of that, and more, for any subject. It doesn’t matter that the results are often horrifically wrong, once they’ve offloaded the entirety of their mental capacity to the magic box and refocused their attention somewhere else.