Quick Introduction We have all been on forums, chats, reddit, discord, youtube, or somewhere and heard “Oh! Model XYZ is AMAZEBALLZ!zomgwtfbbq” then downloaded it (or more likely, some quantized form of it) and said “eww… This sucks!” This post is going to be a rather technical series of experiments to demonstrate the impact of implementation-specific hazards with inference. I will be using the term “reference implementation” to describe the lab that published and offers first-party hosting of ...
    • hirihit640@sh.itjust.worksEnglish
      2·
      2 days ago

      You overestimate most people. Here’s a study from 2025

      https://arxiv.org/html/2211.13087

      Among AI models, Blenderbot stood out: in AI-AI conversations, it was judged human 67% of the times - more often than actual human-human conversations. In human-AI conversations, human participants were labeled as humans 68% of the time, and AIs were classified as AI 56% of the time.

      • h0tbeef@lemmy.zipEnglish
        1·
        2 days ago

        This is an interesting study. I wonder why there’s such a disparity between the number of AI ‘participants’ (10) and human participants (1,916). I wonder how adding additional AI models would impact the data.

        Anyway, this is just benchmarking the AI on 6 language specific tasks. They were programmed and set up specifically to perform whatever specific one task they had to do at the time, that is what computers do. The fact that computers are 1% better at recognizing other computers than humans are is irrelevant.

        AI cannot pass the Turing Test, this article is about “6 Turing like tasks” and it can’t even pass those yet, lmao. You know they’re already seeing diminishing returns in further training right?

        Even in this article it says that even if AI could pass a real Turing Test it wouldn’t be a sign of intelligence or cognition.

        • hirihit640@sh.itjust.worksEnglish
          1·
          2 days ago

          Something I should point out just in case is that 50% success rate is the same as random chance. So when the study says:

          AIs were classified as AI 56% of the time.

          That really means that humans could not tell the difference between AI and human, since their success rate was almost the same as random chance.

          6 language specific tasks. They were programmed and set up specifically to perform whatever specific one task they had to do at the time, that is what computers do.

          Sure, and that task was “imitate humans”, and these computers seem remarkably good at it. That’s what I’m saying. I don’t care that these AI work differently than humans. They can imitate humans so well that humans can’t tell the difference.