Screenshot of this question was making the rounds last week. But this article covers testing against all the well-known models out there.

Also includes outtakes on the ‘reasoning’ models.

  • timestatic@feddit.org
    link
    fedilink
    English
    arrow-up
    4
    arrow-down
    1
    ·
    17 hours ago

    Yeah seems like the training on human data makes it so most AIs will answer at least as unreliable as humans. 71% saying walk from the human side is crazy

    • UltraMagnus@startrek.website
      link
      fedilink
      English
      arrow-up
      5
      ·
      14 hours ago

      I think you misread it - 71% said drive. 29% is still pretty bad, but it is kind of a “who is buried in grants tomb” question.

      • timestatic@feddit.org
        link
        fedilink
        English
        arrow-up
        1
        ·
        8 hours ago

        Oh I actually just switched it up accidentally while typing. I read it right but still almost one out of three doesn’t get it