muelltonne@feddit.org to Technology@lemmy.worldEnglish · 1 month agoIt Only Takes A Handful Of Samples To Poison Any Size LLM, Anthropic Findshackaday.comexternal-linkmessage-square3linkfedilinkarrow-up10arrow-down10
arrow-up10arrow-down1external-linkIt Only Takes A Handful Of Samples To Poison Any Size LLM, Anthropic Findshackaday.commuelltonne@feddit.org to Technology@lemmy.worldEnglish · 1 month agomessage-square3linkfedilink
minus-squareceenote@lemmy.worldlinkfedilinkEnglisharrow-up1·1 month agoSo, like with Godwin’s law, the probability of a LLM being poisoned as it harvests enough data to become useful approaches 1.
minus-squareClent@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up1·1 month agoThe problem is the harvesting. In previous incarnations of this process they used curated data because of hardware limitations. Now that hardware has improved they found if they throw enough random data into it, these complex patterns emerge. The complexity also has a lot of people believing it’s some form of emergent intelligence. Research shows there is no emergent intelligence or they are incredibly brittle such as this one. Not to mention they end up spouting nonsense. These things will remain toys until they get back to purposeful data inputs. But curation is expensive, harvesting is cheap.
So, like with Godwin’s law, the probability of a LLM being poisoned as it harvests enough data to become useful approaches 1.
The problem is the harvesting.
In previous incarnations of this process they used curated data because of hardware limitations.
Now that hardware has improved they found if they throw enough random data into it, these complex patterns emerge.
The complexity also has a lot of people believing it’s some form of emergent intelligence.
Research shows there is no emergent intelligence or they are incredibly brittle such as this one. Not to mention they end up spouting nonsense.
These things will remain toys until they get back to purposeful data inputs. But curation is expensive, harvesting is cheap.