minus-squareanthropozaen@feddit.orgtoTechnology@lemmy.world•Wikipedia blacklists Archive.today, starts removing 695,000 archive linkslinkfedilinkEnglisharrow-up6·10 hours agoIt identifies itself as a google (or other) crawler, which sites often allow and give the full content to, for better SEO. linkfedilink
It identifies itself as a google (or other) crawler, which sites often allow and give the full content to, for better SEO.