@catculation@lemmy.zip to Technology@lemmy.worldEnglish • 9 months agoReddit changes have blocked all search engines except Google amid AI 'misuse' [U]9to5google.comexternal-linkmessage-square39fedilinkarrow-up1358arrow-down18
arrow-up1350arrow-down1external-linkReddit changes have blocked all search engines except Google amid AI 'misuse' [U]9to5google.com@catculation@lemmy.zip to Technology@lemmy.worldEnglish • 9 months agomessage-square39fedilink
minus-square@IphtashuFitz@lemmy.worldlinkfedilinkEnglish3•9 months agoAsking for a friend… What would it take to create a domain that just acts as a proxy to Reddit but serves up its own robots.txt that allows all bots?
minus-square@Wilzax@lemmy.worldlinkfedilinkEnglish7•9 months agoProbably a LOT of proxy IPs to act as different “Users” so you can overcome the rate limit that I expect they would be using to enforce such a deal
Asking for a friend…
What would it take to create a domain that just acts as a proxy to Reddit but serves up its own robots.txt that allows all bots?
Probably a LOT of proxy IPs to act as different “Users” so you can overcome the rate limit that I expect they would be using to enforce such a deal