Nuking the Corporate Web's Web-Scraping Robots
cheapskatesguide.org
Many small website owners can defend their sites from web-crawling robots without big tech's help. Here is how.

Lemmy newb here, not sure if this is right for this /c.

An article I found from someone who hosts their own website and micro-social network, and their experience with web-scraping robots who refuse to respect robots.txt, and how they deal with them.

  • klu9@lemmy.caOPEnglish
    3·
    1 day ago

    You’re welcome.

    I believe I found it originally via the “distribuverse”… specifically, ZeroNet.