Lemmy newb here, not sure if this is right for this /c.

An article I found from someone who hosts their own website and micro-social network, and their experience with web-scraping robots who refuse to respect robots.txt, and how they deal with them.

  • splendoruranium@infosec.pub
    link
    fedilink
    English
    arrow-up
    4
    arrow-down
    1
    ·
    2 days ago

    They block VPN exit nodes. Why bother hosting a web site if you don’t want anyone to read your content?

    Fuck that noise. My privacy is more important to me than your blog.

    It’s a minimalist private blog that sets no 3rd party cookies and loads no 3rd party resources. I presume that alleviates your concerns? 😜