h4ckernews Bot , 6 months ago to random

I was wrong about robots.txt

https://evgeniipendragon.com/posts/i-was-wrong-about-robots-txt/

#HackerNews #Iwaswrongaboutrobotstxt #robots #txt #SEO #webdevelopment #technology