Updated robots.txt to disallow all webscrapers
This commit is contained in:
parent
e122e5f186
commit
bc81dfc6d9
1 changed files with 1 additions and 1 deletions
|
@ -1,2 +1,2 @@
|
|||
User-agent: GPTBot
|
||||
User-agent: *
|
||||
Disallow: /
|
Loading…
Reference in a new issue