robots: Banned GPTBot off the website
This commit is contained in:
parent
4434d76dfb
commit
65e3e48e00
1 changed files with 3 additions and 0 deletions
3
src/robots.txt
Normal file
3
src/robots.txt
Normal file
|
@ -0,0 +1,3 @@
|
||||||
|
User-agent: GPTBot
|
||||||
|
Disallow: /
|
||||||
|
Allow: /ai.html
|
Loading…
Reference in a new issue