5 lines
160 B
Text
5 lines
160 B
Text
|
# Learn more about robots.txt: https://www.robotstxt.org/robotstxt.html
|
||
|
|
User-agent: *
|
||
|
|
# 'Disallow' with an empty value allows all paths to be crawled
|
||
|
|
Disallow:
|