That said, the structure of your crawl-hold off directive for each of such engines is the same. You should area it right following a disallow OR permit directive. Here is an case in point: Look for syntax faults. Just one oversight within the syntax of one's Robots.txt file might cause https://rankingseotoolz.com/robots-txt-generator