If you use a physical robots.txt on a WordPress powered site, then stop it now!
Some web hosts, such as the very popular SiteGround, add crawl-delay: 10 to the physical robots.txt when it is accessed by a bot. If you examine your robots.txt file with a text editor, you won’t see it. It’s not there physically. But, if you examine robots.txt in the Google Webmaster robots.txt tester tool, you’ll see that the crawl-delay line has been added. To learn why this could be a problem, please see this page on Yoast: https://yoast.com/ultimate-guide-robots-txt/#crawl-delay-directive.
https://gist.github.com/vfontjr/636628925e3bd3a4b54530c02df57a67#file-robots_mod-php