User-agent: * Allow: /social_instagram/ Sitemap: https://aarkautomation.com/sitemap.xml ############## # custom # ############## User-agent: Googlebot Disallow: User-agent: googlebot-image Disallow: User-agent: googlebot-mobile Disallow: User-agent: MSNBot Disallow: User-agent: * Disallow: Disallow: /cgi-bin/