this repo has no description
at main 131 lines 3.3 kB view raw
1User-agent: AddSearchBot 2User-agent: AI2Bot 3User-agent: AI2Bot-DeepResearchEval 4User-agent: Ai2Bot-Dolma 5User-agent: aiHitBot 6User-agent: amazon-kendra 7User-agent: Amazonbot 8User-agent: AmazonBuyForMe 9User-agent: Andibot 10User-agent: Anomura 11User-agent: anthropic-ai 12User-agent: Applebot 13User-agent: Applebot-Extended 14User-agent: atlassian-bot 15User-agent: Awario 16User-agent: bedrockbot 17User-agent: bigsur.ai 18User-agent: Bravebot 19User-agent: Brightbot 1.0 20User-agent: BuddyBot 21User-agent: Bytespider 22User-agent: CCBot 23User-agent: Channel3Bot 24User-agent: ChatGLM-Spider 25User-agent: ChatGPT Agent 26User-agent: ChatGPT-User 27User-agent: Claude-SearchBot 28User-agent: Claude-User 29User-agent: Claude-Web 30User-agent: ClaudeBot 31User-agent: Cloudflare-AutoRAG 32User-agent: CloudVertexBot 33User-agent: cohere-ai 34User-agent: cohere-training-data-crawler 35User-agent: Cotoyogi 36User-agent: Crawl4AI 37User-agent: Crawlspace 38User-agent: Datenbank Crawler 39User-agent: DeepSeekBot 40User-agent: Devin 41User-agent: Diffbot 42User-agent: DuckAssistBot 43User-agent: Echobot Bot 44User-agent: EchoboxBot 45User-agent: FacebookBot 46User-agent: facebookexternalhit 47User-agent: Factset_spyderbot 48User-agent: FirecrawlAgent 49User-agent: FriendlyCrawler 50User-agent: Gemini-Deep-Research 51User-agent: Google-CloudVertexBot 52User-agent: Google-Extended 53User-agent: Google-Firebase 54User-agent: Google-NotebookLM 55User-agent: GoogleAgent-Mariner 56User-agent: GoogleOther 57User-agent: GoogleOther-Image 58User-agent: GoogleOther-Video 59User-agent: GPTBot 60User-agent: iAskBot 61User-agent: iaskspider 62User-agent: iaskspider/2.0 63User-agent: IbouBot 64User-agent: ICC-Crawler 65User-agent: ImagesiftBot 66User-agent: imageSpider 67User-agent: img2dataset 68User-agent: ISSCyberRiskCrawler 69User-agent: Kangaroo Bot 70User-agent: KlaviyoAIBot 71User-agent: KunatoCrawler 72User-agent: laion-huggingface-processor 73User-agent: LAIONDownloader 74User-agent: LCC 75User-agent: LinerBot 76User-agent: Linguee Bot 77User-agent: LinkupBot 78User-agent: Manus-User 79User-agent: meta-externalagent 80User-agent: Meta-ExternalAgent 81User-agent: meta-externalfetcher 82User-agent: Meta-ExternalFetcher 83User-agent: meta-webindexer 84User-agent: MistralAI-User 85User-agent: MistralAI-User/1.0 86User-agent: MyCentralAIScraperBot 87User-agent: netEstate Imprint Crawler 88User-agent: NotebookLM 89User-agent: NovaAct 90User-agent: OAI-SearchBot 91User-agent: omgili 92User-agent: omgilibot 93User-agent: OpenAI 94User-agent: Operator 95User-agent: PanguBot 96User-agent: Panscient 97User-agent: panscient.com 98User-agent: Perplexity-User 99User-agent: PerplexityBot 100User-agent: PetalBot 101User-agent: PhindBot 102User-agent: Poggio-Citations 103User-agent: Poseidon Research Crawler 104User-agent: QualifiedBot 105User-agent: QuillBot 106User-agent: quillbot.com 107User-agent: SBIntuitionsBot 108User-agent: Scrapy 109User-agent: SemrushBot-OCOB 110User-agent: SemrushBot-SWA 111User-agent: ShapBot 112User-agent: Sidetrade indexer bot 113User-agent: Spider 114User-agent: TavilyBot 115User-agent: TerraCotta 116User-agent: Thinkbot 117User-agent: TikTokSpider 118User-agent: Timpibot 119User-agent: TwinAgent 120User-agent: VelenPublicWebCrawler 121User-agent: WARDBot 122User-agent: Webzio-Extended 123User-agent: webzio-extended 124User-agent: wpbot 125User-agent: WRTNBot 126User-agent: YaK 127User-agent: YandexAdditional 128User-agent: YandexAdditionalBot 129User-agent: YouBot 130User-agent: ZanistaBot 131Disallow: /