URL >>>

TolkAiBot - Bot Directory

Allow TolkAiBot in Cloudflare

💡 About TolkAiBot

TolkAiBot is Genii's official web crawler, used exclusively by our customers to power their RAG (Retrieval-Augmented Generation) systems. This bot only crawls websites whose owners have explicitly authorized access through our platform.

🔍 Crawler Characteristics

• User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36 TolkAiBot/1.0
• Ethical crawling: Controlled request rate to minimize impact on your servers
• Purpose: Content indexation to power our customers' RAG systems

💡 Why Allow TolkAiBot?

⚠️ Important

If you are a Genii customer and have configured your website as a data source for your RAG, you must allow TolkAiBot to access your content. Without this authorization, the bot will not be able to index your pages and your RAG system will not function properly.

⚙️ Cloudflare Configuration

Option 1: Via Verified Bots ✅ Recommended

Cloudflare maintains a list of verified bots that can be globally authorized. We are currently working on adding TolkAiBot to this list.

In the meantime, please use Option 2 below.

Option 2: Create a Custom WAF Rule

1. Log in to your Cloudflare account and select your domain
2. Navigate to Security → WAF
3. Click "Create rule"