If you would like Cuil to crawl your site and have it included in our index, please submit your site here.
Twiceler is the name of our robot Web crawler. The user-agent is “twiceler.” We understand that many small sites are bandwidth-limited, so we support the robots.txt Crawl-delay directive. You can read about robots.txt at robotstxt.org and there is a simple generator of the file at mcanerin.com.
If you have modified your robots.txt file for Twiceler, it may take several days for us to re-read the file. If you need something blocked right away, please submit a block request.
Got a Twiceler question? If you have questions or concerns about Twiceler you can contact Jim. Jim’s the guy who keeps track of Twiceler, when he’s not busy with his horses.
If you would prefer that we not crawl your site at all we are happy to oblige. Just drop Jim a note to that effect and he will place your site or IP address on our do-not-crawl list. Be sure to be explicit about the site to block as email address domains frequently differ from the site in question.
Occasionally, we have seen other Web crawling robots masquerading as Twiceler. You can be sure it’s Cuil crawling your site if the robot has one of the following IP addresses:
| 38.99.13.116 | 67.218.116.130 | 216.129.119.10 | 216.129.119.40 |
| 38.99.13.117 | 67.218.116.131 | 216.129.119.11 | 216.129.119.41 |
| 38.99.13.118 | 67.218.116.132 | 216.129.119.12 | 216.129.119.42 |
| 38.99.13.119 | 67.218.116.133 | 216.129.119.13 | 216.129.119.43 |
| 38.99.13.120 | 67.218.116.134 | 216.129.119.14 | 216.129.119.44 |
| 38.99.13.121 | 216.129.119.15 | 216.129.119.45 | |
| 38.99.13.122 | 67.218.116.162 | 216.129.119.16 | 216.129.119.46 |
| 38.99.13.123 | 67.218.116.163 | 216.129.119.17 | 216.129.119.47 |
| 38.99.13.124 | 67.218.116.164 | 216.129.119.18 | 216.129.119.48 |
| 38.99.13.125 | 67.218.116.165 | 216.129.119.19 | 216.129.119.49 |
| 38.99.13.126 | 67.218.116.166 | 216.129.119.81 |
To all those who have contacted us to let us know that they are happy to have their site included in a Web index for the first time, thank you for being a part of the biggest search engine on the Web—Cuil!
