Cuil Reported Destroying the Internet

Cuil Reported Destroying the Internet

Cuil, the search engine proclaimed as a “Google killer", is said to have brought many websites down. Cuil’s Twiceler indexing bot eats up too much bandwidth that it’s shutting down sites all over the Web.


A tipster wrote to TechCrunch stating, “I don’t know what spawned it, but when Cuil attempts to index a site, it does so by completely hammering it with traffic.So much, that it completely brings the site down. We’re 24 hours into this “index” of the site, and I’ve had to restrict traffic to the site down to 2 packets per second, while discarding the rest, or otherwise it makes the site unusable.”



In the Admin Zone Forum, several users were fussing about Cuil's indexing method. Twiceler robot is accounted to have caused major bandwidth leeching on numerous sites.  James Akers, Cuil’s Operational Engineer, explained that Twiceler is an experimental crawler that the company is developing for its new search engine. He also added, "It is important to us that it obey robots.txt, and that it not crawl sites that do not wish to be crawled. If you wish I will glad to add your site to our list of sites to exclude, but I need you to tell the site name to block as email return addresses frequently from the domains that wish to be blocked."  Akers also asserts that there are numerous crawlers feigning as Twiceler and site owners should check the company’s IP addresses page before blaming it all on Cuil.