Canada’s New Cyberweapon “Project Arachnid” Crawls the Internet for Child Pornography

Ever since the public gained access to the internet, some entity always found a way to attract law enforcement’s attention. Hacking, drug dealing, and financial crimes like credit card fraud or identity theft often topped the list in pure numbers. However, one type of crime often attracted more attention than others: child pornography and sites that facilitated the sharing of explicit pictures and videos. A Canadian group—The Canadian Centre for Child Protection—created a tool that crawls the darknet, deepweb, and clearnet to end the furthering of child exploitation on the internet.

The tool, or “cyberweapon,” according to The Canadian Centre for Child Protection, scours the internet for illegal material and, in instances where possible, sends take-down notices to the hosting service. The ongoing case against the admin of Freedom Hosting painted a clear picture of situations where the above method would lack success. However, spokespersons explained that the new tool known as Project Arachnid came as close to perfect as currently possible.

Project Arachnid released an “army of spiders” to crawl the internet that move through websites at a speed previously unseen. But before the Project’s launch, The Canadian Centre for Child Protection tested the spider weapon with several law enforcement agencies worldwide. In a short six-week period, the crawler processed 230 million individual webpages. Of those 230 million pages, Arachnid found 5.1 million unique sites that hosted child pornography. Those sites, combined, contained 40,000 unique images of child sexual abuse.

The crawler, Lianna McDonald, the centre’s executive director, said, processed 150 pages every second. She explained that Arachnid was not the first of its kind. However, it absolutely scanned webpages faster than anything else. Other scanners exist for various reasons; Darpa, for instance created the current Memex search engine that hunts for the same type of material. Sarah Jamie Lewis created OnionScan that allowed researchers and investigators to gather information and statistics on darknet or darkweb sites. (And “to help operators of hidden services find and fix operational security issues with their services.”)

Developers created Arachnid to share sensitive content with law enforcement upon discovery. Interpol, McDonald said, already joined forces with The Canadian Centre for Child Protection. Additionally, on the clearnet, hosting services removed 90 percent of the flagged content within 48 hours.

John Carr, secretary of the UK Children’s Charities Coalition on Internet Safety, said: “This has the potential to be a game-changer. The Canadians have taken this to a whole new level and the early results are completely mind-blowing. There has never been anything like it before and Arachnid can work with equal facility on the dark web and the open web.”

The spider starts at sites already known to be hosting child pornography. It then “draws on a database of digital fingerprints of known abuse images” in a method much faster than previous crawlers with the same goal.

“We are never going to arrest our way out of the problem so we set out to choke off the supply. The consistent message we have had from survivors is that they are haunted by the idea these images are online and what they want most is their removal,” Ms. McDonald said. While acknowledging that the so-called cyberweapon officials behind the project said that it provided a roadmap of the problem that could be followed to put a stop to internet child pornography.