If there’s one thing bots are good for, it’s crawling websites on behalf of search engines and social media platforms to help those sites gain visibility. But if there’s two things bots are good for, it’s that, and helping people who don’t know much about internet security sound like they do. When in doubt, blame it on bots. Someone’s website went down for two days to a DDoS attack? Darn bots. Someone else’s website is getting inundated with spam? Geez, bots strike again. And someone else’s website got its content scraped? Come on bots, take a break.
The reason it’s so easy to blame everything on bots is because there are so many bots out there doing so much stuff on the internet that they’re involved in pretty much everything. Most people have no idea how pervasive, busy and even malicious bots can be. But for the good of websites everywhere, that has to change. Here are five facts about bots that will open your eyes to what’s really happening on the internet and on your website.
Fact 1: Your website is very popular with the bot demographic.
Bots both good and bad are busy little buggers, and they account for between 56 and 80% of all website traffic. “But wait,” you may be thinking, “I run a small custom cat sweater website. There’s no way I’m getting that much bot traffic.” Here’s a couple pieces of bad news for you. The first is that you run a custom cat sweater website. The second is that it’s those small sites with up to 1000 visitors per day that are getting 80.3% bot traffic. Medium-sized sites with up to 10,000 visitors are getting 63.2% bot traffic, and large sites with a million plus daily visitors are getting 56.2%.
Even though a large number of those bots are good bots, that’s still a big chunk of your traffic being accounted for by non-humans. This isn’t to say that you can no longer brag about your traffic numbers. Hey, go ahead. Just be aware that you aren’t getting your site in front of as many actual eyeballs as you previously thought, and keep that in mind when you’re laying out your marketing strategies.
Fact 2: Your website is more popular with the bad bot demographic than the good.
Now that you’ve absorbed the shock of how much of your traffic isn’t coming from actual people, here’s another kick in the pants: 29% of all website traffic comes from bad bots. Good bots make up about 27% of all traffic. No matter if your site falls into the above small, medium or large categories, nearly one-third of all your traffic comes from malicious bots. Malicious bots are responsible for things like DDoS floods, hackings, click fraud campaigns, content scraping and spam.
Fact 3: The only kind of bots consistently displaying growth are the worst ones.
With bots accounting for so much web traffic, you wouldn’t be misguided in thinking that bots must be constantly growing. However, you would be incorrect. Overall bot activity actually declined 10% from 2013 to 2014. There is one category of bots that have displayed growth, though, and those would be impersonator bots, the most advanced malevolent bots.
While overall bot activity was declining from 2013 to 2014, impersonator bot traffic grew almost 10% in the same period. Impersonator bots are the only type of bot that have displayed growth consistently over the last three years. So what exactly are impersonator bots? They’re bots that pretend to be something they aren’t in order to bypass website security and wreak havoc, like launching DDoS attacks. Impostor Google bots are common, and other types include bots that imitate browser capabilities, spy bots, bots disguised by proxy servers, and of course DDoS bots.
Source + Hi-Res image: Incapsula 2014 Bot traffic report
Fact 4: Impostor Google bots are the third most frequently used bots in DDoS attacks.
Everyone with a website wants to rank high in Google, and that means Google’s own bots are given unfettered access to any website they visit. Much like how celebrities are given special treatment everywhere they go. The thing is, sometimes it’s only someone pretending to be Bradley Cooper that placed that order for a complimentary bottle of $1000 champagne. And sometimes it’s an impostor posing as a Google bot to creep and crawl all over your website. So while your website is designed to roll out the welcome mat for Google bots, it’s also designed to do the same for convincing and dangerous impostors. In fact, internet security firm Incapsula found that a full 4% of all bots that appear to be Google bots are, in fact, impostors. It only gets worse from there: the same Incapsula study found that 23.5% of those Google impostors are being used in Layer 7 DDoS attacks.
Fact 5: Not everyone involved in a botnet has bad intentions.
A botnet is pretty much what it sounds like: a large grouping of bots organized by attackers to inflict major damage on a target website. Using that broad definition, it’s easy to paint everyone involved with the same brush. However, most people if not everyone whose computers have become involved in a botnet don’t have bad intentions. For that matter, most of the people involved in a botnet don’t have any sort of intention at all. That’s because they probably don’t even know they’re involved in a botnet.
Within the internet security community, botnets are often called zombie armies. That’s because a botnet is a group of computers that have been taken over for malicious purposes, typically with the use of malware. Without the computer owner even knowing, their machine is controlled by an external source and used for malicious reasons. DDoS attacks are a common botnet activity.
So with all that said, you see why blaming it on the bots is such a sound strategy. They’re incredibly common, incredibly busy, and they can be incredibly malicious.