This is a contributed piece from Rami Essaid, Co-Founder and CEO of Distil Networks

Almost 60% of today’s internet traffic is non-human, up more than 30% in the past year alone, and more than a third of that non-human bot traffic is malicious. The Distil Networks 2015 Bad Bot Landscape Report revealed that certain countries and service providers have become productive host environments for bot generators.

Bots are the key culprits behind web scraping, brute force attacks, competitive data mining, brownouts, account hijacking, unauthorized vulnerability scans, spam, man-in-the-middle attacks, and more. They place a huge tax on IT security and web infrastructure teams, and their variety, volume and sophistication wreak havoc across online operations big and small.

Mobile bots arrive in drives, beware of China

For the first time, the Android Webkit Browser appeared on the top five list of user agents leveraged by bad bots to hide their non-human identity at 4.87%. Mobile sites tend to be easier to scrape because they provide bots with more structured access to data.

China leads the world in bad bot mobile traffic at 30.64%, and the three mobile carriers with the highest percentage of bad bot traffic are all based in China. On this side of the world, for the first time we’re seeing a US mobile carrier (T-Mobile USA) appear in the top 20 list of ISPs serving bad bot traffic — 19.7% of the traffic was by bots.

Tracking “Bad Bot GDP”

While the United States is the source of more than 50% of bad bots with its thousands of low-cost hosting providers, absolute numbers can be misleading. Measuring the number of bad bots per online user provides another view into country-specific traffic risk; this is the number we’ve dubbed the Bad Bot GDP of a country.

As an example, the Maldives served up almost 16 bad bots per internet user in 2014. Of course, Bad Bot GDP numbers spike faster with smaller populations. In the summer of 2014, notorious Russian hacker Roman Seleznev was arrested in the Maldives and extradited to the US for allegedly stealing millions of dollars’ worth of credit card information.

Closer examination of the top three Bad Bot GDP countries helps justify their rankings:

• Singapore tops the list with a Bad Bot GDP of 152.87 bad bots per online user. According to the 2014 Global Information Technology Report by the World Economic Forum, Singapore has the best network-ready environment in Asia, so it’s a popular choice as a data center hub for China and Southeast Asia. Its small population relative to its well developed infrastructure boosts its Bad Bot GDP.
• Israel, with a Bad Bot GDP of 34.12, has a similarly small population and the most complete data center and internet infrastructure in the Middle East. A 2013 US National Intelligence Estimate on cyber threats ranked Israel the third most aggressive intelligence service against the U.S., behind only Russia and China.
• Slovenia, at 29.69, found itself in a similar situation to the Maldives. Slovenia was the site of a high-profile hacker arrest when Matjaz Skorjanc, the developer of the Mariposa botnet, was arrested there after his malware hijacked more than 12 million computers around the world.

China and Russia most blocked countries

For 2014, China and Russia were the most blocked countries. Geo-IP Fencing is an effective website security tactic for those organizations with well-defined geographical markets.

A more widely dispersed bad bot landscape

Bad bot threats have taken on less predictable patterns. Bots are now attacking from a more broadly dispersed set of global points of origin. 14 countries, almost double the number in 2013, originated at least 1% of bad bot traffic volume in 2014.

In 2013, the hour-by-hour bad bot data made it look like the attackers were waiting for IT personnel to leave work before launching their attacks. Not so for 2014, as attacks were much more evenly spread throughout the day.

Human or not? The bot dilemma

Some bad bots make little or no attempt to hide their identities, which makes them easy to spot using basic IP blocking or user agent integrity checks. Identifying bad bots becomes much more challenging when they mimic human behavior – which, at 41% of all bad bots tracked in 2014, is alarmingly high.

Mitigating bad bots in 2015 and beyond

The bad bot landscape is continuing to evolve rapidly with the dramatic growth in mobile bot traffic, increasingly sophisticated obfuscation techniques, and an expanding range of geographic and ISP points of origin. This is a clear challenge to IT security and web infrastructure teams under increasing pressure to forecast infrastructure demands and protect their online data. Without insight into bad traffic, the challenge is exacerbated