Mitigating the Memcached DDoS Threat

Several security companies recently detected a series of massive UDP amplification attacks leveraging vulnerabilities in Memcached servers to speed up dynamic Web applications by caching data and objects in RAM.

Link11 security analysts dubbed the new DDoS attack vector “Memcached Reflection,” noting that the attacks are similar to DNS reflection. “The attackers exploit the free caching system’s poorly secured installations: it can be reached unsecured via UDP port 11211 for reading and writing data, as well as querying statistics,” Link11’s Oliver Adam wrote.

Cloudflare’s Marek Majkowski, who called the new attacks “Memcrashed,” noted that Memcashed is unfortunately well suited for these types of attacks. “The protocol specification shows that it’s one of the best protocols to use for amplification ever!” he enthused sarcastically. “There are absolutely zero checks, and the data WILL be delivered to the client, with blazing speed!”

In a blog post examining the threat, Nexusguard researchers wrote that at 51,000 times, the amplification effect achieved by these attacks greatly surpasses anything ever seen before. “To put into perspective how intimidating this new threat is, the 2016 attack on DNS provider DynDNS that knocked major Internet platforms and services in Europe and North America offline had an average amplification factor of 55,” they wrote.

GitHub acknowledged that the attack method was recently leveraged in the largest DDoS attack ever recorded, hitting GitHub.com this past Wednesday, February 28. “The attack originated from over a thousand different autonomous systems (ASNs) across tens of thousands of unique endpoints,” GitHub site reliability engineering manager Sam Kottler wrote. “It was an amplification attack using the memcached-based approach… that peaked at 1.35 Tbps via 126.9 million packets per second.”

Preventive measures

The threat continues to be significant. Cloudflare’s Majkowski noted that while just 5,729 unique source IPs of memcached servers have been detected so far, more than 88,000 open memcached servers can be found easily via Shodan.

“If you are using memcached, please disable UDP support if you are not using it,” Majkowski wrote. “On memcached startup you can specify –listen 127.0.0.1 to listen only to localhost and -U 0 to disable UDP completely.”

In general, Majkowski begged all developers simply to stop using UDP. “If you must, please don’t enable it by default,” he wrote, adding, “If you do not know what an amplification attack is, I hereby forbid you from ever typing SOCK_DGRAM into your editor.”

Arbor Networks principal engineer Roland Dobbins wrote in a blog post that it’s crucial for network operators to “ensure they are prepared to detect, classify, traceback, and mitigate these attacks, as well as ensure that any memcached installations on their networks and/or networks of their end customers cannot be exploited as reflectors/amplifiers.”

“The first step in securing something is understanding it,” ExtraHop vice president of security Matt Cauthorn told eSecurity Planet?by email. “Having the ability to see traffic patterns, transactions, and service configurations from the perspective of the network is a crucial step in understanding and validating the behaviors of systems and the services they expose.”

Synopsys principal scientist Sammy Migues said by email that there are three key steps every operator of memcached servers should take to mitigate the threat:

1. Ensure your memcached server is not exposed to the Internet.
2. In every perimeter-facing firewall you have, immediately block all access from the Internet to UDP port 11211.
3. Disable UDP on all memcached servers.

“On a more macro level, ISPs need to block spoofed packets from exiting their networks, and protocol developers need to better understand velocity checking and amplification attacks,” Migues added.