Last week, KrebsOnSecurity detailed how BackConnect Inc. — a company that defends victims against large-scale distributed denial-of-service (DDoS) attacks — admitted to hijacking hundreds of Internet addresses from a European Internet service provider in order to glean information about attackers who were targeting BackConnect. According to an exhaustive analysis of historic Internet records, BackConnect appears to have a history of such “hacking back” activity.
On Sept. 8, 2016, KrebsOnSecurity exposed the inner workings of vDOS, a DDoS-for-hire or “booter” service whose tens of thousands of paying customers used the service to launch attacks against hundreds of thousands of targets over the service’s four-year history in business.
Within hours of that story running, the two alleged owners — 18-year-old Israeli men identified in the original report — were arrested in Israel in connection with an FBI investigation into the shady business, which earned well north of $600,000 for the two men.
In my follow-up report on their arrests, I noted that vDOS itself had gone offline, and that automated Twitter feeds which report on large-scale changes to the global Internet routing tables observed that vDOS’s provider — a Bulgarian host named Verdina[dot]net — had been briefly relieved of control over 255 Internet addresses (including those assigned to vDOS) as the direct result of an unusual counterattack by BackConnect.
Asked about the reason for the counterattack, BackConnect CEO Bryant Townsend confirmed to this author that it had executed what’s known as a “BGP hijack.” In short, the company had fraudulently “announced” to the rest of the world’s Internet service providers (ISPs) that it was the rightful owner of the range of those 255 Internet addresses at Verdina occupied by vDOS.
In a post on NANOG Sept. 13, BackConnect’s Townsend said his company took the extreme measure after coming under a sustained DDoS attack thought to have been launched by a botnet controlled by vDOS. Townsend explained that the hijack allowed his firm to “collect intelligence on the actors behind the botnet as well as identify the attack servers used by the booter service.”
Short for Border Gateway Protocol, BGP is a mechanism by which ISPs of the world share information about which providers are responsible for routing Internet traffic to specific addresses. However, like most components built into the modern Internet, BGP was never designed with security in mind, which leaves it vulnerable to exploitation by rogue actors.
BackConnect’s BGP hijack of Verdina caused quite an uproar among many Internet technologists who discuss such matters at the mailing list of the North American Network Operators Group (NANOG).
BGP hijacks are hardly unprecedented, but when they are non-consensual they are either done accidentally or are the work of cyber criminals such as spammers looking to hijack address space for use in blasting out junk email. If BackConnect’s hijacking of Verdina was an example of a DDoS mitigation firm “hacking back,” what would discourage others from doing the same, they wondered?
“Once we let providers cross the line from legal to illegal actions, we’re no better than the crooks, and the Internet will descend into lawless chaos,” wrote Mel Beckman, owner of Beckman Software Engineering and a computer networking consultant in the Los Angeles area. “BackConnect’s illicit action undoubtedly injured innocent parties, so it’s not self defense, any more than shooting wildly into a crowd to stop an attacker would be self defense.”
A HISTORY OF HIJACKS
Townsend’s explanation seemed to produce more questions than answers among the NANOG crowd (read the entire “Defensive BGP Hijacking” thread here if you dare). I grew more curious to learn whether this was a pattern for BackConnect when I started looking deeper into the history of two young men who co-founded BackConnect (more on them in a bit).
To get a better picture of BackConnect’s history, I turned to BGP hijacking expert Doug Madory, director of Internet analysis at Dyn, a cloud-based Internet performance management company. Madory pulled historic BGP records for BackConnect, and sure enough a strange pattern began to emerge.
Madory was careful to caution up front that not all BGP hijacks are malicious. Indeed, my DDoS protection provider — a company called Prolexic Communications (now owned by Akamai Technologies) — practically invented the use of BGP hijacks as a DDoS mitigation method, he said.
In such a scenario, an organization under heavy DDoS attack might approach Prolexic and ask for assistance. With the customer’s permission, Prolexic would use BGP to announce to the rest of the world’s ISPs that it was now the rightful owner of the Internet addresses under attack. This would allow Prolexic to “scrub” the customer’s incoming Web traffic to drop data packets designed to knock the customer offline — and forward the legitimate traffic on to the customer’s site.
Given that BackConnect is also a DDoS mitigation company, I asked Madory how one could reasonably tell the difference between a BGP hijack that BackConnect had launched to protect a client versus one that might have been launched for other purposes — such as surreptitiously collecting intelligence on DDoS-based botnets and their owners?
Madory explained that in evaluating whether a BGP hijack is malicious or consensual, he looks at four qualities: The duration of the hijack; whether it was announced globally or just to the target ISP’s local peers; whether the hijacker took steps to obfuscate which ISP was doing the hijacking; and whether the hijacker and hijacked agreed upon the action.
For starters, malicious BGP attacks designed to gather information about an attacking host are likely to be very brief — often lasting just a few minutes. The brevity of such hijacks makes them somewhat ineffective at mitigating large-scale DDoS attacks, which often last for hours at a time. For example, the BGP hijack that BackConnect launched against Verdina lasted a fraction of an hour, and according to the company’s CEO was launched only after the DDoS attack subsided.
Second, if the party conducting the hijack is doing so for information gathering purposes, that party may attempt to limit the number ISPs that receive the new routing instructions. This might help an uninvited BGP hijacker achieve the end result of intercepting traffic to and from the target network without informing all of the world’s ISPs simultaneously.
“If a sizable portion of the Internet’s routers do not carry a route to a DDoS mitigation provider, then they won’t be sending DDoS traffic destined for the corresponding address space to the provider’s traffic scrubbing centers, thus limiting the efficacy of any mitigation,” Madory wrote in his own blog post about our joint investigation.
Thirdly, a BGP hijacker who is trying not to draw attention to himself can “forge” the BGP records so that it appears that the hijack was performed by another party. Madory said this forgery process often fools less experienced investigators, but that ultimately it is impossible to hide the true origin of forged BGP records.
Finally, in BGP hijacks that are consensual for DDoS mitigation purposes, the host under attack stops “announcing” to the world’s ISPs that it is the rightful owner of an address block under siege at about the same time the DDoS mitigation provider begins claiming it. When we see BGP hijacks in which both parties are claiming in the BGP records to be authoritative for a given swath of Internet addresses, Madory said, it’s less likely that the BGP hijack is consensual.
Madory and KrebsOnSecurity spent several days reviewing historic records of BGP hijacks attributed to BackConnect over the past year, and at least three besides the admitted hijack against Verdina strongly suggest that the company has engaged in this type of intel-gathering activity previously. The strongest indicator of a malicious and non-consensual BGP hijack, Madory said, were the ones that included forged BGP records.
Working together, Madory and KrebsOnSecurity identified at least 17 incidents during that time frame that were possible BGP hijacks conducted by BackConnect. Of those, five included forged BGP records. One was an hours-long hijack against Ghostnet[dot]de, a hosting provider in Germany.
Two other BGP hijacks from BackConnect that included spoofed records were against Staminus Communications, a competing DDoS mitigation provider and a firm that employed BackConnect CEO Townsend for three years as senior vice president of business development until his departure from Staminus in December 2015.
“This hijack wasn’t conducted by Staminus. It was BackConnect posing as Staminus,” Dyn’s Madory concluded.
Two weeks after BackConnect hijacked the Staminus routes, Staminus was massively hacked. Unknown attackers, operating under the banner “Fuck ‘Em All,” reset all of the configurations on the company’s Internet routers, and then posted online Staminus’s customer credentials, support tickets, credit card numbers and other sensitive data. The intruders also posted to Pastebin a taunting note ridiculing the company’s security practices.
I asked Townsend to comment on the BGP hijacks identified by KrebsOnSecurity and Dyn as having spoofed source information. Townsend replied that he could not provide any insight as to why these incidents occurred, noting that he and the company’s chief technology officer — 24-year-old Marshal Webb — only had access and visibility into the network after the company BackConnect Inc. was created on April 27, 2016.
According to Townsend, the current BackConnect Inc. is wholly separate from BackConnect Security LLC, which is a company started in 2014 by two young men: Webb and a 19-year-old security professional named Tucker Preston. In April 2016, Preston was voted out of the company by Webb and Townsend and forced to sell his share of the company, which was subsequently renamed BackConnect Inc.
“Before that, the original owner of BackConnect Security LLC was the only one that had the ability to access servers and perform any type of networking commands,” he explained. “We had never noticed these occurred until this last Saturday and the previous owner never communicated anything regarding these hijacks. Wish I could provide more insight, but Marshal and I do not know the reasons behind the previous owners decision to hijack those ranges or what he was trying to accomplish.”
In a phone interview, Preston told KrebsOnSecurity that Townsend had little to no understanding about the technical side of the business, and was merely “a sales guy” for BackConnect. He claims that Webb absolutely had and still has the ability to manipulate BackConnect’s BGP records and announcements.
Townsend countered that Preston was the only network engineer at the company.
“We had to self-learn how to do anything network related once the new company was founded and Tucker removed,” he said. “Marshal and myself didn’t even know how to use BGP until we were forced to learn it in order to bring on new clients. To clarify further, Marshal did not have a networking background and had only been working on our web panel and DDoS mitigation rules.”
L33T, LULZ, W00W00 AND CHIPPY
Preston said he first met Webb in 2013 after the latter admitted to launching DDoS attacks against one of Preston’s customers at the time. Webb had been painted with a somewhat sketchy recent history at the time — being fingered as a low-skilled hacker who went by the nicknames “m_nerva” and “Chippy1337.”
Webb, whose Facebook alias is “lulznet,” was publicly accused in 2011 by the hacker group LulzSec of snitching on the activities of the group to the FBI, claiming that information he shared with law enforcement led to the arrest of a teen hacker in England associated with LulzSec. Webb has publicly denied being an informant for the FBI, but did not respond to requests for comment on this story.
LulzSec members claimed that Webb was behind the hacking of the Web site for the video game “Deus Ex.” As KrebsOnSecurity noted in a story about the Deus Ex hack, the intruder defaced the gaming site with the message “Owned by Chippy1337.”
I was introduced to Webb at the Defcon hacking convention in Las Vegas in 2014. Since then, I have come to know him a bit more as a participant of w00w00, an invite-only Slack chat channel populated mainly by information security professionals who work in the DDoS mitigation business. Webb chose the handle Chippy1337 for his account in that Slack channel.
At the time, Webb was trying to convince me to take another look at Voxility, a hosting provider that I’ve previously noted has a rather checkered history and one that BackConnect appears to rely upon exclusively for its own hosting.
In our examination of BGP hijacks attributed to BackConnect, Dyn and KrebsOnSecurity identified an unusual incident in late July 2016 in which BackConnect could be seen hijacking an address range previously announced by Datawagon, a hosting provider with a rather dodgy reputation for hosting spammers and DDoS-for-hire sites.
That address range previously announced by Datawagon included the Internet address 22.214.171.124, which is hacker “leet speak” for the word “leet,” or “elite.” Interestingly, on the w00w00 DDoS discussion Slack channel I observed Webb (Chippy1337) offering other participants in the channel vanity addresses and virtual private connections (VPNs) ending in 126.96.36.199. In the screen shot below, Webb can be seen posting a screen shot demonstrating his access to the 188.8.131.52 address while logged into it on his mobile phone.
THE MONEY TEAM
The Web address 184.108.40.206 currently does not respond to browser requests, but it previously routed to a page listing the core members of a hacker group calling itself the Money Team. Other sites also previously tied to that Internet address include numerous DDoS-for-hire services, such as nazistresser[dot]biz, exostress[dot]in, scriptkiddie[dot]eu, packeting[dot]eu, leet[dot]hu, booter[dot]in, vivostresser[dot]com, shockingbooter[dot]com and xboot[dot]info, among others.
The Money Team comprised a group of online gaming enthusiasts of the massively popular game Counterstrike, and the group’s members specialized in selling cheats and hacks for the game, as well as various booter services that could be used to knock rival gamers offline.
Datawagon’s founder is an 18-year-old American named CJ Sculti whose 15-minutes of fame came last year in a cybersquatting dispute after he registered the domain dominos.pizza. A cached version of the Money Team’s home page saved by Archive.org lists CJ at the top of the member list, with “chippy1337” as the third member from the top.
Asked why he chose to start a DDoS mitigation company with a kid who was into DDoS attacks, Preston said he got to know Webb over several years before teaming up with him to form BackConnect LLC.
“We were friends long before we ever started the company together,” Preston said. “I thought Marshal had turned over a new leaf and had moved away from all that black hat stuff. He seem to stay true to that until we split and he started getting involved with the Datawagon guys. I guess his lulz mentality came back in a really stupid way.”
Townsend said Webb was never an FBI informant, and was never arrested for involvement with LulzSec.
“Only a search warrant was executed at his residence,” Townsend said. “Chippy is not a unique handle to Marshal and it has been used by many people. Just because he uses that handle today doesn’t mean any past chippy actions are his doing. Marshal did not even go by Chippy when LulzSec was in the news. These claims are completely fabricated.”
As for the apparent Datawagon hijack, Townsend said Datawagon gave BackConnect permission to announce the company’s Internet address space but later decided not to become a customer.
“They were going to be a client and they gave us permission to announce that IP range via an LOA [letter of authorization]. They did not become a client and we removed the announcement. Also note that the date of the screen shot you present of Marshal talking about the 220.127.116.11. is not even the same as when we announced Datawagons IPs.”
SOMETHING SMELLS BAD
When vDOS was hacked, its entire user database was leaked to this author. Among the more active users of vDOS in 2016 was a user who went by the username “pp412” and who registered in February 2016 using the email address email@example.com.
The information about who originally registered the gnu.so domain has long been hidden behind WHOIS privacy records. But for several months in 2015 and 2016 the registration records show it was registered to a Tucker Preston LLC. Preston denies that he ever registered the gnu.so domain, and claims that he never conducted any booter attacks via vDOS. However, Preston also was on the w00w00 Slack channel along with Webb, and registered there using the email address firstname.lastname@example.org.
But whoever owned that pp412 account at vDOS was active in attacking a large number of targets, including multiple assaults on networks belonging to the Free Software Foundation (FSF).
Lisa Marie Maginnis, until very recently a senior system administrator at the FSF, said the foundation began evaluating DDoS mitigation providers in the months leading up to its LibrePlanet2016 conference in the third week of March. The organization had never suffered any real DDoS attacks to speak of previously, but NSA whistleblower Edward Snowden was slated to speak at the conference, and the FSF was concerned that someone might launch a DDoS attack to disrupt the streaming of Snowden’s keynote.
“We were worried this might bring us some extra unwanted attention,” she said.
Maginnis said the FSF had looked at BackConnect and other providers, but that it ultimately decided it didn’t have time to do the testing and evaluation required to properly vet a provider prior to the conference. So the organization tabled that decision. As it happened, the Snowden keynote was a success, and the FSF’s fears of a massive DDoS never materialized.
But all that changed in the weeks following the conference.
“The first attack we got started off kind of small, and it came around 3:30 on a Friday morning,” Maginnis recalled. “The next Friday at about the same time we were hit again, and then the next and the next.”
The DDoS attacks grew bigger with each passing week, she said, peaking at more than 200 Gbps — more than enough to knock large hosting providers offline, let alone individual sites like the FSF’s. When the FSF’s Internet provider succeeded in blacklisting the addresses doing the attacking, the attackers switched targets and began going after larger-scale ISPs further upstream.
“That’s when our ISP told us we had to do something because the attacks were really starting to impact the ISP’s other customers,” Maginnis said. “Routing all of our traffic through another company wasn’t exactly an ideal situation for the FSF, but the other choice was we would just be disconnected and there would be no more FSF online.”
In August, the FSF announced that it had signed up with BackConnect to be protected from DDoS attacks, in part because the foundation only uses free software to perform its work, and BackConnect advertises “open source DDoS protection and security,” and it agreed to provide the service without charge.
The FSF declined to comment for this story. Maginnis said she can’t be sure whether the foundation will continue to work with BackConnect. But she said the timing of the attacks is suspicious.
“The whole thing just smells bad,” she said. “It does feel like there could be a connection between the DDoS and BackConnect’s timing to approach clients. On the other hand, I don’t think we received a single attack until Tucker [Preston] left BackConnect.”
DDoS attacks are rapidly growing in size, sophistication and disruptive impact, presenting a clear and present threat to online commerce and free speech alike. Since reporting about the hack of vDOS and the arrest of its proprietors nearly two weeks ago, KrebsOnSecurity.com has been under near-constant DDoS attack. One assault this past Sunday morning maxed out at more than 210 Gbps — the largest assault on this site to date.
Addressing the root causes that contribute to these attacks is a complex challenge that requires cooperation, courage and ingenuity from a broad array of constituencies — including ISPs, hosting providers, policy and hardware makers, and even end users.
In the meantime, some worry that as the disruption and chaos caused by DDoS attacks continues to worsen, network owners and providers may be increasingly tempted to take matters into their own hands and strike back at their assailants.
But this is almost never a good idea, said Rich Kulawiec, an anti-spam activist who is active on the NANOG mailing list.
“It’s tempting (and even trendy these days in portions of the security world which advocate striking back at putative attackers, never mind that attack attribution is almost entirely an unsolved problem in computing),” Kulawiec wrote. “It’s emotionally satisfying. It’s sometimes momentarily effective. But all it really does [is] open up still more attack vectors and accelerate the spiral to the bottom.”
KrebsOnSecurity would like to thank Dyn and Doug Madory for their assistance in researching the technical side of this story. For a deep dive into the BGP activity attributed to BackConnect, check out Madory’s post, BackConnect’s Suspicious Hijacks.