As with the 2008 attack on YouTube by a Pakistani ISP, it is clear that there are fundamental problems in eBGP.
These become clear due to the trust relationships of the "autonomous systems" (AS). In the case of Facebook, it is a) poor change management and a simple configuration error of a pushed route in eBGP or b) an insider attack in combination with variant a.
On October 04.10.2021th, 17 at approx. 51:XNUMX p.m. German time, Facebook could no longer be found on the Internet. In fact, the DNS servers returned an error.
Cloudflare analyzed this and confirmed the reports on social media that Facebook and its associated services were no longer available.
The BGP is a routing protocol that runs in the large routers that make the Internet the Internet. It transmits the information from the autonomous systems such as Facebook, Telekom and Google, which have one or more AS numbers, to other BGP routers so that they know how to reach the systems.
A schematic representation of how it can work.
Externally, Facebook no longer attached the routes to their DNS entries. This means that at least the Facebook DNS servers were no longer accessible.
Cloudflare stores all the BGP updates they get and at 17:40 p.m. a lot of route updates came from Facebook.
The times in the image are UTC. Cloudflare can distinguish between adding and removing routes and here is the graphic:
You can see very well that a lot of routes have been removed. With the changes, Facebook has taken itself off the internet. The consequence of this was that the DNS servers could no longer resolve Facebook and its services.
Due to the fact that the services were no longer available, the DNS requests increased exponentially because you couldn't believe that Facebook was offline. Many apps didn't give any error messages either, so we tried again. The DNS queries were about 30 times higher than normal. Since Facebook was unavailable, there was an increase in inquiries to the other social networks.
At about 23:20 German time new BGP updates came and Facebook slowly came back online. The services themselves probably took a little longer, but then they made it.
The failure due to a possible configuration error or possible attack is visually clear again on Ripe's BGP Play. Here you can see that shortly before the failure, an eBGP message from the AS 32934 caused the entire AS 32934 to be cut. The AS numbers can be found in the Peering Info of Facebook Inc. under: https://www.facebook.com/peering/.
In a message it was announced that the outage was due to a faulty change in Facebook's own backend.
