RFO (Reason for Outage) Thursday 13th June 2019 12:00:00

Issue summary

On June 10th, at around 15:20 UTC, the Greenhost platform suffered a DDoS attack. This caused serious network disruption for about 40 minutes which affected Greenhost services in different degree making some of them inaccessible.

Incident timelines
Root cause

The main cause was a large scale DDoS attack performed on at least one of our clients on the Greenhost platform. The platform was able to handle the attack however, caused some problems with a few services which were located on the same place as the attacked machine.

In order to mitigate the attack, it was decided to exclude a part of the network to be able to investigate and have a proper handling of the issue.

Unfortunately this lead the unexpected event where the attack traffic was distributed over the network and not discarded, which made multiple services (extremely) slow or unavailable (caused by Unknown Unicast traffic).

It took some time to find the root cause of the distribution of the attack. Eventually the network stabilized.

Affected services
Corrective/Preventative actions

The procedure for handling an attack like this has been reviewed and adjusted based on the experience of this attack.

Some minor parts of the network design will be adjusted to limit the possibility of a distribution of the attack on the network (limiting Unknown Unicast traffic).

Our public communication during this outage was not sufficient. We are updating our emergency communication strategy to be more comprehensive and timely.