Since 10:10 CET, we have been experiencing outage caused by a major DDOS attack.
All VPSes, webhosting and emailservers were affected by this outage.
We have deflected the attack and worked to recover all systems.
VPS that required a reboot finished rebooting around 16:15 CET.
Email sent this morning should mostly have been delivered by now.
All webhosting came back online between 12:00 and 12:15
VPSes are back online for the larger part since approximately 12:30. Individual machines may need a reboot. We are still working on rebooting these machines.
Email has been fully operational again since approximately 12:30. No emails have been lost but, as a result of the outage, it may take a while for emails that were sent this morning to be delivered.
RFO (Reason for Outage)
Issue summary
Between December 17th to December and 20th we suffered several DDoS attack affecting Greenhost platform
availability at different levels.
Incident timelines
2020/12/17 - 09:10 UTC DDoS attack started
The attack was targeted at a Greenhost customer. The attack unexpectedly also affected also other parts of the network. A significant amount of Virtual Servers had to be rebooted afterwards, including some providing services for shared services.
Management and core infrastructure were not impacted.
10:24 UTC DDoS attack was deflected and network stabilized. Hosting and Cloud VPS were partially unavailable at this point. Mail was not available. Service center was working.
11:00-11:15 UTC Hosting platform coming back to capacity
11:30 UTC Mail fully operational starting to process the backlog of emails.
15:00-15:30 UTC Cloud virtual machines which didn't seem to be operational were brought up
2020/12/19 - 23:11-00:12 UTC DDoS attack
It may have caused unavailability of some systems or long latency on connections
2020/12/20 - 12:37-13:37 UTC DDoS attack
Systems were unavialable
2020/12/21 - 16:30-17:16 UTC Issue with DNS resolution
Systems were unavailable due to a change on our DNS setup which made DNS resolution not work properly
Root cause
The main cause was a large scale DDoS attack which targeted at least one of our
customers and after that towards Greenhost servers as well. This included the Greenhost website as well as our nameservers
were targeted.
Affected services
Cloud virtual machines
Hosting services
Mail services
Service center
Corrective/Preventative actions
Make DNS infrastructure more resilient and fault tolerant
Use DDoS mitigation
Parts of our network design will be adjusted to limit the
possibility of a distribution of the attack on the network.
