Networking issues causing packet drop or delays

RFO (Reason for Outage) Event Report

• Issue summary

  On 2nd April our hosting platform was subject of an outgoing DDoS attack that partially affected VPSes and hosting services.

• Incident timelines

  • 2022-04-02 - 22:22 - Attack starts making some VPSes and hosting services unavailable.

  • 2022-04-02 - 23:15 - VPSes are fully available as well as e-mail, shared hosting websites are partially unavailable

  • 2022-04-03 - 00:50 - All shared hosting is up

  • 2022-04-03 - 09:00 - Small amount of shared hosting websites present unavailability due some changes in the infrastructure

  • 2022-04-03 - 19:37 - All shared hosting is up

• Root cause

  A vulnerability on the wordpress tatsu plugin was exploited to launch a DNS amplification attack through vulnerable sites.

• Affected services

  • Partially cloud virtual machines
  • Partially Hosting services
  • Partially Mail services

• Corrective/Preventative actions

  • Upgrade the vulnerable plugin on the affected websites
  • Improve platform restrictions to avoid future attacks
  • Improve monitoring

AMS-ix confirmed the exchange is currently stable. We will continue to monitor the connections closely.

Around 10.23 CET we discovered network disruptions towards our infrastructure. This caused packet loss or slowness on some routers/paths. After investigating we discovered issues at the Amsterdam Internet Exchange (AMS-ix),however, at 10.45 traffic recovered. We are waiting for clarifications from AMS-ix, meanwhile we monitor traffic closely and will reroute traffic if the problems at AMS-ix reoccur.