Networking issues causing packet drop or delays Wednesday 6th April 2022 08:23:00

Around 10.23 CET we discovered network disruptions towards our infrastructure. This caused packet loss or slowness on some routers/paths. After investigating we discovered issues at the Amsterdam Internet Exchange (AMS-ix),however, at 10.45 traffic recovered. We are waiting for clarifications from AMS-ix, meanwhile we monitor traffic closely and will reroute traffic if the problems at AMS-ix reoccur.

RFO (Reason for Outage) Event Report

  • Issue summary

    On 2nd April our hosting platform was subject of an outgoing DDoS attack that partially affected VPSes and hosting services.

  • Incident timelines

    • 2022-04-02 - 22:22 - Attack starts making some VPSes and hosting services unavailable.

    • 2022-04-02 - 23:15 - VPSes are fully available as well as e-mail, shared hosting websites are partially unavailable

    • 2022-04-03 - 00:50 - All shared hosting is up

    • 2022-04-03 - 09:00 - Small amount of shared hosting websites present unavailability due some changes in the infrastructure

    • 2022-04-03 - 19:37 - All shared hosting is up

  • Root cause

    A vulnerability on the wordpress tatsu plugin was exploited to launch a DNS amplification attack through vulnerable sites.

  • Affected services

    • Partially cloud virtual machines
    • Partially Hosting services
    • Partially Mail services

  • Corrective/Preventative actions

    • Upgrade the vulnerable plugin on the affected websites
    • Improve platform restrictions to avoid future attacks
    • Improve monitoring
Posted 2 years ago by Jesse

AMS-ix confirmed the exchange is currently stable. We will continue to monitor the connections closely.

Posted 2 years ago by Mart