We have identified this as DDoS Attack on our customer DNS services.

Our server engineers are currently working on redirecting these attacks and reviewing options to prevent this from happening again in the future.

Update @ 19:00 22/09/20: We are still working to mitigate this issue and roll out robust preventative measures, normal service will resume as soon as possible and we thank you for your patience.

Update @ 01:00 23/09/20: The attack has ceased and we've taken some steps to better protect some infrastructure. We will review the situation over the coming hours and make a report in due course. Please accept our apologies for any inconvenience.

Update @ 12:35 23/09/20: The attack has recommenced. We have deployed a secondary NS server as of yesterday evening in a different IP location, however, it appears the glue records are taking some time to propagate around the Internet. We are continuing to review and look at options.

Update @ 14:38 23/09/20: We concluded that the new DNS server built off-network wasn't serving requests fast enough and this was leading to fatal timeouts, despite the hardware being over-specified (on paper). We abandoned this server and built another server in a different location over the last two hours, and changed the DNS glue records for ns1.webhosting4southafrica.co.za and ns2.webhosting4southafrica.co.za again to this new server and are waiting for propagation to the wider Internet. ns1.webhosting4southafrica.co.za appears to be largely serving traffic to the Internet as of this update.

Update @ 15:27 23/09/20: The attack is ongoing, but DNS lookup services are restoring to availability as the new DNS glue records propagate around the Internet. Traffic levels appear nominal for the time of day (excluding attack bandwidth).

Update @ 16:24 23/09/20: The attack is ongoing, but the situation appears to be stable for customers using our standard DNS servers.

Update @ 20:47 23/09/20: The attack is ongoing, but has now also added our main website to the list of targets. Service to the cloud servers seems otherwise stable, ns1.webhosting4southafrica.co.za and ns1.webhosting4southafrica.co.za continue to provide service and egress traffic is roughly nominal for this time of the evening.

Update @ 14:18 24/09/20: The attack is ongoing, and we've had to relocate our main website IP again. Otherwise, customer services continue to improve and traffic is largely nominal for the time of day.

Update @ 14:51 24/09/20: Please also note that all our support ticket systems are working via e-mail - if you have any trouble with accessing the Client Portals then just send a request to the support address.

Update @ 12:00 25/09/20: The attack against our prior DNS services ceased around 01:00 and has not yet restarted, so we are now answering queries again - which is helping propagation where it has not already happened. We will not be reverting the change to ns1.webhosting4southafrica.co.za, and most issues which we are dealing with are DNS related. All servers are serving data, and traffic appears to be what would be expected on a Friday.

Update @ 20:00 25/09/20: The situation continues to be stable and no further DDoS activity has been observed since 01:00. All systems are operational and we not aware of any problems. We will observe the situation over the weekend and Monday, if there is no further activity then we will issue a full fault report via e-mail thereafter.

Update @ 21:00 26/09/20: There has been no recurrence of the attack, and we have continued to fix up some features which were slightly broken due to the infrastructure changes (e.g. VPS control panel, blog redirection, etc). Observed traffic remains nominal for the weekend.

We will update the status if there is any significant change.



Friday, September 25, 2020

« Back