Everything looks good for now. We think we found the root problem but are just confirming the cause. Closing this off for now.
Back off maintenance mode and testing
Offline temporarily making some changes to db cluster
Things seem to be stabilising. We ruled out the security patch. Need to do some further digging and find the root cause, but we think it may just be a case of "perfect storm".
We believe the performance issue is from the security patches implemented by the data centre earlier. We're working on a solution.
Still seeing instability and working on it still
Back online, we're monitoring and upgrading the status of this incident to degraded performance
Currently trying to isolate the reason for the instability and are temporarily offline, stay tuned!
We're experiencing some db stability issues and are working on a fix