Increased Client Latencies
Incident Report for Layer
Resolved
This issue has remained resolved since our last update. Further monitoring has been put in place to identify health issues with our Cassandra cluster.
Posted 5 months ago. Aug 13, 2018 - 10:28 PDT
Monitoring
Removing offending Cassandra nodes and playbook triage against attached services have restored optimal latency. We're currently working on a RCA to prevent this from happening in the future.
Posted 5 months ago. Aug 13, 2018 - 09:47 PDT
Update
The team has removed unhealthy Cassandra nodes and is working to restore full performance to the system. Another update will follow shortly.
Posted 5 months ago. Aug 13, 2018 - 09:30 PDT
Identified
We're currently mitigating issues with websocket latency and client API timeouts in NA. We will update in the next 30 minutes with more information.
Posted 5 months ago. Aug 13, 2018 - 08:53 PDT
This incident affected: Messaging Service (US Cluster Messaging) and APIs (US Cluster Client API).