Increased Client Latencies
Incident Report for Layer
Resolved
This issue has remained resolved since our last update. Further monitoring has been put in place to identify health issues with our Cassandra cluster.
Posted about 1 year ago. Aug 13, 2018 - 10:28 PDT
Monitoring
Removing offending Cassandra nodes and playbook triage against attached services have restored optimal latency. We're currently working on a RCA to prevent this from happening in the future.
Posted about 1 year ago. Aug 13, 2018 - 09:47 PDT
Update
The team has removed unhealthy Cassandra nodes and is working to restore full performance to the system. Another update will follow shortly.
Posted about 1 year ago. Aug 13, 2018 - 09:30 PDT
Identified
We're currently mitigating issues with websocket latency and client API timeouts in NA. We will update in the next 30 minutes with more information.
Posted about 1 year ago. Aug 13, 2018 - 08:53 PDT
This incident affected: APIs (US Cluster Client API) and Messaging Service (US Cluster Messaging).