This issue is fully resolved. We are caught up on data processing.
Posted Mar 01, 2024 - 04:22 UTC
Update
We had an issue processing data today. This issue would have led to some device requests receiving error responses between 18:15-19:00 UTC, as well as some errors processing data during the same period. You may see processing errors in the UI for that time period.
In addition, we queued a large amount of incoming data for later processing. We're currently churning through the backlog of queued data. We expect to be caught up in ~2 hours.
Posted Feb 29, 2024 - 21:42 UTC
Monitoring
A fix has been implemented and we're monitoring the results. We're churning through the backlog of data we've queued up, and expect to be caught up processing in the next few hours. No data has been lost and everything will eventually be processed.
Posted Feb 29, 2024 - 20:37 UTC
Investigating
Our data processing infrastructure is running behind. No data has been lost and the system should be caught up shortly.
Posted Feb 29, 2024 - 17:46 UTC
This incident affected: API & Dashboard and Ingress.