Heroku still has this marked as "Degraded" because they are still performing a root cause analysis but they believe the incident to be stabilized. We're going to mark this as resolved on our end but will continue to stay informed about the progress on their end
Posted Mar 17, 2020 - 07:39 PDT
Update
We continue to monitor this issue and will update here is anything changes but for now the related Chameleon APIs/applications are functioning normally (wrt performance and uptime)
Latest from Heroku => https://status.heroku.com/incidents/1974 "The investigation into the root cause of this incident continues. No new information to report."
This incident has reoccurred as of this post -- we're seeing CONNECTION_CLOSED errors on our own application servers as well as some internal Heroku services too (i.e. https://devcenter.heroku.com/) -- as a reminder we are on the latest Heroku stack and are up-to-date with security patches.
Posted Mar 16, 2020 - 12:53 PDT
Monitoring
Heroku appears to be investigating the issue => https://status.heroku.com/incidents/1973. We have seen our instances rotated to hew hardware and come back into connectivity. The current impact is a slight queue of events to process but no ongoing impact is detectable at the moment. Will be monitoring on a 1 minute granularity for connection issues
Posted Mar 16, 2020 - 09:31 PDT
Update
We are continuing to investigate this issue.
Posted Mar 16, 2020 - 09:24 PDT
Investigating
Our team have been alerted to issues related to our heroku instances cycling in and out of connectivity with the Heroku router. The impact thus far appears to be a CONNECTION_CLOSED error with no specific connection being made to our application backend.
Posted Mar 16, 2020 - 09:23 PDT
This incident affected: User profile API, Sidebar API, Data API, and Heroku.