Editor and tour assets not loading
Incident Report for Chameleon

We’d like to provide a detailed review of the cause of this incident. It was caused by an unstable connection and limited failover between Chameleon’s CDN nodes and their backing Redis database.

To resolve the issue Chameleon engineers temporarily redirected traffic to a different CDN node, and rebuild the CDN nodes that were causing issue. The redirection took approximately 60 seconds and the rebuild completed in around 45 minutes.

To prevent future issues, the Chameleon engineering team has added specific alerting for this issue and new automation to redirect traffic in the event of issues in individual CDN notes. Prior to this incident, our engineers had started migrating away from our current CDN provider to a new (more robust) service not dependent on a real-time database. This work is continuing, and when completed (in the coming weeks), it will remove the risk of this issue recurring again. Additionally this upcoming change will also have the benefit of being more performant and cacheable for end-users.

Once again, we would like to thank any Chameleon customers for their patience and understanding as we resolved this issue.

Posted Jul 12, 2019 - 15:53 PDT

Chameleon experienced brief downtime today due to a connection error between our CDN servers. This issue was resolved at approximately 5:30pm PST

This, along with other recent downtime events, has prompted us to prioritize updates to our infrastructure and seek new resources in providing more reliable services.

We apologize for any inconvenience this downtime may have caused and thank any Chameleon customers for their patience while we resolved this issue.
Posted Jun 28, 2019 - 19:14 PDT
Chameleon's Editor and tour javascript is currently not loading in North America and European regions.
Posted Jun 28, 2019 - 14:52 PDT
This incident affected: JavaScript CDN and Sidebar API.