Konnected Cloud outage due to AWS IoT outage

Incident Report for Konnected

Postmortem

RESOLVED: Konnected Cloud Outage

This is a follow-up to yesterday's AWS outage that severely impacted Konnected Cloud. Many devices in North America that are connected to SmartThings via our cloud service built on AWS IoT were offline for much of the day yesterday, November 25.

As of 7:00am EST today (November 26), AWS has confirmed that the underlying problem is resolved and Konnected Cloud infrastructure is operating normally now.

If your devices are still showing "offline" in SmartThings, a reboot may be required. You can reboot your devices using one of the following methods:

Press the small REBOOT or RST button on the device itself.
Briefly un-plug and re-plug the power to power-cycle the device.
Using the Konnected app (on the same WiFi network as the devices) tap on "View Device Status" and then tap the reboot button at the bottom of the page.
Temporarily disable your WiFi network to induce a self-reboot.

We regret that intervention is required to get the devices back online. In principle, our devices are designed to re-connect and recover from temporary outages and errors to ensure consistent operation. However, this particular failure with AWS IoT was not known before, and unfortunately our devices were not able to recognize this as a failure to self-recover.

We will be addressing this particular failure case in a future device software update.

Thank you for your understanding and Happy Thanksgiving!

Posted Nov 26, 2020 - 10:03 EST

Resolved

AWS is reporting that the all issues are resolved, and the Konnected Cloud service is now operating normally. If your device(s) are still offline, they may require a reboot or power-cycle to reconnect.

Posted Nov 26, 2020 - 08:34 EST

Update

We are continuing to monitor for any further issues.

Posted Nov 26, 2020 - 08:32 EST

Update

The widspread AWS outage caused a number of different issues throughout the day, and now many devices are offline. We've seen some signs of recovery, but many devices are still offline. Unfortunately due to the nature of this odd connectivity outage, the Konnected device may not reconnect on its own. A reboot or power-cycle of the device may be required to get it back online.

Posted Nov 25, 2020 - 23:40 EST

Update

Posted Nov 25, 2020 - 21:22 EST

Update

Amazon Web Services continues to report issues with AWS IoT connectivity. Many Konnected devices that use our cloud service to communicate with SmartThings are offline. There is no resolution at this time. We are closely monitoring the situation.

Posted Nov 25, 2020 - 21:14 EST

Monitoring

Konnected devices are currently unable to connect to the Konnected Cloud service built on AWS IoT due to a widespread AWS outage. This is affecting all users in the North America region using Konnected devices connected to SmartThings via our cloud service and app.

We're actively monitoring the issue and will communicate updates here.

Posted Nov 25, 2020 - 10:00 EST

This incident affected: Konnected Cloud.