Multiple AZURE Regions: SI-20210315
Incident Report for Snowflake
Postmortem

Dear Customer,
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis is made available on the Snowflake Community site.

https://community.snowflake.com/s/article/SI-20210315

We apologize for the service incident and the impact on your applications.

If you have any questions or difficulty accessing the Snowflake community site, please send feedback by submitting a support case or calling Snowflake Technical Support.

Thank you,
Snowflake Support

Posted Mar 31, 2021 - 12:54 PDT

Resolved
The Service Incident is now closed.

A Root Cause Analysis (RCA) will be posted within the next ten business days.

We apologize for the inconvenience caused by this incident.

If you have any questions or experience any related issues, please open a support case via the Snowflake Community Site.
Posted Mar 16, 2021 - 05:19 PDT
Monitoring
We applied a fix to resolve the issue.

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021
Incident End Time: 03:43 PT March 16, 2021

We are now monitoring the system for any further recurrence of the problem. We will remain in this state for 60 minutes.

If you have any questions or experience any related issues, please open a support case via the Snowflake Community Site.
Posted Mar 16, 2021 - 04:25 PDT
Update
The issue related to AZURE Active Directory has been resolved. The Snowflake Data Warehouse and Snowpipe services are operating normally.

We continue to work with our Cloud Service Provider AZURE on the Replication services, which is impacted by the AZURE Key Vault and Storage Services.

Affected Services: Replication
Incident Start Time: 12:15 PT March 15, 2021

Additional details can be found at this link: https://status.azure.com/en-us/status.

We will update the status as soon as the issue with Replication service is resolved or as events warrant.
Posted Mar 16, 2021 - 00:34 PDT
Update
The issue related to AZURE Active Directory has been resolved. The Snowflake Data Warehouse and Snowpipe services are operating normally.

We continue to work with our Cloud Service Provider AZURE on the Replication services, which is impacted by the AZURE Key Vault and Storage Services.

Affected Services: Replication
Incident Start Time: 12:15 PT March 15, 2021

Additional details can be found at this link: https://status.azure.com/en-us/status.

We will update the status as soon as the issue with Replication service is resolved or will provide an update within 60 minutes.
Posted Mar 15, 2021 - 23:31 PDT
Update
The issue related to AZURE Active Directory has been resolved. The Snowflake Data Warehouse and Snowpipe services are operating normally.

We continue to work with our Cloud Service Provider AZURE on the Replication services, which is impacted by the AZURE Key Vault and Storage Services.

Affected Services: Replication
Incident Start Time: 12:15 PT March 15, 2021

Additional details can be found at this link: https://status.azure.com/en-us/status.

We will update the status as soon as the issue with Replication service is resolved or will provide an update within 60 minutes.
Posted Mar 15, 2021 - 22:30 PDT
Update
The issue related to AZURE Active Directory has been resolved. The Snowflake Data Warehouse and Snowpipe services are operating normally.

We continue to work with our Cloud Service Provider AZURE on the Replication services, which is impacted by the AZURE Key Vault and Storage Services.

Affected Services: Replication
Incident Start Time: 12:15 PT March 15, 2021

Additional details can be found at this link: https://status.azure.com/en-us/status.

We will update the status as soon as the issue with Replication service is resolved or will provide an update within 60 minutes.
Posted Mar 15, 2021 - 21:32 PDT
Update
The issue related to AZURE Active Directory has been resolved. The Snowflake Data Warehouse and Snowpipe services are operating normally.

We continue to work with our Cloud Service Provider AZURE on the Replication services, which is impacted by the AZURE Key Vault.

Affected Services: Replication
Incident Start Time: 12:15 PT March 15, 2021

Additional details can be found at this link: https://status.azure.com/en-us/status.

We will update the status as soon as the issue with Replication service is resolved or will provide an update within 60 minutes.
Posted Mar 15, 2021 - 20:31 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We continue to work with our Cloud Service Provider AZURE on mitigating this issue. Additional details can be found at this link: https://status.azure.com/en-us/status.

Microsoft AZURE has rolled out a fix to all affected regions for Azure AD and Microsoft's services are in the process of recovery. Microsoft reports that various downstream Azure services continue their recovery actions following the completed rollout of the fix. Microsoft recommends that customers with custom applications that use Azure AD refresh their metadata cache.

We will update the status as soon as the issue is resolved or provide an update within 60 minutes.
Posted Mar 15, 2021 - 19:15 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We continue to work with our Cloud Service Provider AZURE on mitigating this issue. Additional details can be found at this link: https://status.azure.com/en-us/status.

Microsoft AZURE has rolled out a fix to all affected regions for Azure AD. Microsoft services are in the process of recovery. Microsoft reports that various downstream Azure services continue their recovery actions following the completed rollout of the fix. Microsoft recommends that customers with custom applications that use Azure AD refresh their metadata cache.

We will update the status as soon as the issue is resolved or provide an update within 60 minutes.
Posted Mar 15, 2021 - 18:18 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We continue to work with our Cloud Service Provider AZURE on mitigating this issue. Additional details can be found at this link: https://status.azure.com/en-us/status.

Microsoft AZURE has rolled out a fix to all affected regions for Azure AD. Microsoft services are in the process of recovery. Some services and offerings may have varying times of recovery following the underlying fix. Microsoft recommends that customers with custom applications that use Azure AD refresh their metadata cache.

We will update the status as soon as the issue is resolved or provide an update within 60 minutes.
Posted Mar 15, 2021 - 17:12 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We continue to work with our Cloud Service Provider AZURE on mitigating this issue. Additional details can be found at this link: https://status.azure.com/en-us/status.

Microsoft AZURE has rolled out a fix to all affected regions for Azure AD. Microsoft services are in the process of recovery. Some services and offerings may have varying times of recovery following the underlying fix. Microsoft recommends that customers with custom applications that use Azure AD refresh their metadata cache.

We will update the status as soon as the issue is resolved or provide an update within 60 minutes.
Posted Mar 15, 2021 - 16:14 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We continue to work with our Cloud Service Provider AZURE on mitigating this issue. Additional details can be found at this link: https://status.azure.com/en-us/status.

AZURE continues to roll out mitigation measures.

We will update the status as soon as the issue is resolved or provide an update within 60 minutes.
Posted Mar 15, 2021 - 15:16 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We continue to work with our Cloud Service Provider AZURE on mitigating this issue. Additional details can be found at this link: https://status.azure.com/en-us/status.

AZURE continues to roll out mitigation measures.
Posted Mar 15, 2021 - 14:47 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We continue to work with our Cloud Service Provider AZURE on mitigating this issue. Additional details can be found at this link: https://status.azure.com/en-us/status.

We have received an update from AZURE stating they are rolling out mitigation measures and that US Gov Virginia (Virginia) is NOT impacted.

We will update the status as soon as the issue is resolved or provide an update within 30 minutes.
Posted Mar 15, 2021 - 14:15 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We are working with our Cloud Service Provider AZURE on this issue. Additional details can be found at this link: https://status.azure.com/en-us/status

We will update the status as soon as the issue is resolved or provide an update within 30 minutes.
Posted Mar 15, 2021 - 14:00 PDT
Update
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We continue to work with our Cloud Service Provider AZURE on this issue. Additional details can be found at this link: https://status.azure.com/en-us/status

We will update the status as soon as the issue is resolved or provide an update within 30 minutes.
Posted Mar 15, 2021 - 13:34 PDT
Identified
We have identified the problem with the Snowflake Service that is causing the following symptom(s):

Symptom(s): Cannot connect to Snowflake by Active Directory connector
Incident Start Time: 12:15 PT March 15, 2021

We are working with our Cloud Service Provider AZURE on this issue. Additional details can be found at this link: https://status.azure.com/en-us/status

We will update the status as soon as the issue is resolved or provide an update within 30 minutes.
Posted Mar 15, 2021 - 13:09 PDT
This incident affected: AZURE - West Europe (Netherlands) (Snowflake Data Warehouse (Database), Snowpipe (Data Ingestion), Replication), AZURE - US East 2 (Virginia) (Snowflake Data Warehouse (Database), Snowpipe (Data Ingestion), Replication), AZURE - Switzerland North (Snowflake Data Warehouse (Database), Snowpipe (Data Ingestion), Replication), AZURE - Australia East (New South Wales) (Snowflake Data Warehouse (Database), Snowpipe (Data Ingestion), Replication), AZURE - US West2 (Snowflake Data Warehouse (Database), Snowpipe (Data Ingestion), Replication), AZURE - US Gov Virginia (Virginia) (Snowflake Data Warehouse (Database), Snowpipe (Data Ingestion), Replication), AZURE - Canada Central (Toronto) (Snowflake Data Warehouse (Database), Snowpipe (Data Ingestion), Replication), and AZURE - Southeast Asia (Singapore) (Snowflake Data Warehouse (Database), Snowpipe (Data Ingestion), Replication).