AWS - US East (N. Virginia): INC0107390
Incident
Report for Snowflake
Postmortem
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis (RCA) is available on the Snowflake Community site: https://community.snowflake.com/s/article/INC0107390
Resolved
Current status: We've coordinated with our third-party service provider to implement the fix for this issue, and we've monitored the environment to confirm that service was restored. If you experience additional issues or have questions, please open a support case via Snowflake Community.
Customer experience: A subset of customers hosted in the specified region would have experienced intermittent delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Incident end time: 22:40 UTC April 29, 2024
Preliminary root cause: The issue originated from a recent disruption in our cloud service provider's storage service, which affected queries. After resolving this disruption, service utilization in the region increased due to the resubmission of the affected queries, which led to warehouse resume delays.
Customer experience: A subset of customers hosted in the specified region would have experienced intermittent delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Incident end time: 22:40 UTC April 29, 2024
Preliminary root cause: The issue originated from a recent disruption in our cloud service provider's storage service, which affected queries. After resolving this disruption, service utilization in the region increased due to the resubmission of the affected queries, which led to warehouse resume delays.
Update
Current status: Since 22:40 UTC, there has been a significant improvement in warehouse resumes. Our engineering team is continuously monitoring the environment until we're confident all services are functioning properly.
Customer experience: A subset of customers hosted in the specified region would have experienced intermittent delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Incident end time: 22:40 UTC April 29, 2024
Preliminary root cause: The issue originated from a recent disruption in our cloud service provider's storage service, which affected queries. After resolving this disruption, service utilization in the region increased due to the resubmission of the affected queries, which led to warehouse resume delays.
Customer experience: A subset of customers hosted in the specified region would have experienced intermittent delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Incident end time: 22:40 UTC April 29, 2024
Preliminary root cause: The issue originated from a recent disruption in our cloud service provider's storage service, which affected queries. After resolving this disruption, service utilization in the region increased due to the resubmission of the affected queries, which led to warehouse resume delays.
Monitoring
Current status: Telemetry indicates intermittent connectivity issues have significantly subsided as of 22:40 UTC, thus alleviating the issue. We continue to monitor virtual warehouse resume delays and will provide updates on the situation until services return to normal operational levels.
Customer experience: A subset of customers hosted in the specified region would have experienced intermittent delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Incident end time: 22:45 UTC April 29, 2024
Preliminary root cause: The issue arose following a recent disruption in our cloud service provider's storage service, which impacted some queries. Subsequent to addressing this disruption, service utilization in the region increased due to the resubmission of the previously affected queries. This led to intermittent delays in warehouse operations resuming fully.
Customer experience: A subset of customers hosted in the specified region would have experienced intermittent delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Incident end time: 22:45 UTC April 29, 2024
Preliminary root cause: The issue arose following a recent disruption in our cloud service provider's storage service, which impacted some queries. Subsequent to addressing this disruption, service utilization in the region increased due to the resubmission of the previously affected queries. This led to intermittent delays in warehouse operations resuming fully.
Update
Current status: We're continuing to investigate to determine the source of an increase in intermittent connectivity issues within the region. We'll provide another update within 2 hours.
Customer experience: Customers hosted in the specified region may experience intermittent delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Customer experience: Customers hosted in the specified region may experience intermittent delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Investigating
Current status: We're investigating an issue with Snowflake Data Cloud. We'll provide an update within 60 minutes.
Customer experience: Customers hosted in the specified region may experience delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
Customer experience: Customers hosted in the specified region may experience delays while attempting to resume virtual warehouses or execute queries.
Incident start time: 18:16 UTC April 29, 2024
This incident affected: AWS - US East (N. Virginia) (Snowflake Data Warehouse (Database)).