AWS - US East (Ohio): INC0113045
Incident Report for Snowflake
Postmortem

Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis (RCA) is available on the Snowflake Community site:
https://community.snowflake.com/s/article/INC0113045

Posted Aug 08, 2024 - 12:59 PDT

Resolved
Current status: We've implemented the fix for this issue and monitored the environment to confirm that service was restored. If you experience additional issues or have questions, please open a support case via Snowflake Community.

Customer experience: Customers hosted in the specified region(s) may have experienced an inability to execute scheduled tasks via automated or manual procedures.

Please note: In the Task History UI page, task executions impacted by this incident may appear in the table function for showing task graphs that are currently running (CURRENT_TASK_GRAPHS) even though they are not currently running, and those same executions may not appear in the table function for showing task graphs that are complete (COMPLETE_TASK_GRAPHS).

Additionally, tasks that were impacted by this incident may not have upcoming scheduled task entries in task history if the next scheduled execution is over 2 hours in the future.

Incident start time: 06:30 UTC July 30, 2024
Incident end time: 16:50 UTC July 30, 2024

Preliminary root cause: An error within a task maintenance activity resulted in task schedules becoming unavailable, preventing them from running as expected.

A root cause analysis (RCA) document will be published within seven business days.
Posted Jul 30, 2024 - 12:28 PDT
Update
Current status: We've implemented the fix for this issue, and we'll continue to monitor the environment until we're confident all services are functioning properly.

Customer experience: Customers hosted in the specified region(s) may have experienced an inability to execute scheduled tasks via automated or manual procedures.

Please note: In the Task History UI page, task executions impacted by this incident may appear in the table function for showing task graphs that are currently running (CURRENT_TASK_GRAPHS) even though they are not currently running, and those same executions may not appear in the table function for showing task graphs that are complete (COMPLETE_TASK_GRAPHS).

Additionally, tasks that were impacted by this incident may not have upcoming scheduled task entries in task history if the next scheduled execution is over 2 hours in the future.

Incident start time: 06:30 UTC July 30, 2024
Incident end time: 16:50 UTC July 30, 2024

Preliminary root cause: An error within a task maintenance activity resulted in task schedules becoming unavailable, preventing them from running as expected.
Posted Jul 30, 2024 - 11:49 PDT
Monitoring
Current status: We've implemented the fix for this issue, and we'll continue to monitor the environment until we're confident all services are functioning properly.

Customer experience: Customers hosted in the specified region(s) may experience an inability to execute scheduled tasks via automated or manual procedures.

Workaround: As an immediate workaround, customers are advised to recreate and replace any business critical tasks while Snowflake teams continue to actively work on restoring service.

Incident start time: 06:30 UTC July 30, 2024
Incident end time: 16:50 UTC July 30, 2024

Preliminary root cause: An error within a task maintenance activity resulted in task schedules becoming unavailable, preventing them from running as expected.
Posted Jul 30, 2024 - 10:52 PDT
Update
Current status: We've identified an issue with Snowflake Data Cloud, resulting in the reported inability to execute scheduled tasks, along with a mitigation plan. The execution of the mitigation plan is in progress. We'll provide an update within 2 hours.

Customer experience: Customers hosted in the specified region(s) may experience an inability to execute scheduled tasks via automated or manual procedures.

Workaround: As an immediate workaround, customers are advised to recreate and replace any business critical tasks while Snowflake teams continue to actively work on restoring service.

Incident start time: 06:30 UTC July 30, 2024
Posted Jul 30, 2024 - 08:38 PDT
Identified
Current status: We've identified an issue with Snowflake Data Cloud resulting in the reported inability to execute scheduled tasks and are working to mitigate this issue. We'll provide an update within 60 minutes.

Customer experience: Customers hosted in the specified region(s) may experience an inability to execute scheduled tasks via automated or manual procedures.

Workaround: As an immediate workaround, customers are advised to recreate and replace any business critical tasks while Snowflake teams continue to actively work on restoring service.

Incident start time: 06:30 UTC July 30, 2024
Posted Jul 30, 2024 - 07:23 PDT
Investigating
Current status: We're investigating an issue with Snowflake Data Cloud relating to issues with running scheduled tasks. We'll provide an update within 60 minutes.

Customer experience: Customers hosted in the specified region(s) may experience an inability to execute scheduled tasks via automated or manual procedures.

Workaround: As an immediate workaround, customers are advised to recreate and replace any business critical tasks while Snowflake teams actively work to restore service.
Posted Jul 30, 2024 - 06:40 PDT
This incident affected: AWS - US East (Ohio) (Snowpipe (Data Ingestion)).