13 Deployments Affected: MI-20220525
Incident
Report for Snowflake
Postmortem
Dear Customer,
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis (RCA) is available on the Snowflake Community site.
https://community.snowflake.com/s/article/MI-20220525
We apologize for the service incident and the impact on your applications.
If you have any questions or difficulty accessing the Snowflake community site, please send feedback by submitting a support case or calling Snowflake Technical Support.
Resolved
Current status: We've implemented the fix for this issue and monitored the environment to confirm that service was restored. If you experience additional issues or have questions, please open a support case via Snowflake Community.
Customer experience: Customers hosted in the specified regions may have experienced delays or failures while performing Snowpipe data ingestion workflows that require the copy_history function. If the timespan set for the copy_history query is between May 9, 2022, and May 25, 2022, at 06:30 PT, then your queries may continue to be impacted. If the timespan is set for a period of time that does not query copy_history prior to when the mitigation was implemented at 06:30 PT, then your queries are no longer impacted.
Workaround: As a workaround, you can alter the timespan of your queries to a shorter time frame (e.g., one or two hours).
Incident start time: 03:37 PT May 25, 2022
Incident end time: 06:30 PT May 25, 2022
A root cause analysis (RCA) document will be published within seven business days.
Customer experience: Customers hosted in the specified regions may have experienced delays or failures while performing Snowpipe data ingestion workflows that require the copy_history function. If the timespan set for the copy_history query is between May 9, 2022, and May 25, 2022, at 06:30 PT, then your queries may continue to be impacted. If the timespan is set for a period of time that does not query copy_history prior to when the mitigation was implemented at 06:30 PT, then your queries are no longer impacted.
Workaround: As a workaround, you can alter the timespan of your queries to a shorter time frame (e.g., one or two hours).
Incident start time: 03:37 PT May 25, 2022
Incident end time: 06:30 PT May 25, 2022
A root cause analysis (RCA) document will be published within seven business days.
Monitoring
Current status: We've implemented the fix for this issue, and we'll continue to monitor the environment until we're confident all services are functioning properly.
Customer experience: Customers hosted in the specified regions may have experienced delays or failures while performing Snowpipe data ingestion workflows that require the copy_history function. If the timespan set for the copy_history query is between May 9, 2022, and May 25, 2022, at 06:30 PT, then your queries may continue to be impacted. If the timespan is set for a period of time that does not query copy_history prior to when the mitigation was implemented at 06:30 PT, then your queries are no longer impacted.
Workaround: As a workaround, you can alter the timespan of your queries to a shorter time frame (e.g., one or two hours).
Incident start time: 03:37 PT May 25, 2022
Customer experience: Customers hosted in the specified regions may have experienced delays or failures while performing Snowpipe data ingestion workflows that require the copy_history function. If the timespan set for the copy_history query is between May 9, 2022, and May 25, 2022, at 06:30 PT, then your queries may continue to be impacted. If the timespan is set for a period of time that does not query copy_history prior to when the mitigation was implemented at 06:30 PT, then your queries are no longer impacted.
Workaround: As a workaround, you can alter the timespan of your queries to a shorter time frame (e.g., one or two hours).
Incident start time: 03:37 PT May 25, 2022
Identified
Current status: The investigation determined that a recent configuration update caused the issue. We implemented a solution at approximately 06:30 PT; however, Snowpipe queries involving copy_history may continue to be impacted if you are using a timespan that queries copy_history prior to the mitigation at 06:30 PT. We're monitoring the environment to confirm recovery, and we're investigating any potential options for resolving issues with copy_history query timespans that involve time ranges prior to 06:30 PT.
Customer experience: Customers hosted in the specified regions may experience delays or failures while performing Snowpipe data ingestion workflows that require the copy_history function. If the timespan set for the copy_history query is between May 9, 2022, and May 25, 2022, at 06:30 PT, then your queries may continue to be impacted. If the timespan is set for a period of time that does not query copy_history prior to when the mitigation was implemented at 06:30 PT, then your queries are no longer impacted.
Workaround: As a workaround, you can alter the timespan of your queries to a shorter time frame (e.g., one or two hours).
Incident start time: 03:37 PT May 25, 2022
Customer experience: Customers hosted in the specified regions may experience delays or failures while performing Snowpipe data ingestion workflows that require the copy_history function. If the timespan set for the copy_history query is between May 9, 2022, and May 25, 2022, at 06:30 PT, then your queries may continue to be impacted. If the timespan is set for a period of time that does not query copy_history prior to when the mitigation was implemented at 06:30 PT, then your queries are no longer impacted.
Workaround: As a workaround, you can alter the timespan of your queries to a shorter time frame (e.g., one or two hours).
Incident start time: 03:37 PT May 25, 2022
Investigating
Current status: We're investigating an issue with Snowpipe (Data Ingestion). We'll provide an update within 60 minutes.
Customer experience: Customers hosted in the specified regions may experience delays while attempting to execute queries. Affected customers may encounter query failures.
Incident start time: 03:37 PT May 25, 2022
Customer experience: Customers hosted in the specified regions may experience delays while attempting to execute queries. Affected customers may encounter query failures.
Incident start time: 03:37 PT May 25, 2022
This incident affected: Azure - West Europe (Netherlands) (Snowpipe (Data Ingestion)), AWS - Asia Pacific (Singapore) (Snowpipe (Data Ingestion)), Azure - Australia East (New South Wales) (Snowpipe (Data Ingestion)), AWS - Canada (Central) (Snowpipe (Data Ingestion)), AWS - Asia Pacific (Tokyo) (Snowpipe (Data Ingestion)), Azure - West US 2 (Washington) (Snowpipe (Data Ingestion)), GCP - Europe West 2 (London) (Snowpipe (Data Ingestion)), AWS - Asia Pacific (Seoul) (Snowpipe (Data Ingestion)), AWS - US East (N. Virginia) (Snowpipe (Data Ingestion)), GCP - US Central 1 (Iowa) (Snowpipe (Data Ingestion)), GCP - Europe West 4 (Eemshaven) (Snowpipe (Data Ingestion)), AWS - US West (Oregon) (Snowpipe (Data Ingestion)), and AWS - Asia Pacific (Sydney) (Snowpipe (Data Ingestion)).