AWS - Europe (Paris): INC0089352
Incident
Report for Snowflake
Postmortem
Snowflake Engineering has completed the postmortem of this service incident. A detailed Root Cause Analysis (RCA) is available on the Snowflake Community site:
https://community.snowflake.com/s/article/INC0089352
Resolved
Current status: We've implemented the fix for this issue and monitored the environment to confirm that service was restored. If you experience additional issues or have questions, please open a support case via Snowflake Community.
Customer experience: Customers were unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users were unable to log in, queries may have not executed, and users may have observed HTTP 403 errors.
Incident start time: 05:39 UTC August 02, 2023
Incident end time: 14:05 UTC August 03, 2023
Preliminary root cause: An application role granted to Snowsight was misconfigured, preventing users from leveraging regionless URLs to use Snowflake services.
A root cause analysis (RCA) document will be published within seven business days.
Customer experience: Customers were unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users were unable to log in, queries may have not executed, and users may have observed HTTP 403 errors.
Incident start time: 05:39 UTC August 02, 2023
Incident end time: 14:05 UTC August 03, 2023
Preliminary root cause: An application role granted to Snowsight was misconfigured, preventing users from leveraging regionless URLs to use Snowflake services.
A root cause analysis (RCA) document will be published within seven business days.
Monitoring
Current status: We've implemented the fix for this issue, and we'll continue to monitor the environment until we're confident all services are functioning properly.
Customer experience: Customers were unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users were unable to log in, queries may have not executed, and users may have observed HTTP 403 errors.
Incident start time: 05:39 UTC August 02, 2023
Incident end time: 14:05 UTC August 03, 2023
Preliminary root cause: An application role granted to Snowsight was misconfigured, preventing users from leveraging regionless URLs to use Snowflake services.
Customer experience: Customers were unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users were unable to log in, queries may have not executed, and users may have observed HTTP 403 errors.
Incident start time: 05:39 UTC August 02, 2023
Incident end time: 14:05 UTC August 03, 2023
Preliminary root cause: An application role granted to Snowsight was misconfigured, preventing users from leveraging regionless URLs to use Snowflake services.
Update
Current status: We have confirmed the source of the issue and tests of the potential fix have been successful. We continue with our efforts of implementing the fix. We will provide another update in eight hours, or earlier, pending any significant change in status.
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Identified
Current status: We have confirmed the source of the issue and tests of the potential fix have been successful. We are now moving forward with implementing the fix, which is expected to be completed in approximately eight hours. We will provide another update in eight hours.
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Update
Current status: Our efforts to confirm the source of the issue and the efficacy of a fix are ongoing, and we're currently implementing a change in our testing environment to validate further. We'll provide another update within the next two hours.
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Update
Current status: Additional investigation identified a recent change related to authentication that may be the source of the issue, and we're reviewing further to validate reverting it will mitigate impact. We'll provide another update within the next two hours.
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Investigating
Current status: We completed a rollback activity as a potential mitigation; however, this did not resolve the issue. We're continuing our efforts to isolate and address the source of the issue. We'll provide another update within the next two hours.
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL or by going to https://app.snowflake.com and pasting in their account URL to log in.
Incident start time: 05:39 UTC August 02, 2023
Identified
Current status: We've identified the source of the issue, and we're developing and implementing a potential fix to restore service. We'll provide another update within 60 minutes.
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL.
Incident start time: 05:39 UTC August 02, 2023
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL.
Incident start time: 05:39 UTC August 02, 2023
Update
Current status: We're continuing to investigate to determine the source of the issue. We'll provide another update within 60 minutes.
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL.
Incident start time: 05:39 UTC August 02, 2023
Customer experience: Customers may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Workaround: As a workaround, users can still access Snowsight via a regional locator URL.
Incident start time: 05:39 UTC August 02, 2023
Investigating
Current status: We're investigating an issue wherein users hosted in the AWS - EU West 3 (Paris) region may be unable to access and use Snowsight services and features when leveraging a regionless URL. As a result, users may be unable to log in, queries may not execute, and users may observe HTTP 403 errors.
Customer experience: Customers hosted in the specified regions may be unable to access or use Snowflake services and features using Snowsight.
Incident start time: 05:39 UTC August 02, 2023
Customer experience: Customers hosted in the specified regions may be unable to access or use Snowflake services and features using Snowsight.
Incident start time: 05:39 UTC August 02, 2023
This incident affected: AWS - Europe (Paris) (Snowsight).