Wallet Pass Updates Delayed
Incident Report for Airship
Postmortem

Between 12:24 AM 5 May and 5:33 AM 8 May as well as between 9:21 PM 8 May and 4:20 PM 9 May US Pacific time, Wallet pass updates may have been delayed over one hour. This issue did not impact new pass creations and downloads, updates via segment, or scheduled updates, or any of our non-Wallet push notification systems. However, customers may have also noticed some brief periods of elevated error response rates from Wallet API endpoints during these times. This issue was caused by increased workload on this system, which our Engineering team has ameliorated with queueing and database improvements. They are also in the process of improving alerting and monitoring for this system to help us respond more quickly to similar issues in the future. We apologize for any inconvenience.

Posted May 11, 2021 - 11:35 PDT

Resolved
This issue has been resolved.

All services within Airship are now working as expected.

If you have any additional questions or concerns, please contact Airship Technical Support at support@airship.com.
Posted May 09, 2021 - 18:07 PDT
Monitoring
A fix for the delays to Wallet pass updates has been implemented and we are currently monitoring the results.

At this time, all other services through both the dashboard and the API are working as expected.

If you have any additional questions or concerns, please contact Airship Technical Support at support@airship.com.
Posted May 09, 2021 - 16:51 PDT
Identified
We have identified the issue that is resulting in delays to pass updates. Our engineers are taking steps to mitigate delays for pass updates.

At this time, all other services through both the dashboard and the API are working as expected.

If you have any additional questions or concerns, please contact Airship Technical Support at support@airship.com.
Posted May 09, 2021 - 12:01 PDT
Update
We are continuing to investigate this issue.
Posted May 09, 2021 - 12:01 PDT
Investigating
We're currently investigating an issue that is causing a delay in processing updates for wallet passes.

All other services through both the dashboard and the API are working as expected.

Our engineering teams are working to resolve this issue. We will update our status page as soon as we have more information.

If you have any additional questions or concerns, please contact Airship Technical Support at support@airship.com.
Posted May 09, 2021 - 08:35 PDT
This incident affected: Mobile Wallet Passes.