Postmortem -
Read details
Mar 27, 08:18 PDT
Resolved -
This incident has been resolved. We have achieved significant improvements to the reporting delays, and delays to Unique/Influenced Opens are within normal parameters. Inactivity Automations have been re-enabled at this time. All other notifications, messages, and automation continue to send normally.
If you have any questions or concerns please contact Airship Technical Support at
https://support.airship.com
Mar 25, 09:51 PDT
Update -
We are continuing to monitor the progress of the fix currently in place, and are seeing significant improvements. There are still some reporting delays, particularly with opens (including influenced opens) and device-side custom events. Automations based on region events and device-side custom events remain delayed. Inactivity automations are still paused at this time to avoid sending in error. All other notifications, messages, and automation continue to send normally.
If you have any questions or concerns please contact Airship Technical Support at
https://support.airship.com
Mar 25, 08:46 PDT
Monitoring -
A fix has been implemented to resolve this issue, and we are currently monitoring the results. At this time, we are still seeing significant reporting delays, particularly with opens (including influenced opens), and device-side custom events. These are also delayed in Real-Time Data Streaming and Performance Analytics. Automations based on region events and device-side custom events are also still impacted. Inactivity Automations are still paused in order to prevent sends erroneously. All other notifications, messages, and automation continue to send normally.
If you have any questions or concerns please contact Airship Technical Support at
https://support.airship.com
Mar 25, 07:05 PDT
Update -
We are currently experiencing an issue which is resulting in multiple hours of reporting delays, particularly with opens (including influenced opens), and device-side custom events. These are also delayed in Real-Time Data Streaming and Performance Analytics. Automations based on region events and device-side custom events are also delayed. Inactivity automations have also been paused for the duration of the incident so they are not sent in error. All other notifications, messages, and automations should be sending normally
Mar 24, 22:06 PDT
Update -
We are continuing to work on a fix for this issue.
Mar 24, 22:05 PDT
Update -
Our engineers are continuing to work with a third party provider to resolve this issue. Send counts should now be up to date, but there are still significant delays with opens (including influenced opens) and device-side custom events. Automations based on region events and device-side custom events are also delayed. Inactivity automations have been paused for the duration of the incident so they are not sent in error. All other notifications, messages, and automations should be sending normally.
Mar 24, 21:31 PDT
Update -
Our engineers are continuing to work to resolve this issue. Current status is we are seeing multiple hours of reporting delays, particularly with sends, opens (including influenced opens), and device-side custom events. Automations based on region events and device-side custom events are also delayed. Inactivity automations have also been paused for the duration of the incident so they are not sent in error. All other notifications, messages, and automations should be sending normally.
Mar 24, 17:55 PDT
Update -
We are continuing to work on a fix for this issue.
Mar 24, 16:05 PDT
Update -
We are continuing to work on a fix for this issue. Delays to reporting and Automation triggers are expected at this time.
Mar 24, 15:02 PDT
Update -
We are continuing to work on a fix for this issue.
Mar 24, 14:01 PDT
Update -
We are continuing to work on a fix for this issue.
Mar 24, 12:52 PDT
Update -
We are continuing to work on a fix for this issue. Delays to reporting and Automation triggers are expected at this time.
Mar 24, 11:38 PDT
Update -
We are continuing to work on a fix for this issue.
Mar 24, 10:14 PDT
Identified -
We’ve identified the root cause of this issue and our engineers are actively working to resolve the issue. At this time Reporting data, Device Properties and Device Events are still delayed.
If you have any questions or concerns please contact Airship Support at support.airship.com
Mar 24, 09:20 PDT
Investigating -
We are currently investigating an issue that is resulting in delays in event reporting for devices. Events that are used for Automations are currently delayed by an hour or more. Some device property data updates (such as location history and certain device property tags) are also delayed/impacted. All other services, including notification delivery, are working as expected at this time.
If you have any questions or concerns please contact Airship Support at support.airship.com
Mar 24, 08:50 PDT