Resolved -
This incident has been resolved. After extended monitoring, our teams have confirmed that all data processing is up to date and has returned to normalcy.
Nov 10, 23:00 PST
Update -
The team has identified the bottleneck areas and increased resources significantly. We are seeing positive progress following these measures and will continue to monitor to ensure consistent progress.
Nov 10, 07:05 PST
Update -
The backlog processing is ongoing. We are actively monitoring the progress and assessing the allocation of additional resources to speed up the process.
Nov 9, 15:20 PST
Update -
The system is steadily reducing the backlog, and data updates are progressing well. We will continue to monitor the situation closely and share updates as we make progress.
Nov 8, 15:02 PST
Update -
Our technical teams have been working diligently to expedite the recovery process. An unforeseen inventory issue temporarily delayed progress, but corrective measures have been implemented, and the service is now stable. Data ingestion has resumed, and we continue to closely monitor the situation.
Nov 8, 07:19 PST
Update -
The database instance has returned to a healthy state. We will continue to monitor the system and provide an updated ETA.
Nov 8, 02:49 PST
Update -
The sync process is progressing smoothly. The primary sync has been completed successfully, and we’re making steady progress on the remaining nodes. Further updates will be provided once synchronization is complete for all nodes.
Nov 7, 20:57 PST
Identified -
The data processing issue was due to a disk space problem affecting a database node. We have been actively working with our service provider, and corrective actions have been taken. The system is currently in the process of re-syncing. The initial sync is stable, with completion expected soon, after which additional nodes will be synchronized. We continue to monitor the situation and will provide further updates as progress is made.
Nov 7, 17:32 PST
Update -
To expedite the recovery process, we have engaged our service provider and are actively collaborating with them to review and assess impacted services. We will continue to provide updates as we make progress.
Nov 7, 15:17 PST
Investigating -
Incident Description: We are currently experiencing a data processing delay affecting IT Visibility (ITV) in the NA region. While the user interface and Dashboards remain accessible, inventory data updates are delayed. All other ITV services continue to operate normally.
Priority: P2
Restoration Activity: Our technical teams have been engaged and are actively working to resolve the underlying cause. Mitigation measures have been implemented to reduce impact while we investigate further. We will provide updates as more information becomes available.
Nov 7, 14:25 PST