[2026-11-12][Resolved]Notice of Job Failures Due to AWS Tokyo Region Outage
Due to an outage in the AWS Tokyo Region (ap-northeast-1), some opswitch jobs failed or were delayed.
The AWS outage has been resolved, and opswitch jobs are now running normally.
Date and time of occurrence
Section titled “Date and time of occurrence”Wednesday, November 11, 2026, at approximately 11:46 p.m. (JST) to Thursday, November 12, 2026, at approximately 5:31 a.m. (JST).
Failure Description
Section titled “Failure Description”Increased AWS API error rates and EC2 instance launch failures occurred, primarily in a single Availability Zone (apne1-az1) of the AWS Tokyo Region.
As a result, many opswitch jobs failed with errors such as the following:
An error occurred (InternalError) when calling the XXXX operation (reached max retries: 4): An internal error has occurredXXXX is replaced with an operation name such as CreateImage, DeregisterImage, CreateSnapshot, DescribeVolumes, StartInstances, StopInstances, or StopDBCluster.
For details of the outage, please refer to the AWS Health Dashboard.
Request to Customers
Section titled “Request to Customers”For jobs that ran during the period above, please check the execution logs from the menu “LOGS” - “Run logs” in the header.
Even if a job’s status is “Success”, its processing may have been delayed due to slow AWS API responses. In addition, according to AWS, instances newly launched in the affected Availability Zone during the outage experienced connectivity issues until the outage was resolved.
Therefore, regardless of whether a job succeeded or failed, please check in the AWS Management Console that the target resources are in the expected state.
- EC2 instances and DB instances have started or stopped as expected
- You can connect to EC2 instances that were started
- AMIs and snapshots have been created and are available
- Deletion of old AMIs and snapshots by generation management has completed
If a job failed or a resource is not in the expected state, please re-run the job or operate it manually from the Management Console.
We apologize for any inconvenience this may have caused. Thank you for your continued use of opswitch.
