Event Summary On 10/22/2025 @ 10:37am under change CHG0098473 (ATS maintenance) both main breakers experienced a loss of power for 111 minutes.
Root Cause Analysis Main breakers malfunctioned during the listed change window. This situation combined with a battery fault led to an inability to restore utility or generator power quickly. Third party consulted for emergency manual intervention to restore power.
Mitigation Third party was engaged to review the MOP and found it to be error free. The identified battery fault was located and replaced. Main breaker [1B1] identified for replacement. Target date for removal is 11/01/25.
Posted Oct 30, 2025 - 14:28 EDT
Monitoring
3 IPMI Switches are left to replace.
Once we receive the official RFO from CenterSquare, we will share it here.
Posted Oct 23, 2025 - 12:33 EDT
Update
Many systems have now been brought back online which did not come back online once a and b power was restored at this facility. We continue to check and work through monitoring reports as well as tickets on cases as fast as possible.
Posted Oct 22, 2025 - 19:41 EDT
Update
All VPs nodes are up. About 20 servers are still down.
Posted Oct 22, 2025 - 18:31 EDT
Identified
We are working through any reports of systems that did not come back online, or are having boot issues.
Posted Oct 22, 2025 - 16:08 EDT
Update
All switches are now operational.
There are about 50 VPS servers that did not come back online.
Posted Oct 22, 2025 - 14:05 EDT
Update
Update from CenterSquare.
"Notification Update - 10-22-2025 12:21:02 CT We have stabilized the power event, and power has been restored. The data center staff will continue to monitor the situation.
Next Update: Upon the next status change. "
Posted Oct 22, 2025 - 13:22 EDT
Update
Working on restoring 5 racks that are having switch issues. Working on powering up servers that did not automatically turn on. Working on replacing failed storage drives.
Posted Oct 22, 2025 - 13:20 EDT
Update
Power is restored. We will be working on verifying all servers shortly.
Posted Oct 22, 2025 - 12:15 EDT
Update
We found a few racks that are still powered on. We are using those to bring the network back online with long extension cables. Both A + B power is down in 90% of our racks.
Posted Oct 22, 2025 - 11:39 EDT
Update
Our techs are on-site. The majority of the building is down. There is clearly a catastrophic power event that occurred in the building. The damage assessment is being conducted by Centersquare with whatever available contractors are on site. No ETA is available yet.
Posted Oct 22, 2025 - 11:37 EDT
Update
The Centersquare EWR5-A (teb6) has experienced a power loss that is being investigated by technicians now.
Posted Oct 22, 2025 - 11:02 EDT
Investigating
We are currently investigating connectivity issues in the Teb6 datacenter.