Core Site Teb5 Power Incident

Incident Report for InterServer

Resolved

A new incident is being created for openvz31
Posted Jul 15, 2025 - 14:01 EDT

Update

An update from Datacenter Techs at Coresite:

Update: Switchgear A1/A2 and B1/B2 has restored back to utility. Chiller 4 is back online

Incident Ticket: 03044792
Posted Jul 15, 2025 - 14:01 EDT

Update

Openvz31 system is being replaced due to disk controller errors after this incident.
Posted Jul 15, 2025 - 06:13 EDT

Update

Update From CoreSite: Load on switchgear A1/A2 and B1/B2 remains on the emergency source generators and we have greater than 24 hours of run time available at current load, we have spoken with our fuel vendor and still awaiting an ETA for a refueling. Our main focus remains UPS A2 which is currently on static bypass with 2 of 4 modules offline with a fault. We are working to bring those modules online so we can work on restoring UPS A2 back to normal. Chillers are running and supplying chilled water at an expected level with temps remaining stable.
Posted Jul 14, 2025 - 21:19 EDT

Update

da900 has been resolved.
Posted Jul 14, 2025 - 21:19 EDT

Update

KVM71 and vda1800 have been resolved.
Posted Jul 14, 2025 - 20:39 EDT

Update

Currently three servers are down
* da900
* vda1800
* kvm71

Local technicians are en-route to check these systems.
Posted Jul 14, 2025 - 20:29 EDT

Update

Coresite technicians have updated "Update: Data Center Operations is continuing to perform rounds and clear alarms. We are focused on chiller 4 which is currently offline with 3 other chillers running, as well as UPS A2 which has 2 modules on inverter and 2 modules offline, we are placing a dispatch to our UPS vendor support troubleshooting this issue. Load on switchgear A1/A2 and B1/B2 remains on the emergency source generators and we are awaiting an ETA from our fuel vendor to refill our fuel tanks."
Posted Jul 14, 2025 - 20:20 EDT

Update

We are continuing to investigate this issue.
Posted Jul 14, 2025 - 19:58 EDT

Investigating

A power incident is being investigated at CoreSite Teb5. Data Center Operations has notified us the load on switchgear A1/A2 and B1/B2 have been transferred to the emergency source generators.

Additionally coresite operations has stated "We have technicians actively resetting CRAC units throughout the facility, temps may have risen during the transfer while the chiller plant cycled. Load on switchgear A1/A2 and B1/B2 remain on the emergency source generators and we are placing a dispatched to our fuel vendor to ensure we have our fuel tanks filled"
Posted Jul 14, 2025 - 19:51 EDT
This incident affected: Datacenters (TEB5).