South Orange Campus Network
Incident Report for Seton Hall University
Postmortem

The preliminary cause seems to be a bad update loaded into a staging area on the core (standard procedure to prepare for a future upgrade) this corrupted the core switches and put them in “ROMMON” mode, which required several hours of work with vendor support to resolve. As of now the main code is loaded back on the switches, but networking has to retest the failover configurations and fix ports marked as disabled due to the initial outage.

Posted May 19, 2020 - 08:58 EDT

Resolved
The campus network outage has been resolved.
Posted May 19, 2020 - 08:57 EDT
Update
We are continuing to monitor for any further issues.
Posted May 19, 2020 - 08:52 EDT
Monitoring
Partial outages remain in McQuaid and Alfieri halls. All off campus and other on campus buildings are back online. The two remaining buildings should be operational in the next 1-2 hours.
Posted May 18, 2020 - 22:37 EDT
Update
Partial service has been restored. We are continuing to work with our vendor to restore service.
Posted May 18, 2020 - 22:02 EDT
Identified
The South Orange campus network is exhibiting performance issues. The problems are sporadic and may affect the entire campus network, including Wireless, Wired, Network Printing, and VoIP phones in Human Resources, the Office of Admissions and Health Services.

Users on campus may see intermittent outages and phone forwarding may be temporarily unavailable for employees in Human Resources and Office of Admissions.
Posted May 18, 2020 - 16:01 EDT
This incident affected: Network and Internet.