Skip to main content

Unplanned Outage: Many VM servers being rebooted

Last Updated:
2014-08-10 00:00:00
Event:
2014-08-10 00:00:00
Status:
Closed
Brief Description:
Multiple VM servers
User Impact:
N/A
Workaround:
There is no workaround for this issue
Current Status:
N/A
Services Affected:
Full Description:
The SAN maintenance this morning has caused problems on one or more of our ESX servers hosting multiple VMs. The following VMs are being shut down and will be rebooted shortly.\n\n \n\nName Guest\n\n---- -----\n\nacadtech06 acadtech06:Red Hat Enterprise Linux 5 (32-bit)\n\nashford10 ashford10:Red Hat Enterprise Linux 6 (64-bit)\n\nbmw052 bmw052:Red Hat Enterprise Linux 6 (64-bit)\n\nbmw087 bmw087:Red Hat Enterprise Linux 6 (64-bit)\n\nbmw112 bmw112:Red Hat Enterprise Linux 6 (64-bit)\n\nlib-discovr-02 lib-discovr-02:Red Hat Enterprise Linux 6 (64-bit)\n\nlib-discovr-03 lib-discovr-03:Red Hat Enterprise Linux 6 (64-bit)\n\nlib-discovr-07 lib-discovr-07:Red Hat Enterprise Linux 6 (64-bit)\n\nlib-euclid-001 lib-euclid-001:Red Hat Enterprise Linux 6 (64-bit)\n\nnetman08 netman08:Red Hat Enterprise Linux 6 (64-bit)\n\noxton131 oxton131:Red Hat Enterprise Linux 6 (64-bit)\n\nsf-agnorth-003 sf-agnorth-003:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-balsamic01 sf-balsamic01:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-bmrtest2012-bmr sf-bmrtest2012-bmr:\n\nsf-capps-np002 sf-capps-np002:Red Hat Enterprise Linux 6 (64-bit)\n\nsf-citdsrv-018 sf-citdsrv-018:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-ecm-np001 sf-ecm-np001:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-ecm-np003 sf-ecm-np003:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-ecm-np004 sf-ecm-np004:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-fs-lb-003 sf-fs-lb-003:Microsoft Windows Server 2012 (64-bit)\n\nsf-fs-srv-011 sf-fs-srv-011:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-fs-srv-020 sf-fs-srv-020:Microsoft Windows Server 2012 (64-bit)\n\nsf-fs-tsrv-010 sf-fs-tsrv-010:Microsoft Windows Server 2012 (64-bit)\n\nsf-infranode036 sf-infranode036:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-infranode046 sf-infranode046:\n\nsf-infranode421 sf-infranode421:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-infranode426 sf-infranode426:Microsoft Windows Server 2012 (64-bit)\n\nsf-infranode516 sf-infranode516:Red Hat Enterprise Linux 6 (64-bit)\n\nsf-infranode525 sf-infranode525:Red Hat Enterprise Linux 6 (64-bit)\n\nsf-infranode527 sf-infranode527:Red Hat Enterprise Linux 6 (64-bit)\n\nsf-kradm-np006 sf-kradm-np006:Microsoft Windows Server 2008 R2 (64-bit)\n\nsf-kualibuild02 sf-kualibuild02:Microsoft Windows Server 2012 (64-bit)\n\nsf-lib-app-008 sf-lib-...



Timeline of Changes

Description Current Status Date Time
VMs are being shutdown for rebooting at this time. VMs are being shutdown for rebooting at this time. 2014-08-10 00:00:00
CIT Technical Staff estimate the issue will be resolved by 1:30 PM. The VM servers listed as 24x7 have been restarted at this time. The remaining VM servers in this list should have reboots completed by 1:30 PM.\n\n\nName Guest\n---- -----\nacadtech06 acadtech06:Red Hat Enterprise Linux 5 (32-bit)\nashford10 ashford10:Red Hat Enterprise Linux 6 (64-bit)\nbmw052 bmw052:Red Hat Enterprise Linux 6 (64-bit)\nbmw087 bmw087:Red Hat Enterprise Linux 6 (64-bit)\nbmw112 bmw112:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-02 lib-discovr-02:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-03 lib-discovr-03:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-07 lib-discovr-07:Red Hat Enterprise Linux 6 (64-bit)\nlib-euclid-001 lib-euclid-001:Red Hat Enterprise Linux 6 (64-bit)\nnetman08 netman08:Red Hat Enterprise Linux 6 (64-bit)\noxton131 oxton131:Red Hat Enterprise Linux 6 (64-bit)\nsf-agnorth-003 sf-agnorth-003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-balsamic01 sf-balsamic01:Microsoft Windows Server 2008 R2 (64-bit)\nsf-bmrtest2012-bmr sf-bmrtest2012-bmr:\nsf-capps-np002 sf-capps-np002:Red Hat Enterprise Linux 6 (64-bit)\nsf-citdsrv-018 sf-citdsrv-018:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np001 sf-ecm-np001:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np003 sf-ecm-np003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np004 sf-ecm-np004:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-lb-003 sf-fs-lb-003:Microsoft Windows Server 2012 (64-bit)\nsf-fs-srv-011 sf-fs-srv-011:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-srv-020 sf-fs-srv-020:Microsoft Windows Server 2012 (64-bit)\nsf-fs-tsrv-010 sf-fs-tsrv-010:Microsoft Windows Server 2012 (64-bit)\nsf-infranode036 sf-infranode036:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode046 sf-infranode046:\nsf-infranode421 sf-infranode421:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode426 sf-infranode426:Microsoft Windows Server 2012 (64-bit)\nsf-infranode516 sf-infranode516:Red Hat Enterprise Linux 6 (64-bit)\nsf-infranode525 sf-infranode525:Red Hat Enterprise Linux 6 (64-bit)\nsf-infranode527 sf-infranode527:Red Hat Enterprise Linux 6 (64-bit)\nsf-kradm-np006 sf-kradm-np006:Microsoft Windows Server 2008 R2 (64-bit)\nsf-kualibuild02 sf-kualibuild02:Microsoft Windows Server 2012 (64-bit)\nsf-lib-app-008 sf-lib-app-008:Red Hat Enterprise Linux 6 (6... CIT Technical Staff estimate the issue will be resolved by 1:30 PM. The VM servers listed as 24x7 have been restarted at this time. The remaining VM servers in this list should have reboots completed by 1:30 PM.\n\n\nName Guest\n---- -----\nacadtech06 acadtech06:Red Hat Enterprise Linux 5 (32-bit)\nashford10 ashford10:Red Hat Enterprise Linux 6 (64-bit)\nbmw052 bmw052:Red Hat Enterprise Linux 6 (64-bit)\nbmw087 bmw087:Red Hat Enterprise Linux 6 (64-bit)\nbmw112 bmw112:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-02 lib-discovr-02:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-03 lib-discovr-03:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-07 lib-discovr-07:Red Hat Enterprise Linux 6 (64-bit)\nlib-euclid-001 lib-euclid-001:Red Hat Enterprise Linux 6 (64-bit)\nnetman08 netman08:Red Hat Enterprise Linux 6 (64-bit)\noxton131 oxton131:Red Hat Enterprise Linux 6 (64-bit)\nsf-agnorth-003 sf-agnorth-003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-balsamic01 sf-balsamic01:Microsoft Windows Server 2008 R2 (64-bit)\nsf-bmrtest2012-bmr sf-bmrtest2012-bmr:\nsf-capps-np002 sf-capps-np002:Red Hat Enterprise Linux 6 (64-bit)\nsf-citdsrv-018 sf-citdsrv-018:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np001 sf-ecm-np001:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np003 sf-ecm-np003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np004 sf-ecm-np004:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-lb-003 sf-fs-lb-003:Microsoft Windows Server 2012 (64-bit)\nsf-fs-srv-011 sf-fs-srv-011:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-srv-020 sf-fs-srv-020:Microsoft Windows Server 2012 (64-bit)\nsf-fs-tsrv-010 sf-fs-tsrv-010:Microsoft Windows Server 2012 (64-bit)\nsf-infranode036 sf-infranode036:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode046 sf-infranode046:\nsf-infranode421 sf-infranode421:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode426 sf-infranode426:Microsoft Windows Server 2012 (64-bit)\nsf-infranode516 sf-infranode516:Red Hat Enterprise Linux 6 (64-bit)\nsf-infranode525 sf-infranode525:Red Hat Enterprise Linux 6 (64-bit)\nsf-infranode527 sf-infranode527:Red Hat Enterprise Linux 6 (64-bit)\nsf-kradm-np006 sf-kradm-np006:Microsoft Windows Server 2008 R2 (64-bit)\nsf-kualibuild02 sf-kualibuild02:Microsoft Windows Server 2012 (64-bit)\nsf-lib-app-008 sf-lib-app-008:Red Hat Enterprise Linux 6 (6... 2014-08-10 00:00:00
ESX node “Norway01” was powercycled, necessitating the shutdown of 50 VMs at 11:20am. The following servers were down from approximately 11:20 – 12:15. They were listed as “24x7” so were brought back up immediately.\n \nlib-discovr-02 lib-discovr-02:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-03 lib-discovr-03:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-07 lib-discovr-07:Red Hat Enterprise Linux 6 (64-bit)\nlib-euclid-001 lib-euclid-001:Red Hat Enterprise Linux 6 (64-bit)\nsf-citdsrv-018 sf-citdsrv-018:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-srv-011 sf-fs-srv-011:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode036 sf-infranode036:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode046 sf-infranode046:\nsf-lib-app-008 sf-lib-app-008:Red Hat Enterprise Linux 6 (64-bit)\nsf-paprika08 sf-paprika08:Microsoft Windows Server 2012 (64-bit)\nsf-sas-app-007 sf-sas-app-007:Microsoft Windows Server 2008 R2 (64-bit)\nsf-sas-app-008 sf-sas-app-008:Microsoft Windows Server 2008 R2 (64-bit)\nsf-sas-db-009 sf-sas-db-009:Microsoft Windows Server 2008 R2 (64-bit)\n \nWe are working with the vendor to resolve the remaining problems. To minimize risk, the following servers, listed as 5x9, remain down at this time.\n \nacadtech06 acadtech06:Red Hat Enterprise Linux 5 (32-bit)\nashford10 ashford10:Red Hat Enterprise Linux 6 (64-bit)\nbmw052 bmw052:Red Hat Enterprise Linux 6 (64-bit)\nbmw087 bmw087:Red Hat Enterprise Linux 6 (64-bit)\nbmw112 bmw112:Red Hat Enterprise Linux 6 (64-bit)\nnetman08 netman08:Red Hat Enterprise Linux 6 (64-bit)\noxton131 oxton131:Red Hat Enterprise Linux 6 (64-bit)\nsf-agnorth-003 sf-agnorth-003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-balsamic01 sf-balsamic01:Microsoft Windows Server 2008 R2 (64-bit)\nsf-bmrtest2012-bmr sf-bmrtest2012-bmr:\nsf-capps-np002 sf-capps-np002:Red Hat Enterprise Linux 6 (64-bit)\nsf-ecm-np001 sf-ecm-np001:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np003 sf-ecm-np003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np004 sf-ecm-np004:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-lb-003 sf-fs-lb-003:Microsoft Windows Server 2012 (64-bit)\nsf-fs-srv-020 sf-fs-srv-020:Microsoft Windows Server 2012 (64-bit)\nsf-fs-tsrv-010 sf-fs-tsrv-010:Microsoft Windows Server 2012 (64-bit)\nsf-infranode421 sf-infranode421:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode426 sf-infranode426:Microsoft Windows Server 2012 (64-bit)\nsf-infranode516 sf-infrano... ESX node “Norway01” was powercycled, necessitating the shutdown of 50 VMs at 11:20am. The following servers were down from approximately 11:20 – 12:15. They were listed as “24x7” so were brought back up immediately.\n \nlib-discovr-02 lib-discovr-02:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-03 lib-discovr-03:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-07 lib-discovr-07:Red Hat Enterprise Linux 6 (64-bit)\nlib-euclid-001 lib-euclid-001:Red Hat Enterprise Linux 6 (64-bit)\nsf-citdsrv-018 sf-citdsrv-018:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-srv-011 sf-fs-srv-011:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode036 sf-infranode036:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode046 sf-infranode046:\nsf-lib-app-008 sf-lib-app-008:Red Hat Enterprise Linux 6 (64-bit)\nsf-paprika08 sf-paprika08:Microsoft Windows Server 2012 (64-bit)\nsf-sas-app-007 sf-sas-app-007:Microsoft Windows Server 2008 R2 (64-bit)\nsf-sas-app-008 sf-sas-app-008:Microsoft Windows Server 2008 R2 (64-bit)\nsf-sas-db-009 sf-sas-db-009:Microsoft Windows Server 2008 R2 (64-bit)\n \nWe are working with the vendor to resolve the remaining problems. To minimize risk, the following servers, listed as 5x9, remain down at this time.\n \nacadtech06 acadtech06:Red Hat Enterprise Linux 5 (32-bit)\nashford10 ashford10:Red Hat Enterprise Linux 6 (64-bit)\nbmw052 bmw052:Red Hat Enterprise Linux 6 (64-bit)\nbmw087 bmw087:Red Hat Enterprise Linux 6 (64-bit)\nbmw112 bmw112:Red Hat Enterprise Linux 6 (64-bit)\nnetman08 netman08:Red Hat Enterprise Linux 6 (64-bit)\noxton131 oxton131:Red Hat Enterprise Linux 6 (64-bit)\nsf-agnorth-003 sf-agnorth-003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-balsamic01 sf-balsamic01:Microsoft Windows Server 2008 R2 (64-bit)\nsf-bmrtest2012-bmr sf-bmrtest2012-bmr:\nsf-capps-np002 sf-capps-np002:Red Hat Enterprise Linux 6 (64-bit)\nsf-ecm-np001 sf-ecm-np001:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np003 sf-ecm-np003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np004 sf-ecm-np004:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-lb-003 sf-fs-lb-003:Microsoft Windows Server 2012 (64-bit)\nsf-fs-srv-020 sf-fs-srv-020:Microsoft Windows Server 2012 (64-bit)\nsf-fs-tsrv-010 sf-fs-tsrv-010:Microsoft Windows Server 2012 (64-bit)\nsf-infranode421 sf-infranode421:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode426 sf-infranode426:Microsoft Windows Server 2012 (64-bit)\nsf-infranode516 sf-infrano... 2014-08-10 00:00:00
At 14:40, the scheduled SAN maintenance was completed with the help of the vendor. We are now bringing up the remaining systems which were kept down during the maintenance. Estimated time to completion is 15:00.\n\n-----\nESX node “Norway01” was powercycled, necessitating the shutdown of 50 VMs at 11:20am. The following servers were down from approximately 11:20 – 12:15. They were listed as “24x7” so were brought back up immediately.\n \nlib-discovr-02 lib-discovr-02:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-03 lib-discovr-03:Red Hat Enterprise Linux 6 (64-bit)\nlib-euclid-001 lib-euclid-001:Red Hat Enterprise Linux 6 (64-bit)\nsf-citdsrv-018 sf-citdsrv-018:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode046 sf-infranode046:\nsf-lib-app-008 sf-lib-app-008:Red Hat Enterprise Linux 6 (64-bit)\nsf-sas-db-009 sf-sas-db-009:Microsoft Windows Server 2008 R2 (64-bit)\n\nThe following servers were rebooted at 12:20 but had network problems which left them off the network until approximately 14:35:\n\nlib-discovr-07 lib-discovr-07:Red Hat Enterprise Linux 6 (64-bit)\nsf-fs-srv-011 sf-fs-srv-011:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode036 sf-infranode036:Microsoft Windows Server 2008 R2 (64-bit\nsf-paprika08 sf-paprika08:Microsoft Windows Server 2012 (64-bit)\nsf-sas-app-007 sf-sas-app-007:Microsoft Windows Server 2008 R2 (64-bit)\nsf-sas-app-008 sf-sas-app-008:Microsoft Windows Server 2008 R2 (64-bit)\n\nTo minimize risk, the following servers, listed as 5x9, remained down as we completed the maintenance. They are being powered on now.\n\nacadtech06 acadtech06:Red Hat Enterprise Linux 5 (32-bit)\nashford10 ashford10:Red Hat Enterprise Linux 6 (64-bit)\nbmw052 bmw052:Red Hat Enterprise Linux 6 (64-bit)\nbmw087 bmw087:Red Hat Enterprise Linux 6 (64-bit)\nbmw112 bmw112:Red Hat Enterprise Linux 6 (64-bit)\nnetman08 netman08:Red Hat Enterprise Linux 6 (64-bit)\noxton131 oxton131:Red Hat Enterprise Linux 6 (64-bit)\nsf-agnorth-003 sf-agnorth-003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-balsamic01 sf-balsamic01:Microsoft Windows Server 2008 R2 (64-bit)\nsf-bmrtest2012-bmr sf-bmrtest2012-bmr:\nsf-capps-np002 sf-capps-np002:Red Hat Enterprise Linux 6 (64-bit)\nsf-ecm-np001 sf-ecm-np001:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np003 sf-ecm-np003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np004 sf-ecm-np004:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-lb-003 sf-fs-lb-003:Microsoft Windows Server 2012 (64-bit)\nsf-fs-srv-020 sf-fs-srv-020:Microsoft Windows Server 2012 (64-bit)\nsf-fs-tsrv-010 sf-fs-tsrv-010:Micr... At 14:40, the scheduled SAN maintenance was completed with the help of the vendor. We are now bringing up the remaining systems which were kept down during the maintenance. Estimated time to completion is 15:00.\n\n-----\nESX node “Norway01” was powercycled, necessitating the shutdown of 50 VMs at 11:20am. The following servers were down from approximately 11:20 – 12:15. They were listed as “24x7” so were brought back up immediately.\n \nlib-discovr-02 lib-discovr-02:Red Hat Enterprise Linux 6 (64-bit)\nlib-discovr-03 lib-discovr-03:Red Hat Enterprise Linux 6 (64-bit)\nlib-euclid-001 lib-euclid-001:Red Hat Enterprise Linux 6 (64-bit)\nsf-citdsrv-018 sf-citdsrv-018:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode046 sf-infranode046:\nsf-lib-app-008 sf-lib-app-008:Red Hat Enterprise Linux 6 (64-bit)\nsf-sas-db-009 sf-sas-db-009:Microsoft Windows Server 2008 R2 (64-bit)\n\nThe following servers were rebooted at 12:20 but had network problems which left them off the network until approximately 14:35:\n\nlib-discovr-07 lib-discovr-07:Red Hat Enterprise Linux 6 (64-bit)\nsf-fs-srv-011 sf-fs-srv-011:Microsoft Windows Server 2008 R2 (64-bit)\nsf-infranode036 sf-infranode036:Microsoft Windows Server 2008 R2 (64-bit\nsf-paprika08 sf-paprika08:Microsoft Windows Server 2012 (64-bit)\nsf-sas-app-007 sf-sas-app-007:Microsoft Windows Server 2008 R2 (64-bit)\nsf-sas-app-008 sf-sas-app-008:Microsoft Windows Server 2008 R2 (64-bit)\n\nTo minimize risk, the following servers, listed as 5x9, remained down as we completed the maintenance. They are being powered on now.\n\nacadtech06 acadtech06:Red Hat Enterprise Linux 5 (32-bit)\nashford10 ashford10:Red Hat Enterprise Linux 6 (64-bit)\nbmw052 bmw052:Red Hat Enterprise Linux 6 (64-bit)\nbmw087 bmw087:Red Hat Enterprise Linux 6 (64-bit)\nbmw112 bmw112:Red Hat Enterprise Linux 6 (64-bit)\nnetman08 netman08:Red Hat Enterprise Linux 6 (64-bit)\noxton131 oxton131:Red Hat Enterprise Linux 6 (64-bit)\nsf-agnorth-003 sf-agnorth-003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-balsamic01 sf-balsamic01:Microsoft Windows Server 2008 R2 (64-bit)\nsf-bmrtest2012-bmr sf-bmrtest2012-bmr:\nsf-capps-np002 sf-capps-np002:Red Hat Enterprise Linux 6 (64-bit)\nsf-ecm-np001 sf-ecm-np001:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np003 sf-ecm-np003:Microsoft Windows Server 2008 R2 (64-bit)\nsf-ecm-np004 sf-ecm-np004:Microsoft Windows Server 2008 R2 (64-bit)\nsf-fs-lb-003 sf-fs-lb-003:Microsoft Windows Server 2012 (64-bit)\nsf-fs-srv-020 sf-fs-srv-020:Microsoft Windows Server 2012 (64-bit)\nsf-fs-tsrv-010 sf-fs-tsrv-010:Micr... 2014-08-10 00:00:00
As of 15:40 SAN maintenance is complete and all servers have been brought back up. As of 15:40 SAN maintenance is complete and all servers have been brought back up. 2014-08-10 00:00:00