Skip to main content

Performance: SAN issue

Last Updated:
2011-07-13 04:00:00
Event:
2011-07-12 04:00:00
Status:
Closed
Brief Description:
Storage
User Impact:
N/A
Workaround:
There is no workaround for this issue
Current Status:
N/A
Services Affected:
Full Description:
Most services affected by the SAN issue are now performing well. A workaround is in place.
CIT TDX ID:



Timeline of Changes

Description Current Status Date Time
All patches have been applied and seem to be working.\n\nCIT will continue to monitor the situation. All patches have been applied and seem to be working.\n\nCIT will continue to monitor the situation. 2011-07-13 04:00:00
All patches have been applied, and the SAN issue is now considered resolved. All patches have been applied, and the SAN issue is now considered resolved. 2011-07-13 04:00:00
CIT is working with IBM Support. The intent is to initially restore performance to the degree possible, and then to resolve the root issue. A workaround is currently being attempted and we are analyzing the results to determine the magnitude of its impact. CIT is working with IBM Support. The intent is to initially restore performance to the degree possible, and then to resolve the root issue. A workaround is currently being attempted and we are analyzing the results to determine the magnitude of its impact. 2011-07-12 04:00:00
Most services affected by this issue are now performing well. Most ColdFusion sites are now available; the rest should be by the end of the business day.\n\nOther affected services included some static web sites, CommonSpot 5, some KFS jobs, and MyCornell. There are still some MyCornell portlet issues; CIT is working to resolve them. Most services affected by this issue are now performing well. Most ColdFusion sites are now available; the rest should be by the end of the business day.\n\nOther affected services included some static web sites, CommonSpot 5, some KFS jobs, and MyCornell. There are still some MyCornell portlet issues; CIT is working to resolve them. 2011-07-12 04:00:00
A workaround for the SAN issue has restored storage performance to normal ranges. We are continuing to diagnose the problem with the vendor and will schedule the installation of a fix as soon as is practical.\n\nThe workaround halted one of the two redundant I/O nodes in our "SAN Volume Controller" (SVC) that was causing the performance problem. As a result, the SAN is in a degraded mode with no redundancy should the remaining I/O node fail. A workaround for the SAN issue has restored storage performance to normal ranges. We are continuing to diagnose the problem with the vendor and will schedule the installation of a fix as soon as is practical.\n\nThe workaround halted one of the two redundant I/O nodes in our "SAN Volume Controller" (SVC) that was causing the performance problem. As a result, the SAN is in a degraded mode with no redundancy should the remaining I/O node fail. 2011-07-12 04:00:00
Correction on this issue's start date and time: The SAN issue began at 10:20 am on Monday, July 11. Correction on this issue's start date and time: The SAN issue began at 10:20 am on Monday, July 11. 2011-07-12 04:00:00
Vendor supplied patches will be applied on 7/13/2011 from 5 AM to 7 AM to resolve this issue.\nAt this time we expect this be a transparent change. Vendor supplied patches will be applied on 7/13/2011 from 5 AM to 7 AM to resolve this issue.\nAt this time we expect this be a transparent change. 2011-07-12 04:00:00