Skip to main content

Unplanned Outage: Some web sites unavailable

Last Updated:
2012-02-01 05:00:00
Event:
2012-01-26 05:00:00
Status:
Closed
Brief Description:
User Impact:
N/A
Workaround:
There is no workaround for this issue
Current Status:
N/A
Services Affected:
Full Description:
For all available sites, we are continuing to investigate this incident and are working closely with site owners to resolve some small issues.\n\nFor sites that remain unavailable, CIT is working closely with each site owner to restore the sites. This may take some time; we will provide updates at the beginning/close of business until all sites have been restored.
CIT TDX ID:



Timeline of Changes

Description Current Status Date Time
The remaining sites that were affected by the LAMP issue have been restored and are now available. The remaining sites that were affected by the LAMP issue have been restored and are now available. 2012-02-01 05:00:00
We are continuing to work with site owners to get the matter resolved. We are continuing to work with site owners to get the matter resolved. 2012-01-30 05:00:00
CIT is continuing to work to restore the affected LAMP server. Over 200 websites are affected, including www.cuinfo.cornell.edu, many CALS department sites, and www.ezra.cornell.edu CIT is continuing to work to restore the affected LAMP server. Over 200 websites are affected, including www.cuinfo.cornell.edu, many CALS department sites, and www.ezra.cornell.edu 2012-01-27 05:00:00
The server remains unstable. A number of issues need to be done to resolve the problem. There is no ETA at this time. The server remains unstable. A number of issues need to be done to resolve the problem. There is no ETA at this time. 2012-01-27 05:00:00
CIT is continuing to work to restore access to the affected websites. CIT is continuing to work to restore access to the affected websites. 2012-01-27 05:00:00
CIT is continuing to work to restore access to the affected websites. CIT is continuing to work to restore access to the affected websites. 2012-01-27 05:00:00
CIT is continuing to work to restore access to the affected websites. CIT is continuing to work to restore access to the affected websites. 2012-01-27 05:00:00
Access to most of the affected websites is now possible for individuals on the Ithaca campus networks. Some off-campus users may not have access yet; it can take several hours for the DNS changes to finish propagating across the Internet. Twenty-five sites are still not available. CIT is continuing to work to restore full access. Access to most of the affected websites is now possible for individuals on the Ithaca campus networks. Some off-campus users may not have access yet; it can take several hours for the DNS changes to finish propagating across the Internet. Twenty-five sites are still not available. CIT is continuing to work to restore full access. 2012-01-27 05:00:00
Websites that are still down:\n\n4hclothesline.cce.cornell.edu\nagout.cals.cornell.edu\nblog.acsf.cornell.edu\nemotikinz.cit.cornell.edu\nevents.alumni.cornell.edu\nevidencebasedliving.human.cornell.edu\ngofigure.cce.cornell.edu\nkrasnyt.cornell.edu\nlegacyproject.human.cornell.edu\nm.cornell.edu\nmedia.career.cornell.edu\nmobile.cornell.edu\nm-test.alumni.cornell.edu\nncrn.cornell.edu\nphonebook.directory.cornell.edu\nplaybook.univcomm.cornell.edu\npreservenet-stg.cit.cornell.edu\npwg.human.cornell.edu\nrelief.cornell.edu\nresearchupdate.human.cornell.edu\nwiki.ncrn.cornell.edu\nwww.agout.cals.cornell.edu\nwww.firefightergear.human.cornell.edu\nwww.hdtoday.human.cornell.edu\nwww.linkmagazine.human.cornell.edu\n Websites that are still down:\n\n4hclothesline.cce.cornell.edu\nagout.cals.cornell.edu\nblog.acsf.cornell.edu\nemotikinz.cit.cornell.edu\nevents.alumni.cornell.edu\nevidencebasedliving.human.cornell.edu\ngofigure.cce.cornell.edu\nkrasnyt.cornell.edu\nlegacyproject.human.cornell.edu\nm.cornell.edu\nmedia.career.cornell.edu\nmobile.cornell.edu\nm-test.alumni.cornell.edu\nncrn.cornell.edu\nphonebook.directory.cornell.edu\nplaybook.univcomm.cornell.edu\npreservenet-stg.cit.cornell.edu\npwg.human.cornell.edu\nrelief.cornell.edu\nresearchupdate.human.cornell.edu\nwiki.ncrn.cornell.edu\nwww.agout.cals.cornell.edu\nwww.firefightergear.human.cornell.edu\nwww.hdtoday.human.cornell.edu\nwww.linkmagazine.human.cornell.edu\n 2012-01-27 05:00:00
We are continuing to investigate this incident and are working closely with site owners to resolve some small issues for those sites that are now available. For some off-campus users, it may take longer before the available sites become accessible due to DNS changes.\n\nThe following sites continue to be unavailable. CIT is working closely with each site owner to restore the sites. This may take some time; we will provide updates at the beginning/close of business until all sites have been restored.\n\n4hclothesline.cce.cornell.edu\nagout.cals.cornell.edu\nblog.acsf.cornell.edu\nemotikinz.cit.cornell.edu\nevents.alumni.cornell.edu\nevidencebasedliving.human.cornell.edu\ngofigure.cce.cornell.edu\nkrasnyt.cornell.edu\nlegacyproject.human.cornell.edu\nm.cornell.edu\nmedia.career.cornell.edu\nmobile.cornell.edu\nm-test.alumni.cornell.edu\nncrn.cornell.edu\nphonebook.directory.cornell.edu\nplaybook.univcomm.cornell.edu\npreservenet-stg.cit.cornell.edu\npwg.human.cornell.edu\nrelief.cornell.edu\nresearchupdate.human.cornell.edu\nwiki.ncrn.cornell.edu\nwww.agout.cals.cornell.edu\nwww.firefightergear.human.cornell.edu\nwww.hdtoday.human.cornell.edu\nwww.linkmagazine.human.cornell.edu We are continuing to investigate this incident and are working closely with site owners to resolve some small issues for those sites that are now available. For some off-campus users, it may take longer before the available sites become accessible due to DNS changes.\n\nThe following sites continue to be unavailable. CIT is working closely with each site owner to restore the sites. This may take some time; we will provide updates at the beginning/close of business until all sites have been restored.\n\n4hclothesline.cce.cornell.edu\nagout.cals.cornell.edu\nblog.acsf.cornell.edu\nemotikinz.cit.cornell.edu\nevents.alumni.cornell.edu\nevidencebasedliving.human.cornell.edu\ngofigure.cce.cornell.edu\nkrasnyt.cornell.edu\nlegacyproject.human.cornell.edu\nm.cornell.edu\nmedia.career.cornell.edu\nmobile.cornell.edu\nm-test.alumni.cornell.edu\nncrn.cornell.edu\nphonebook.directory.cornell.edu\nplaybook.univcomm.cornell.edu\npreservenet-stg.cit.cornell.edu\npwg.human.cornell.edu\nrelief.cornell.edu\nresearchupdate.human.cornell.edu\nwiki.ncrn.cornell.edu\nwww.agout.cals.cornell.edu\nwww.firefightergear.human.cornell.edu\nwww.hdtoday.human.cornell.edu\nwww.linkmagazine.human.cornell.edu 2012-01-27 05:00:00
The issue is a load balancer failure. CIT is working with the vendor to resolve the problem as quickly as possible. The issue is a load balancer failure. CIT is working with the vendor to resolve the problem as quickly as possible. 2012-01-26 05:00:00
Efforts are focused on problems with a LAMP server that hosts many websites, among them www.cuinfo.cornell.edu.\n\nThe previously affected load balancer is now functioning, and PeopleSoft is available. Efforts are focused on problems with a LAMP server that hosts many websites, among them www.cuinfo.cornell.edu.\n\nThe previously affected load balancer is now functioning, and PeopleSoft is available. 2012-01-26 05:00:00