Performance: Mail issue for some Office 365 users (fac/staff email system)
Last Updated:
2013-12-03 05:00:00
Event:
2013-11-22 05:00:00
Status:
Closed
Brief Description:
Email
User Impact:
N/A
Workaround:
There is no workaround for this issue
Current Status:
N/A
Services Affected:
Full Description:
CIT is continuing to investigate reports of delayed or missing mail with Microsoft, as a result of Office 365 issues Nov. 22-24. Contact the IT Service Desk (255-5500) if you were affected.
CIT TDX ID:
Timeline of Changes
| Description | Current Status | Date | Time |
|---|---|---|---|
| Microsoft has provided a post-incident report for this issue:\n\nStarting November 22, 2013, at 1:03 PM UTC [8:03 AM EST], Microsoft received reports of an issue in which some customers served from the Americas region experienced delays when receiving email from the Internet.\n\nInvestigation determined that a configuration issue caused email delays. The issue was determined to have begun at 11:30 AM UTC [6:30 AM EST]. \n\nMicrosoft engineers failed over affected tenants to the secondary datacenter, which restored service. The issue was resolved on November 22, 2013, at 4:00 PM UTC [11:00 AM EST]. | Microsoft has provided a post-incident report for this issue:\n\nStarting November 22, 2013, at 1:03 PM UTC [8:03 AM EST], Microsoft received reports of an issue in which some customers served from the Americas region experienced delays when receiving email from the Internet.\n\nInvestigation determined that a configuration issue caused email delays. The issue was determined to have begun at 11:30 AM UTC [6:30 AM EST]. \n\nMicrosoft engineers failed over affected tenants to the secondary datacenter, which restored service. The issue was resolved on November 22, 2013, at 4:00 PM UTC [11:00 AM EST]. | 2013-12-03 | 05:00:00 |
| CIT has not received any new reports about this issue since before the Thanksgiving break. If you believe this issue affected you and you have not already reported it, please contact the IT Service Desk (255-5500). | CIT has not received any new reports about this issue since before the Thanksgiving break. If you believe this issue affected you and you have not already reported it, please contact the IT Service Desk (255-5500). | 2013-12-02 | 05:00:00 |
| We're continuing to investigate reports of delayed or missing mail with Microsoft. Please contact the IT Service Desk at 255-5500.\n\n\n\nPREVIOUS DESCRIPTION:\nA workaround has fixed the issue with some Office 365 users not receiving email and some email sent to them being returned to the sender. Contact the IT Service Desk (255-5500) if you were affected and are missing email. | We're continuing to investigate reports of delayed or missing mail with Microsoft. Please contact the IT Service Desk at 255-5500.\n\n\n\nPREVIOUS DESCRIPTION:\nA workaround has fixed the issue with some Office 365 users not receiving email and some email sent to them being returned to the sender. Contact the IT Service Desk (255-5500) if you were affected and are missing email. | 2013-11-27 | 05:00:00 |
| Microsoft is continuing to work on this issue; Microsoft has confirmed that there was delayed email. \n\nCIT is continuing to seek information from affected users who are either missing email or have received significantly delayed email. Please contact the IT Service Desk at 255-5500. | Microsoft is continuing to work on this issue; Microsoft has confirmed that there was delayed email. \n\nCIT is continuing to seek information from affected users who are either missing email or have received significantly delayed email. Please contact the IT Service Desk at 255-5500. | 2013-11-26 | 05:00:00 |
| CIT is continuing to work with Microsoft on the underlying cause. Affected individuals who believe they are missing email, or who received email that was significantly delayed, are urged to contact the IT Service Desk at 255-5500. This information will help with troubleshooting. | CIT is continuing to work with Microsoft on the underlying cause. Affected individuals who believe they are missing email, or who received email that was significantly delayed, are urged to contact the IT Service Desk at 255-5500. This information will help with troubleshooting. | 2013-11-25 | 05:00:00 |
| Microsoft is continuing to work on this issue. | Microsoft is continuing to work on this issue. | 2013-11-24 | 05:00:00 |
| We have identified the approximately 5,000 accounts affected by this issue and have implemented a workaround. Mail delivery has resumed for these accounts. Microsoft is continuing to work to repair the Office 365 synchronization issue. | We have identified the approximately 5,000 accounts affected by this issue and have implemented a workaround. Mail delivery has resumed for these accounts. Microsoft is continuing to work to repair the Office 365 synchronization issue. | 2013-11-24 | 05:00:00 |
| We believe that all email is now being delivered. For those people affected by this issue, some have experienced delayed email and some have also had email bounced back to the sender. | We believe that all email is now being delivered. For those people affected by this issue, some have experienced delayed email and some have also had email bounced back to the sender. | 2013-11-24 | 05:00:00 |
| CIT is investigating and working with Microsoft.\n\n\nPREVIOUS DESCRIPTION:\nSome individuals have reported that they can send mail but are not receiving mail. A fix is in progress. | CIT is investigating and working with Microsoft.\n\n\nPREVIOUS DESCRIPTION:\nSome individuals have reported that they can send mail but are not receiving mail. A fix is in progress. | 2013-11-23 | 05:00:00 |
| CIT is working with Microsoft to solve this issue. Early indications are that about 1% of users may be affected. Synchronization between Office 365 and CornellAD does not appear to be working correctly. | CIT is working with Microsoft to solve this issue. Early indications are that about 1% of users may be affected. Synchronization between Office 365 and CornellAD does not appear to be working correctly. | 2013-11-23 | 05:00:00 |
| Microsoft has determined that there is a problem with account synchronization in Office 365. The issue has been escalated within Microsoft. | Microsoft has determined that there is a problem with account synchronization in Office 365. The issue has been escalated within Microsoft. | 2013-11-23 | 05:00:00 |
| CIT is investigating | CIT is investigating | 2013-11-22 | 05:00:00 |
| Affected accounts are being fixed. CIT does not yet know what caused the issue. | Affected accounts are being fixed. CIT does not yet know what caused the issue. | 2013-11-22 | 05:00:00 |
| A process has been run to update an internal email address that ceased to function. The process completed at 3:15 pm. Since then, another process (DirSync) has been running to synchronize the updated information with Office 365. \n\nFor some users, mail from non-Cornell Exchange accounts was being returned to the sender as of 5 pm. We believe this has been fixed as of 5:15 pm and primarily affected individuals who use a Cornell Optional Email Alias (for example, schmertz@cornell.edu) as their primary address.\n\nCIT is working with Microsoft. | A process has been run to update an internal email address that ceased to function. The process completed at 3:15 pm. Since then, another process (DirSync) has been running to synchronize the updated information with Office 365. \n\nFor some users, mail from non-Cornell Exchange accounts was being returned to the sender as of 5 pm. We believe this has been fixed as of 5:15 pm and primarily affected individuals who use a Cornell Optional Email Alias (for example, schmertz@cornell.edu) as their primary address.\n\nCIT is working with Microsoft. | 2013-11-22 | 05:00:00 |