Making your public folder migrations faster and more reliable
The key for a successful migration (of any type) is to ensure that source data is in a healthy condition. Public folder migrations are no different, especially if you have been using public folders for years. Orphaned ACLs, mis-matched Mail Enabled Public Folder objects (MEPF’s), or corrupted dumpster folders can cause public folder migrations to slow down considerably, if not fail altogether.How to Configure S/MIME in Office 365
S/MIME in Office 365 S/MIME (Secure/Multipurpose Internet Mail Extensions) is a standard for public key encryption and digital signingof MIME data. Configuring S/MIME in Office 365 is a slightly different procedure than configuring S/MIME on-premises. This blog is for people who want to move from on-premises to Exchange Online and want to continue to use S/MIME. This article will also apply to any Office 365 customers who want to use S/MIME for sending digitally signed and encrypted mails. Configuring S/MIME will allow users to encrypt and/or digitally sign an email. S/MIME provides the following cryptographic security services for electronic messaging applications: authentication, message integrity, non-repudiation of origin (using digital signatures), privacy, and data security (using encryption). Further, Office 365 also provides the capability for end users to compose, encrypt, decrypt, read, and digitally sign emails between two users in an organization using Outlook, Outlook Web App (OWA) or Exchange ActiveSync (EAS) clients. Below, we will take you through the configuration steps that you will need to follow to configure S/MIME for Exchange Online Only (Scenario 1), and for Exchange Hybrid(Scenario 2). Scenario 1: Exchange Online In this scenario, all the users are hosted on cloud and there is no on-premises Exchange organization. Requirements .SST File (Serialized store): The SST file contains all the root and intermediate certificates that are used when validating the S/MIME message in Office 365. The .SST file is created from certificate store explained below. End user’s certificate for signing and encrypting the message issued from Certificate Authorities(CA) either Windows based CA or Third party CA. Configuration Remember that in Exchange Online, only the SST will be used for S/MIME certificate validation. 1. Create a .SST file for the Trusted Root CA / Intermediate CA of the certificate issued to the users: You can use either Certificate MMC or PowerShellcmdlets to export SST file. I am using Certificate console to export the .SST here: Open certmgr.msc snap-in, expand Trusted Root Certificate Authorities > Certificates > select the CA Certificates which issued the certificates to end users for S/MIME and right click > All Tasks > Export… Note: There may be some Intermediate CA’s. You can move them to Trust Root CA folder and select them (including the Trusted CA certificates) and export it all in one .SST file. 2. Select Microsoft Serialized Certificate Store(.SST) > Click Next and save the SST file: 3. Upload .SST to office 365 server: Update the SST on office 365 exchange server by executing the following commands using remote PowerShell. $sst = Get-Content <sst file copied from the box>.sst -Encoding Byte (Example: $sst = Get-Content TenantRoot.sst -Encoding Byte) Set-SmimeConfig -SMIMECertificateIssuingCA $sst 4. Publish user’s certificate to the Exchange Online GAL (Global Address List) using Outlook. If not published, users will not be able to exchange S/MIME encrypted messages. Note: To publish the certificate, the user must first have the certificate installed on their local machine. On the File menu in Outlook 2013, click Options. On the Outlook Options window, click Trust Center, click Trust Center Settings..., and then click Email Security. In the Trust Center window, click Settings… (Here, you need to choose certificate issued by the CA you are going to use for S/MIME). In the Change Security Settings window, type the Security Settings Name (you can name it anything) and choose Signing and Encryption certificate. Select the appropriate certificate assigned in previous steps, leave the Algorithm default and click OK. Once the information is selected, you will notice the Default Setting is populated with Security Settings Name. Now you can click the Publish to GAL button. To publish the certificate to the GAL, click OK. 5. To confirm the certificate is published in AAD (Azure Active Directory), connect to Exchange Online using remote PowerShelland run following command. Check to make sure that the UserSMimeCertificate attribute is populated with the certificate information. If not, return to step 4. Get-Mailbox <user> | FL or FT *user* 6. Once you confirm the end user has the certificate on their machine under certificates > personal store and also published in AAD, the users can use Outlook, OWA, or EASto send and receive S/MIME messages. Note: Make sure you check S/MIME Supported Clients section below before exchanging S/MIME messages. Scenario 2: Exchange Hybrid In Exchange Hybrid topology, some mailboxes are homed on-premises and some mailboxes are homed online, and users share the same e-mail address space. Requirements: Public Key Infrastructure (PKI). You can use Active Directory Certificate Services to issue certificates to the end users. SST File (Microsoft serialized certificate store). Tenant admins will have to configure their tenant in O365 with signing certificates issuing CA & Intermediate certs information. They will have to produce a SST file, which is a collection of certificates, and then later import it into O365 to validate S/MIME. DirSync. You will need version 6593.0012 or higher of the DirSync tool. DirSync is used to synchronize the Active Directory user object to the Azure AD, so that cloud users can also see the certificate information of recipients when performing S/MIME (encrypt) operation. You can verify the DirSync version following these steps: Open Control Panel. Click Programs. Click Programs and Features. Click Windows Azure Active Directory Sync tool. Check the version as the screenshot below: Configuration: 1. Public Key Infrastructure (PKI) The users in your organization must have certificates issued for digitally signing and encryption purposes. You can either install Certificate Authority On-premises to issue certificates to the end users or have third party certificates issued to them. There are two attributes in a user object where certificate information stored: 1) UserCertificate and 2) UserSMimeCertificate. UserCertificateis populated automatically in on-premises deployment with a Windows root CA. This is populated at the time the user enrolls for a user certificate. This could be done manually for each user, or an administrator can set a GPO to automatically enroll all users. Certificates are stored in the userSMimeCertificate attribute when an Outlook client publishes a certificate to GAL. Outlook 2010 and above will populate both attributes with the same certificate http://support.microsoft.com/kb/2840546. But Outlook 2007 and below will not. http://support.microsoft.com/kb/822504 2.When setting a SST file, remember in Exchange online, only the SST will be used for S/MIME certificate validation. Create a SST file for the Trusted Root CA / Intermediate CA of the certificate issued to the users: You can use either Certificate MMC or PowerShellcmdlets to export the SST file. I am using the Certificate console to export the SST here: Open certmgr.msc snap-in, Expand Trusted Root Certificate Authorities > Certificates > select the CA Certificates which issued the certificates to end users for S/MIME, and right click > All Tasks > Export… Note: There may be some Intermediate CA. If there are, move them to Trust Root CA folder and select them, including the Trusted CA certificates, and export them all in one .SST file. Select SST > Click Next and save the SST file: Upload .SST to Office 365 server: Update the SST on Office 365 Exchange server by running the commands below using remote PowerShell: $sst = Get-Content <sst file copied from the box>.sst -Encoding Byte (Example: $sst = Get-Content TenantRoot.sst -Encoding Byte) Set-SmimeConfig -SMIMECertificateIssuingCA $sst 3.If end users are issued third party certificates, they can publish the certificate information to the GAL by following these steps: Note: To publish the certificate, the users must first have the certificate installed on their local machine. On the File menu in Outlook 2013, click Options. On the Outlook Options window, click Trust Center, click Trust Center Settings..., then Email Security. On Trust Center window, click Settings… (Here, you need to choose which certificate you are going to use for S/MIME). In the Change Security Settings window, type the Security Settings Name (you can name it anything), Choose Signing and Encryption certificate, select the appropriate certificate assigned in previous steps, leave the Algorithm default, and click OK. Once the information is selected, you will notice the Default Setting is populated with Security Settings Name. Now you can click the Publish to GAL button. To publish the certificate to the GAL, click OK. To confirm that the certificate is published in AAD (Azure Active Directory), connect to Exchange Online using remote PowerShell and run the following command. Check to see if the UserSMimeCertificate attribute is populated with the certificate information. If not, return to step 4. Get-Mailbox <user> | FL or FT *user* If Windows Certificate Authority is used, then the CA will publish the certificate information into the user object. In both cases, you need to use DirSync to replicate the on-premises Active Directory information to the cloud so that cloud users can exchange S/MIME messages. 4. After the above steps, your end users can use Outlook, OWA, or EASto send and receive S/MIME messages. Note: Make sure you check S/MIME Supported Clients section below before exchanging S/MIME messages. S/MIME Supported Clients All the client machines should have the PKI issued user certificate installed under (whichever is applicable) Certificates - Current User - Personal - Certificates - Trusted Root Certification Authorities - Certificates - Intermediate Certification Authorities - Certificates If the PKI issued certificate is not available, users will not be able to send digitally signed messages or decrypt the S/MIME encrypted messages. Outlook Web App: OWA for S/MIME - Supported only on Windows Vista or greater with browser IE9 and above. Not supported on other browsers or on MOWA (Mobile for Outlook Web Access). Third party certificates aren’t supported for OWA S/MIME; only Windows Certificate Authority issued certificates are supported. To use Outlook Web Access with the S/MIME control, the client system on which the user is running Internet Explorer must have Outlook Web Access with the S/MIME control installed. S/MIME functionality in Outlook Web Access cannot be used on a system that does not have Outlook Web Access with the S/MIME control installed. SMIME control in OWA requires .Net 4.5. All users accessing their mailboxes using OWA should install this on their machine. .Net 4.5 can be installed from Microsoft Downloads page. Outlook Outlook 2010 and above are supported. EAS Clients Windows phone 8.1 is a supported EAS client for S/MIME. To learn how to install a certificate on Windows Phone 8.1, see Installing digital certificates. For any other devices, you need to check with the device vendors. FAQ 1. Do both of these user object attributes (UserSMIMECertificate and UserCertificate) need to be populated with certificate information? Either, or both. 2. Do we support S/MIME for Cross Org/Cross Tenant? Cross Org/Cross Tenant S/MIME is not supported in Outlook Web App and EAS (Exchange Active Sync) With Outlook, it is a supported scenario. A tenant administrator may create contact objects with associated S/MIME public certificates, for users external to their organization that’d synchronize to Office 365 directory. Also, when we are looking for certificates for recipients, we check in all the Address Books. This includes the Global Address Book (GAL), the Contact Address Book (contacts folder), as well as any other address books (which includes LDAP address books). As long as we can find an entry in an address book for the recipient and it contains a certificate that we trust, then we can use it and send S/MIME mail. Note: Certificate in Exchange online GAL (for contact) is supported, however OWA client doesn’t support this scenario at present.. 3. When I select Encrypt mail and click on Send button in Outlook/OWA, I get error saying that the sender does not have a certificate. Why? In the example below, David is a sender. He was trying to send an S/MIME encrypted email message to a couple of recipients who have certificates published in the Active Directory, but David himself doesn’t have a certificate. When he clicks Send, he gets the below error. So, when sending an S/MIME encrypted message, we always check the sender’s certificate so that the message is encrypted such that the sender himself can see it from his Outlook ‘sent items’ folder. References Understanding S/MIME Special thanks to Frank Brown, Mike Brown, Timothy Heeney, Tariq Sharif, Vikas Malhotra and Eduardo Melo for reviewing this post! Suresh KumarWant more control over Sent Items when using shared mailboxes?
Edit 6/15/2015: We are starting to make this feature available again in Office 365. Note that the new shared mailbox Sent Items behavior is disabled by default. If you want to enable it for your users, you can do so by using cmdlets mentioned below. Additionally, we are still on track to ship this to our on-premises customers in Exchange 2013 CU9, as mentioned before. Whether a mailbox is used by multiple users as a collaborative tool or a communication gateway to customers, retaining a record of emails sent from a shared mailbox remains an important business requirement. In Exchange 2010, there was a way to configure this behavior, but we did not have this feature starting with Exchange 2013. Our customers have told us that a shared mailbox should keep a copy of emails sent from the mailbox by all members of the mailbox in its own Sent Items folder. We have taken that feedback and decided to make some changes to how sent mails are handled for shared mailboxes. We are excited to announce that once this feature is enabled for you (see below), by default all shared mailboxes will retain a copy of emails sent from the mailbox. You will no longer have to figure out which mailbox member sent an email as the shared mailbox or on behalf of it. How does it work? Emails can be sent as the shared mailbox itself or on behalf of it by member(s) of the mailbox, assuming proper permissions have been granted. This feature is designed to retain a copy of an email sent from the shared mailbox in the Sent Items folder of the shared mailbox. The same behavior can be expected for emails sent on behalf of the shared mailbox, when configured to do so. A copy of the sent mail will also reside in the Sent Items folder of the member’s personal mailbox. Note: If the user has used the Outlook 2013 feature to change the folder that Sent Items are saved to, the messages will be copied to that folder instead of the user’s Sent Items folder. Users can reconfigure this by clicking the “Save Sent Items To” button on the Email Options tab. Administrators have control over this feature for either mails Sent As or Sent on Behalf of a shared mailbox. The table below summarizes where sent mails reside when members of a shared mailbox send mail from the shared mailbox. User Mailbox Shared Mailbox Sent Items Exchange 2010 Exchange 2010 Controlled through settings in KB2632409 Exchange 2010 Exchange 2013 (any version) Controlled through settings in KB2632409 Exchange 2013 CU9 (or newer) Office 365 Exchange 2010 The sent mail will be delivered to both the Inbox of the shared mailbox as an email attachment* and to the user mailbox' sent items Exchange 2013 CU9 (or newer) Office 365 Exchange 2013 CU9 (or newer) Office 365 The sent mail will be delivered to the Sent Items folder of shared mailbox and to the user mailbox' sent items *In a scenario where the user’s mailbox is on an Exchange 2013 CU9 server (or an Exchange 2013 without CU9 installed) and the shared mailbox is on an Exchange 2010 server, the shared mailbox will get an email message that looks like the following: This behavior will continue until the shared mailbox is moved to an Exchange 2013 CU9 server. Because this can happen even between Exchange 2013 servers (pre-CU9 and CU9), you might want to wait in turning the feature on for your on-premises deployment until you have completely rolled out Exchange 2013 CU9 to all of your servers. Who can use this feature? The feature is going to be available to all customers with shared mailboxes in Office 365 (phased rollout starting on 6/15/2015), as well as our on-premises customers (starting with Exchange 2013 CU9). Note: if after installation of Exchange 2013 CU9 in an on-premises environment you do not see the new CMDlet parameters, you should run /preparead explicitly with CU9 setup. How do I enable/disable this feature? In Office 365 and Exchange 2013 CU9, this feature is disabled by default. Enable the feature In Office 365, using the Admin portal, you can select your shared mailbox by going to Groups > Shared mailboxes first: Then Edit the Sent items copy status of the mailbox: Using PowerShell, for emails Sent As the shared mailbox: set-mailbox <mailbox name> -MessageCopyForSentAsEnabled $True For emails Sent On Behalf of the shared mailbox: set-mailbox <mailbox name> -MessageCopyForSendOnBehalfEnabled $True Disable the feature For messages Sent As the shared mailbox: set-mailbox <mailbox name> -MessageCopyForSentAsEnabled $False For emails Sent On Behalf of the shared mailbox: set-mailbox <mailbox name> -MessageCopyForSendOnBehalfEnabled $False What else do I need to know? The administrator for your organization has to create the shared mailbox and add you to it as an user, before you can use it. If you are an Office 365 Small Business administrator, see Create and use shared mailboxes. If you are an administrator for a different version of Office 365 or an on-premises Exchange administrator, see the TechNet article Create a Shared Mailbox. If you are currently using a client registry key to duplicate messages sent from a shared mailbox to a sent items folder of the sender, you should remember to remove client registry keys if you enable this feature server side. If you do not remove client registry keys, you could see duplicate messages in Sent Items folders (one created by client registry keys and one by the server). If one of the transport rules prevents messages larger than certain size and the message being sent is over that size, then the message will not be delivered to the its original recipient and will also not be copied to the sent items of the shared mailbox. Philip LohDemystifying Hybrid Free/Busy: Finding errors and troubleshooting
EDIT 9/19/2023: This blog post has received significant update. In this second part of the Demystifying Hybrid Free/Busy, we will cover troubleshooting of Hybrid Free/Busy scenarios, more specifically – how and where to find an actual error that will indicate where the problem is. Before venturing forth, please make sure that you have seen Part 1 of this demystifying series! Here is the graphics we posted in the previous post; use this as a reference for users that we will be referring to when troubleshooting: Do you really have a Free/Busy issue? Usually when a user creates a new meeting in Outlook on the web (OWA) or Outlook, clicks on Scheduling Assistant, adds his or her colleague to the meeting, they try to see when the user is available to meet. If they see the hash marks \\\\\\\ instead of seeing if the other user is free or busy, there is an issue. Here, we do seem to have a bunch of Free/Busy issues: You can often see an error message by hovering over hash marks, however we usually find that the error is not very specific. Instead, we would need to take slightly more advanced steps to diagnose the issues by checking things like the Remote Connectivity Analyzer tool, Fiddler, F12 Network tab, Outlook logging or SARA tool. Where is the actual Free/Busy error message? First, we need to understand in which direction we have a lookup problem. Please see Part 1 for discussion of directionality. Sources of logs: Remote Connectivity Analyzer tool Outlook logging SARA tool OWA F12 Network Tab Fiddler – Outlook and OWA These steps are important for us to see the relevant message error for Free/Busy issues. Once we know the error message, it’s much easier to resolve the issue. Remote Connectivity Analyzer A few things to know about this tool: Source Mailbox: the user that will be requesting the free/busy information. This will be the user that is logged in Outlook or OWA and cannot see free/busy for other people. This is also called Requester or Organizer of the meeting. Authentication type for Source Mailbox: you will choose Modern Authentication Source Mailbox credentials: you will need to authenticate with the credentials of the Source Mailbox. The tool doesn’t support Basic Authentication for Exchange Online mailboxes because this is disabled in Exchange Online. While it is still used by Exchange On-premises environments, currently, if you select Basic Authentication for the on-premises source mailbox, the test will fail before doing the actual Free/Busy process. It works if your Exchange on-premises has enabled Modern Authentication for client protocols. In conclusion, Source Mailbox login needs to be using OAuth for this test to work, regardless of where it is hosted. Target Mailbox: the user that the Source Mailbox is requesting free/busy for. This is the Attendee of the meeting. The tool simulates Outlook’s way of querying Free/Busy. If you have a free/busy issue that is only happening in OWA but not in Outlook Desktop, then this test will likely not catch the error. To be able to perform the test, you must allow connectivity for the Remote Connectivity Analyzer tool’s IP addresses. These are part of the "Microsoft 365 Common and Office Online" ranges published in the Office 365 URLs and IP address ranges. The IPs for the Remote Connectivity Analyzer are part of the range specified as "Allow Required" (currently ID 46 in the documentation). Check https://testconnectivity.microsoft.com/Pages/ChangeList.htm for any future changes. Note that you can only insert one Target Mailbox email address per test. If you have errors for multiple target mailboxes, run multiple tests, for each user. Connectivity Test Results: With these 3 buttons on the top right corner, you can expand all the results and save them as XML or HTML files. Usually, support people appreciate these files a lot, so please do upload them in your support case workspace. When you expand the results, there are 3 important checks: Determining where the source mailbox is hosted (cloud or not). If the Mailbox is hosted in cloud, you will see something like this: IsOffice365Mailbox=True. The mailbox is hosted in Office 365. <ASURL>https://outlook.office365.com/EWS/Exchange.asmx</ASURL> If the Mailbox is not hosted in cloud, you will see something like this: IsOffice365Mailbox=False. The mailbox isn't hosted in Office 365. Determining where the target mailbox is hosted (cloud or not). Test Autodiscover for the Target Mailbox SMTP to retrieve External EWS url. Quick tip: on your side, in Windows PoweShell, you can also use the following commands to see the External EWS url of an user based on the Autodiscover call to Office 365, replace what is in Email= with your actual email addresses. Invoke-RestMethod -Uri "https://outlook.office365.com/autodiscover/autodiscover.json?Email=CLOUDUSER@CONTOSO.COM&Protocol=EWS" Invoke-RestMethod -Uri "https://outlook.office365.com/autodiscover/autodiscover.json?Email=ONPREMUSER@CONTOSO.COM&Protocol=EWS" Performing the Free/Busy Lookup. This will be Success or Failed. If it failed, look under the Additional details to see the error message. If success, be happy, maybe the issue is resolved, or not be happy as it might be an intermittent issue (which is harder to troubleshoot) or a local issue only (happening in your specific network, machine, Outlook version). In my case, I see that I have a NoFreeBusyAccessException, given by the Exchange on-premises server HHE1601. OUTLOOK Note: The Modern Outlook clients log Free/Busy information in Outlook ETL files and you won’t be able to see the Free/Busy error in plain text here. This was possible with Outlook 2010 logs, back in the old days. But this method is still useful, because you can provide the Outlook ETL log containing the error to Microsoft Support to parse it for you and help you fix it also. If you want to see the error for yourself, check the Fiddler method. For the Outlook F/B error, we need to first enable Outlook logging and after this we will need to reproduce the issue (\\\\\\). After repro, we will collect the Outlook logs. Steps: Enable Outlook logging: Follow this KB article and check the “Enable troubleshooting logging (this requires restarting Outlook)” option. Restart Outlook. Reproduce the issue for the non-working free/busy direction. Suppose Free/Busy direction not working is cloud to on-premises, you will be logged on as a cloud user (Source Mailbox), go to Calendar tab, New Meeting, Scheduling Assistant, add some on-premises users to a meeting until you see the hash marks (instead of Free/Busy information). You do not need to save or send a meeting request. Collect the Outlook-#####.etl log from %temp%\Outlook Logging folder (reference here). You would need to send the ETL file to Microsoft Support to get it analyzed as we are parsing this log with an internal tool. You might not know this, but Hybrid free/busy support cases are free of charge! Of course, you can still use the other methods (fiddler for Outlook/OWA or browser for OWA) to see Free/Busy error yourself, however we (Support) might ask you additionally to get this log as well for a further dive into the Free/Busy errors. SARA I would also like to mention that there is a Free/Busy troubleshooter in Beta version, incorporated into SARA tool (Microsoft Support and Recovery Assistant for Office 365) which you can download it from here : https://diagnostics.outlook.com/#/ Open SARA and select Outlook scenario, click Next, then select I’m having problems with my calendar, input email address and password of the source mailbox (cloud mailbox if direction not working is cloud > on-premises) and then select I can’t see when someone is free or busy. Due to the underlying complexity of it all, this is not a completely reliable way of determining the cause of free/busy issues in Hybrid Deployments, but it is a good start when troubleshooting. This F/B test from SARA covers mostly cloud to cloud scenarios but I recommend it here because it does connectivity and additional checks on tenant, licensing and Autodiscover. And sometimes it shows the underlying Free/Busy error message. Here are some screenshots with the SARA process: After the Office 365 readiness checks, the tool will ask you for the email address of the Target Mailbox: In the failed results, expand the Support Message and User Message: OWA / Outlook on the web F12 Network Tab Cloud OWA F12 Network tab You need to login to OWA as the source mailbox, hit F12 (Developer Tools for browser) and select the Network Tab. You would then lookup Free/Busy for the target mailbox (reproduce the issue). If the source mailbox is hosted in Cloud, to look for the F/B here, you can find the Search Icon and type there “GetSchedule” or find the Filter Icon and type “graphql”, then look at Response or Preview tab to see the error message by expanding GetSchedule until you reach to the error. (click thumbnail to view larger) If the Source Mailbox is hosted in Exchange On-Premises, you would look after GetUserAvailabilityInternal: Fiddler –Outlook or OWA You would need to download and install Fiddler tool from the internet, enable HTTPS decryption in Fiddler and then reproduce the Free/Busy issue in Outlook or OWA or both. Fiddler - Exchange Online Source Mailbox logged in Outlook desktop. Look for “GetUserAvailability” calls and then on the right side, you have Request on the top and Response on the bottom. Switch to XML tabs for a nicer view. In the Request you will see the attendees’ email addresses and, in the Response, you will have ResponseMessage with ResponseClass=Error or ResponseClass=Success. Fiddler – Exchange Online Source Mailbox logged in OWA. In Fiddler, you can check in the Request pane, under Raw tab the ClientRequestID, you can for example search after this specific value in your on-premises Exchange server logs: IIS W3SVC2 logs, HTTPProxy EWS logs and EWS logs (more information on these logs, location and extracts, later in the article). Example here from a lab: ClientRequestID: {72741DFF-A6AC-402B-991B-C6B5D56B1422} Date: Mon, 11 Sep 2023 19:01:25 GMT If you are fan of SQL language, you can use a tool like Log Parser Studio and search through these logs, for example, here is a query on the ClientRequestID from earlier: SELECT DateTime, ClientRequestID, RequestID, UserAgent, SoapAction, ErrorCode, GenericErrors, GenericInfo, FileName FROM '[LOGFILEPATH]' WHERE ClientRequestID LIKE '%{72741DFF-A6AC-402B-991B-C6B5D56B1422}%' You can also use findstr.exe utility to look for the client request id or other keywords like the requester’s email address or “CrossForest”. Example of command: findstr.exe /I /S "{72741DFF-A6AC-402B-991B-C6B5D56B1422}" *.log When troubleshooting Free/Busy issues, the following on-premises logs can be very useful, especially for Cloud to On-Premises Free/Busy direction. IIS logs Default Web Site (DWS) Path: %SystemDrive%\inetpub\logs\LogFiles\W3SVC1 Path example: C:\inetpub\logs\LogFiles\W3SVC1 Extract of Autodiscover and EWS log entries with IOC Enabled in IIS W3SVC1 logs: Autodiscover – OAUTH (autodiscover.svc without /WSSecurity) 2016-01-06 17:45:27 10.0.0.5 POST /autodiscover/autodiscover.svc &CorrelationID=<empty>;&ClientId=QNFNHKEEKYENCJITQQ&cafeReqId=7972d1fc-a9d9-44c6-8851-480d3601cbd7; 443 S2S~00000002-0000-0ff1-ce00-000000000000 132.245.65.28 ASAutoDiscover/CrossForest/EmailDomain//15.01.0361.007 200 0 0 109 EWS – OAUTH (exchange.asmx without /WSSecurity) 2016-01-06 17:45:27 10.0.0.5 POST /ews/exchange.asmx &CorrelationID=<empty>;&ClientId=WSIVGUUAUWWRFACJBWDA&cafeReqId=6ce8864c-74a0-4ad2-a3dc-7b69e0415403; 443 <unverified>actas1(sip:joe@contoso.com|smtp:joe@contoso.com|upn:joe@contoso.com) 132.245.65.28 ASProxy/CrossForest/EmailDomain//15.01.0361.007 200 0 0 703 Example of EWS entry with Organization Relationship Enabled in IIS W3SVC1 logs: EWS – DAUTH (exchange.asmx with /WSSecurity) 2016-01-06 18:04:41 10.0.0.5 POST /ews/exchange.asmx/WSSecurity &CorrelationID=<empty>;&ClientId=VOMGJKAWURSVKOXQLBVA&cafeReqId=18fd3a2e-7b1c-4828-8943-6b20912e2e44; 443 - 132.245.65.28 ASProxy/CrossForest/EmailDomain//15.01.0361.007 200 0 0 296 IIS logs Exchange BackEnd (BE) Path: %SystemDrive%\inetpub\logs\LogFiles\W3SVC2 Path example: C:\inetpub\logs\LogFiles\W3SVC2 Example of EWS entry with Organization Relationship Enabled (DAUTH) in IIS W3SVC2 logs: 2016-01-06 18:04:41 fe80::f17f:beef:a5e3:7d3c%25 POST /ews/exchange.asmx/WSSecurity - 444 - fe80::f17f:beef:a5e3:7d3c%25 ASProxy/CrossForest/EmailDomain//15.01.0361.007 200 0 0 93 HTTPProxy logs for Autodiscover Path: %ExchangeInstallPath%Logging\HttpProxy\Autodiscover Path example: C:\Program Files\Microsoft\Exchange Server\V15\Logging\HttpProxy\Autodiscover Example of Autodiscover entry with Organization Relationship Enabled (DAUTH) 2016-01-06T18:05:20.552Z,bcdfbed5-f11f-4250-a616-e38cb475cd3f,15,0,1104,2,,Autodiscover,autodiscover.contoso.com,/autodiscover/autodiscover.svc /WSSecurity,,,false,,contoso.com,Smtp~joe@contoso.com,ASAutoDiscover/CrossForest/EmailDomain/ /15.01.0361.007,132.245.65.28,exch-2013,200,200,,POST,Proxy,exch-2013.contoso.com,15.00.1104.000,IntraForest,AnchorMailboxHeader-SMTP,[…],BeginRequest=2016-01-06T18:05:20.192Z;CorrelationID=<empty>;ProxyState-Run=None;FEAuth=BEVersion-1941996624;NewConnection=fe80::f17f:beef:a5e3:7d3c%25&0; HTTPProxy logs for EWS Path: %ExchangeInstallPath%Logging\HttpProxy\Ews Path example: C:\Program Files\Microsoft\Exchange Server\V15\Logging\HttpProxy\Ews Example of EWS entry with Organization Relationship Enabled (DAUTH): 2016-01-06T18:04:41.490Z,4757ab2c-8ccc-4d1a-ae39-0780ecc8eabb,15,0,1104,2,{02CD833F-18AB-413A-83CB-0E86F4DA5362},Ews,mail.contoso.com,/ews/exchange.asmx/WSSecurity,,,false,,contoso.com, Smtp~joe@contoso.com,ASProxy/CrossForest/EmailDomain//15.01.0361.007,132.245.65.28,exch-2013,200,200,,POST,Proxy,exch-2013.contoso.com,15.00.1104.000,IntraForest,AnchorMailboxHeader-SMTP,[…],BeginRequest=2016-01-06T18:04:41.380Z; EWS logs Path: %ExchangeInstallPath%Logging\Ews Path example: C:\Program Files\Microsoft\Exchange Server\V15\Logging\Ews Example of EWS entry with Organization Relationship Enabled (DAUTH): 2016-01-06T18:04:41.490Z,4757ab2c-8ccc-4d1a-ae39-0780ecc8eabb,15,0,1104,2,{02CD833F-18AB-413A-83CB-0E86F4DA5362}, External,true,jane@contoso.mail.onmicrosoft.com,, ASProxy/CrossForest/EmailDomain//15.01.0361.007,Target=None;Req=Exchange2012/Exchange2013; ,132.245.65.28,exch-2013,exch-2013.contoso.com,GetUserAvailability,200,12150,,,,,,ebd34d71ac7342c19d947d881db4ad55,f866c73e-6c91-475e-bdec-0428bdeaa423,PrimaryServer; Requester=jane@contoso.mail.onmicrosoft.com; Failures=0 Event Viewer Application logs on Exchange Server References here and here. Example of Event ID 4002 for MSExchange Availability: Log Name: Application Source: MSExchange Availability Event ID: 4002 Task Category: Availability Service Level: Error Description: Process 4568: ProxyWebRequest CrossSite from S-1-5-21-391720751-1508397712-925700815-508779 to https://hybrid.contoso.com/ews/exchange.asmx failed. Caller SIDs: NetworkCredentials. The exception returned is Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequestProcessingException: System.Web.Services.Protocols.SoapException: You have exceeded the available concurrent connections for your account. Try again once your other requests have completed. at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall) IIS tracing for the error code in the IIS logs Reference here. Free/Busy errors and fixes Based on cumulative support team experience, we created a table (see the attachment to this post) with Free/Busy errors encountered so far and their possible resolutions. We cannot cover all possible scenarios and errors even though we have a good-sized list. This is meant to illustrate ways we can resolve specific errors and these suggestions might not work for you even if you have the same error. If you know the exact Free/Busy error that you get and checked configuration as discussed in part 1 of this series, this is already a tremendous progress, and this will help us resolve your issue faster. Of course, you can follow these suggestions on your own as most of the actions are harmless but if you don’t feel confident in troubleshooting on your own or you fear that actions are dangerous or irreversible, please contact us. Free/Busy Errors discussed in the attached document (FB_Errors_FixesV7): “An internal server error occurred. The operation failed” LID: 59916. 500 Internal Server error. "The remote user mailbox must specify the the explicit local mailbox in the header" "An error occurred when verifying security for the message" "Unable to connect to the remote server" “Autodiscover failed for email address <> with error ‘The request failed with HTTP status 404: Not Found’ ” “The request failed with HTTP status 401: Unauthorized - The user specified by the user-context in the token is ambiguous” LID: 43532 "An existing connection was forcibly closed by the remote host - An unexpected error occurred on a receive " "An existing connection was forcibly closed by the remote host - An unexpected error occurred on a send ” "Configuration information for forest/domain could not be found in Active Directory" "Proxy web request failed.,inner exception: The request failed with HTTP status 401: Unauthorized." "The response from the Autodiscover service at 'https://autodiscover/autodiscover.svc/WSSecurity' failed due to an error in user setting 'ExternalEwsUrl'. Error message: InvalidUser." LID: 33676 “The caller does not have access to free/busy data" LID: 47652 LID: 44348 “The request failed with HTTP status 403: Forbidden (The server denied the specified Uniform Resource Locator (URL). “ LID: 43532 “Unable to resolve e-mail address user@notes.domain.com to an Active Directory object” LID: 57660 “An error occurred when processing the security tokens in the message.” LID: 59916 “The cross-organization request for mailbox yyy@contoso.com is not allowed because the requester is from a different organization” LID: 39660 “The request failed with HTTP status 401: Unauthorized - Microsoft.Exchange.Security.OAuth.OAuth TokenRequestFailedException: Missing signing certificate “ “The application is missing a linked account for RBAC roles, or the linked account has no RBAC role assignments, or the calling users account is logon disabled” “The entered and stored passwords do not match“ “The password has to be changed.” “The password for the account has expired” or “Provision is needed before federated account can be logged in” “The request timed out” “The specified member name is either invalid or empty” “The result set contains too many calendar entries” LID: 54796 “The request failed with HTTP status 401: Unauthorized - The token has an invalid signature.” “The request failed with HTTP status 401: Unauthorized - Client assertion contains an invalid signature. [Reason - The key was not found., Thumbprint of key used by client: '<>’ “ “Proxy web request failed., inner exception: Response is not well-formed XML “ “Failed to communicate with https://login.microsoftonline.com/extSTS.srf., inner exception: Unable to connect to the remote server” “Autodiscover failed for E-Mail Address <> with error System.Net.WebException: The remote name could not be resolved: '<>'” “Failed to get ASURL. Error 8004010F” “Proxy web request failed. , inner exception: System.Net.WebException: The request failed with the error message: -- <head><title>Object moved” “The request was aborted: Could not create SSL/TLS secure channel.” “The user specified by the user-context in the token does not exist.";error_category="invalid_user“ “The hostname component of the audience claim value 'https://<>’ is invalid";error_category="invalid_resource“ “Proxy web request failed. , inner exception: System.Net.WebException: The request failed with HTTP status 503: Service Unavailable” “Proxy web request failed. , inner exception: System.Net.WebException: The request failed with HTTP status 504: Gateway Timeout.” Thanks to all that contributed to this content: Ray Fong, Nino Bilic, Tim Heeney, Greg Taylor and Brian Day. Mirela BuruianaExchange, Firewalls, and Support… Oh, my!
Over the years Exchange Server architecture has gone through a number of changes. As a product matures over time you may see us change what is supported as we react to changes in the product architecture, the state of technology as a whole, or major support issues we see come in through our support infrastructure. Over the years a large volume of support calls have ended up being caused by communication issues between Exchange servers or between Exchange servers and domain controllers. Often times this results from a network device between the servers not allowing some port or protocol to communicate to the other servers. I tried to get Harrison Ford to co-write this article with me given his specific talents, but alas he was busy and regretfully couldn’t partake. Please allow me to start with the short version up front so there is no confusion about what we currently DO and DO NOT support before I lose some of you to; Image Courtesy of: http://knowyourmeme.com/memes/tldr Starting with Exchange Server 2007 and current as of Exchange Server 2013, having network devices blocking ports/protocols between Exchange servers within a single organization or between Exchange servers and domain controllers in an organization is not supported. A network device may sit in the communication path between the servers, but a rule allowing “ANY/ANY” port and protocol communication must be in place allowing free communication between Exchange servers as well as between Exchange servers and domain controllers. For Exchange Server 2010 this is already articulated at http://technet.microsoft.com/en-us/library/bb331973(v=EXCHG.141).aspx under Client Access Server Connectivity in the Client Access Server section in the following paragraph. “In addition to having a Client Access server in every Active Directory site that contains a Mailbox server, it’s important to avoid restricting traffic between Exchange servers. Make sure that all defined ports that are used by Exchange are open in both directions between all source and destination servers. The installation of a firewall between Exchange servers or between an Exchange 2010 Mailbox or Client Access server and Active Directory isn’t supported. However, you can install a network device if traffic isn’t restricted and all available ports are open between the various Exchange servers and Active Directory.” Why has this seemingly simple support statement become so muddied and confusing over the years? Maybe we didn’t make it blunt enough to start with, but there could be some other compounding points adding to the confusion. Confusion point #1… Exchange Server 2003 was the last version of Exchange Server to allow deploying (at the time) a Front-End server in a perimeter network (aka DMZ) while locating the Back-End server in the intranet. While this could be made to work it required a specialized set of rules that essentially turned your perimeter network security model into the following: Image Courtesy of: The Internet During the time of Exchange Server 2003 adoption of reverse proxies within perimeter networks was on the rise. Reverse proxies allowed customers to more securely publish Exchange Server for remote access while only allowing a single port and protocol to traverse from the Internet to the perimeter network, and then a single port and protocol to traverse from the perimeter network to the intranet. You could go from something complicated like this with endless port and protocol requirements…. Figure 1: Legacy Exchange 2003 deployment with Front-End server in a perimeter network. What a mess. Who’s hungry? To something simple like this… Figure 2: Reverse proxy in the perimeter network and all Exchange servers within the intranet. Simplicity at its best! The resulting increase in simplicity as well as the drop in support cases was strong enough for Microsoft to determine during the lifecycle of the next major version of Exchange Server, 2007, that we would no longer support deploying what is now the Client Access Server role in a perimeter network. From that time on all Exchange servers, except for Edge Transport Server role, were to be deployed on the intranet with unfettered access to each other. We do have this documented in http://technet.microsoft.com/en-us/library/bb232184.aspx. Confusion point #2… TechNet includes a number of articles that document many if not all of the ports and protocols Exchange Server utilizes during the course of normal operation. These documents are often misunderstood as “configure your firewall this way” articles. The information is only being provided for educational purposes on the inner-workings of Exchange Server, or to aid with the configuration of load balancing or service monitoring mechanisms which often require specific port/protocol definitions to perform their functions correctly. In case you come across them in the future, here is a list of most of those articles. Exchange Network Port References Exchange Server 2000: http://support.microsoft.com/kb/278339/ Exchange Server 2003: http://technet.microsoft.com/en-us/library/bb124075(v=exchg.65).aspx Exchange Server 2007: http://technet.microsoft.com/en-us/library/bb331973(v=exchg.80).aspx Exchange Server 2010: http://technet.microsoft.com/en-us/library/bb331973(v=exchg.142).aspx Exchange Server 2013: https://technet.microsoft.com/library/bb331973(v=exchg.150).aspx Understanding Protocols, Ports, and Services in Unified Messaging Exchange Server 2007: http://technet.microsoft.com/en-us/library/aa998265(v=exchg.80).aspx Exchange Server 2010: http://technet.microsoft.com/en-us/library/aa998265(v=exchg.141).aspx Exchange Server 2013: Currently no plans to create. I don’t trust my clients and I like restricting their access. Is that supported? This is a different story and yes there are things you can do here to remain supported. Exchange Server has for a number of revisions supported configuring static client communication ports for Windows based Outlook clients. After the client contacts the endpoint mapper service running on Windows under TCP Port 135 it will be handed back the static TCP port you have chosen to use in your environment. For Exchange Server 2010 you may be familiar with the following article describing how to configure static client communication ports for the Address Book Service and the RPC Client Access Service, thereby leaving you with 4 ports required for clients to operate in MAPI/RPC mode. TCP 135 for the Endpoint Mapper TCP 443 for Autodiscover/EWS/ECP TCP <your choice #1> for the Address Book service TCP <your choice #2> for the RPC Client Access service UDP ANY from CAS to the Outlook 2003 client if you’re in online mode and utilizing UDP notifications. http://social.technet.microsoft.com/wiki/contents/articles/864.configure-static-rpc-ports-on-an-exchange-2010-client-access-server.aspx TechNet also has resources for versions prior to Exchange Server 2010: http://support.microsoft.com/kb/270836. Starting in Exchange Server 2013 the only protocol supported for Windows Outlook clients is RPC over HTTPS. This architectural change reduces your required port count to one, TCP 443 for HTTPS, to be utilized by Autodiscover, Exchange Web Services, and RPC over HTTPS (aka Outlook Anywhere). This is going to make your life easy, but don’t tell your boss as they’ll expect you to start doing other things as well. It’ll be our secret. Promise. I’ll go through some examples of supported deployments, but will keep it easy and only use Outlook clients. The same ideas apply to other POP/IMAP/EAS clients as well, just don’t restrict Exchange servers from talking to each other. A setup like the following Outlook 2010 / Exchange 2010 diagram would be entirely supported where we have a firewall between the clients and the servers. In all of the following examples I have chosen static TCP port 59531 for my RPC Client Access Service on CAS and Mailbox, and static TCP port 59532 for my Address Book Service on CAS. UDP notifications are also thrown in for fun for those of you running Outlook 2003 in Online Mode, which I hope is very few and far between these days. Domain controllers were left out of these diagrams to focus on communication directly between clients and Exchange, and load balancers were also kept out for simplicity. Figure 3: Firewall between clients and all Exchange servers. Supported if firewall is configured correctly to allow all necessary client access. AD not shown. However, if you attempted to do something naughty like the following diagram and for reasons unknown to us put a firewall between CAS and Mailbox then there had better be ANY/ANY rules in place allowing conversations to originate from either side between Exchange servers. Figure 4: Firewall between CAS and other Exchange servers. Supported only if the firewall is configured for unfettered access between Exchange servers, and Exchange servers and AD. AD not shown. Well what if you have multiple datacenters with Exchange and want to firewall everything everywhere because you believe that as the number of firewalls goes up your security must exponentially increase? We’ve got you covered there too, deploy it like this where you’ll see both MAPI/RPC and RPC/HTTPS user examples. I didn’t bother putting load balancers or Domain Controllers into any of these diagrams by the way. I’m putting faith in all of you that you know where those go. Figure 5: Firewalls between users and Exchange servers as well as between datacenters. Supported if the firewalls are configured to allow unfettered access between Exchange servers, between Exchange servers and AD, and appropriate client rules. AD not shown. Boy this is going to be easy when all of you migrate to Exchange Server 2013 and are only dealing with RPC/HTTPS connections from clients and SMTP or HTTPS between servers. Except for maybe those pesky POP/IMAP/UM clients… Figure 6 below depicts what Exchange 2013 network conversations may look like at a high level. A load balancer and additional CAS were introduced to show we don’t care what CAS a client’s traffic goes through as they all end up at the same Mailbox Server anyways where the user’s database is mounted. You may have read previously Exchange Server 2013 does not require affinity for client traffic and hopefully this visual helps show why. The one tricky bit to consider if placing a firewall in between clients and Exchange Server 2013 would be UM traffic as it is not all client to CAS in nature. In Exchange Server 2013 a telephony device first makes a SIP connection through CAS (orange arrows) which after speaking with the UM Service on Mailbox Server will redirect the client so it may establish a direct SIP+RTP session (blue arrow) to the Mailbox Server holding the user’s active database copy for the RTP connection. Figure 6: Showing at a high level SMTP, Windows Outlook Client, and UM traffic with a firewall between users and Exchange Server 2013. So, Microsoft, if you’re saying this should be simple then what can I do and remain in a supported state? The key here is to not block traffic between Exchange servers, or between Exchange servers and Domain Controllers. As long no traffic blocking is performed between these servers you will be in a fully supported deployment and will not have to waste time with our support staff proving you really do have all necessary communications open before you can start to troubleshoot an issue. We know many customers will continue to test the boundaries of supportability regardless, but be aware this may drag out your troubleshooting experience and possibly extend an active outage. We prefer to help our customers resolve any and all issues as fast as possible. Staying within support guidelines does in fact help us help you as expeditiously as possible, and in the end will save you time, support costs, labor costs, and last but not least aggravation. Brian Day Program Manager Exchange Customer ExperienceLog Parser Studio 2.0 is now available
Since the initial release of Log Parser Studio (LPS) there have been over 30,000 downloads and thousands of customers use the tool on a daily basis. In Exchange support many of our engineers use the tool to solve real world issues every day and in turn share with our customers, empowering them to solve the same issues themselves moving forward. LPS is still an active work in progress; based on both engineer and customer feedback many improvements have been made with multiple features added during the last year. Below is a short list of new features: Improved import/export functionality For those who create their own queries this is a real time-saver. We can now import from multiple XML files simultaneously only choosing the queries we wish to import from multiple query libraries or XML files. Search Query Results The existing feature allowing searching of queries in the library is now context aware meaning if you have a completed query in the query window, the search option searches that query. If you are in the library it searches the library and so on. This allows drilling down into existing query results without having to run a new query if all you want to do is narrow down existing result sets. Input/Output Format Support All LP 2.2 Input and Output formats contain preliminary support in LPS. Each format has its own property window containing all known LP 2.2 settings which can be modified to your liking. Exchange Extensible Logging Support Custom parser support was added for most all Exchange logs. These are covered by the EEL and EELX log formats included in LPS which cover Exchange logs from Exchange 2003 through Exchange 2013. Query Logging I can't tell you how many times myself or another engineer spent lots of time creating the perfect query for a particular issue we were troubleshooting, forgetting to save the query in the heat of the moment and losing all that work. No longer! We now have the capability to log every query that is executed to a text file (Query.log). What makes this so valuable is if you ran it, you can retrieve it. Queries There are now over 170 queries in the library including new sample queries for Exchange 2013. PowerShell Export You can now export any query as a standalone PowerShell script. The only requirement of course is that Log Parser 2.2 is installed on the machine you run it on but LPS is not required. There are some limitations but you can essentially use LPS as a query editor/test bed for PowerShell scripts that run Log Parser queries for you! Query Cancellation The ability to submit a request to cancel a running query has been added which will allow you to cancel a running query in many cases. Keyboard Shortcuts There are now 23 Keyboard shortcuts. Be sure to check these out as they will save you lots of time. To display the short cuts use CTRL+K or Help > Keyboard Shortcuts. There are literally hundreds of improvements and features; far too many to list here so be sure and check out our blog series with existing and upcoming tutorials, deep dives and more. If you are installing LPS for the first time you'll surely want to review the getting started series: Getting started with Log Parser Studio Getting started with Log Parser Studio, part 2 Getting started with Log Parser Studio, part 3 If you are already familiar with LPS and are installing this latest version, you'll want to check out the upgrade blog post here: Log Parser Studio: upgrading from v1 to v2 Additional LPS articles can be found here: http://blogs.technet.com/b/karywa/ LPS doesn't require an install so just extract to the folder of your choice and run LPS.EXE. If you have the previous version of LPS and you have added your own custom queries to the library, be sure to export those queries as a backup before running the newest version. See the "Upgrading to LPS V2" blog post above when upgrading. Kary Wall
