Outlook Connectivity with MAPI over HTTP
Among the many new features delivered in Exchange 2013 SP1 is a new method of connectivity to Outlook we refer to as MAPI over HTTP (or MAPI/HTTP for short). We’ve seen a lot of interest about this new connection method and today we’ll give you a full explanation of what it is, what it provides, where it will take us in the future, and finally some tips of how and where to get started enabling this for your users. What is MAPI over HTTP? MAPI over HTTP is a new transport used to connect Outlook and Exchange. MAPI/HTTP was first delivered with Exchange 2013 SP1 and Outlook 2013 SP1 and begins gradually rolling out in Office 365 in May. It is the long term replacement for RPC over HTTP connectivity (commonly referred to as Outlook Anywhere). MAPI/HTTP removes the complexity of Outlook Anywhere’s dependency on the legacy RPC technology. Let’s compare the architectures. MAPI/HTTP moves connectivity to a true HTTP request/response pattern and no longer requires two long-lived TCP connections to be open for each session between Outlook and Exchange. Gone are the twin RPC_DATA_IN and RPC_DATA_OUT connections required in the past for each RPC/HTTP session. This change will reduce the number of concurrent TCP connections established between the client and server. MAPI/HTTP will generate a maximum of 2 current connections generating one long lived connection and an additional on-demand short-lived connection. Outlook Anywhere also essentially double wrapped all of the communications with Exchange adding to the complexity. MAPI/HTTP removes the RPC encapsulation within HTTP packets sent across the network making MAPI/HTTP a more well understood and predictable HTTP payload. An additional network level change is that MAPI/HTTP decouples the client/server session from the underlying network connection. With Outlook Anywhere connectivity, if a network connection was lost between client and server, the session was invalidated and had to be reestablished all over again, which is a time-consuming and expensive operation. In MAPI/HTTP when a network connection is lost the session itself is not reset for 15 minutes and the client can simply reconnect and continue where it left off before the network level interruption took place. This is extremely helpful for users who might be connecting from low quality networks. Additionally in the past, an unexpected server-side network blip would result in all client sessions being invalidated and a surge of reconnections being made to a mailbox server. Depending on the number of Outlook clients reconnecting, the re-establishing of so many RPC/HTTP connections might strain the resources of the mailbox server, and possibly extend the outage in scope (to Outlook clients connected to multiple servers) and time, caused by a single server-side network blip. Why MAPI over HTTP? You are probably asking yourself why the Exchange team would create a complete replacement for something so well-known and used. Let us explain. The original Outlook Anywhere architecture wasn’t designed for today’s reality of clients connecting from a wide variety of network types – many of these are not as fast or reliable as what was originally expected when Outlook Anywhere was designed. Consider connections from cellular networks, home networks, or in-flight wireless networks as a few examples. The team determined the best way to meet current connection needs and also put Exchange in the position to innovate more quickly was to start with a new simplified architecture. The primary goal of MAPI/HTTP is provide a better user experience across all types of connections by providing faster connection times to Exchange – yes, getting email to users faster. Additionally MAPI/HTTP will improve the connection resiliency when the network drops packets in transit. Let’s quantify a few of these improvements your users can expect. These results represent what we have seen in our own internal Microsoft user testing. When starting Outlook users often see the message “Connecting to Outlook” in the Outlook Status bar. MAPI/HTTP can reduce the amount of time a user waits for this connection. In the scenario when a user first launches Outlook the time to start synchronization improved to 30 seconds vs. 90 seconds for Outlook Anywhere for 70% of the monitored clients. Improvements are also delivered when clients are resuming from hibernation or simply re-connecting to a new network. Testing showed that 80% of the clients using MAPI/HTTP started syncing in less than 30 seconds vs. over 40 seconds for Outlook Anywhere clients when resuming from hibernation. This improvement was made possible as MAPI/HTTP implements a pause/resume feature enabling clients to resume using an existing connection rather than negotiating a new connection each time. Current sessions for MAPI/HTTP are valid for 15 minutes, but as we fine tune and expand this duration, these improvements will be even more noticeable. Improvements aren’t limited to end users. IT administrators will gain greater protocol visibility allowing them to identify and remediate situations faster and with more confidence. Due to MAPI/HTTP moving to a more traditional HTTP protocol payload, the ability to utilize already known tools common to HTTP debugging is a reality. IIS and HttpProxy logs will now contain information similar to other HTTP based protocols like Outlook Web App and be able to pass information via headers. At times in the past, certain debug procedures for RPC/HTTP were only available via proprietary internal Microsoft tools. This move should put all customers on the same level playing field as far as what tools are available for debug purposes. Exchange administrators also will find response returned by Autodiscover for MAPI/HTTP to Outlook is greatly simplified. The settings returned are just protocol version and endpoint URLs for Outlook to connect to Exchange mailbox and directory fr om inside or outside the customer’s corporate network. Outlook treats the URLs returned as opaque and uses as-is, minimizing the risk of connectivity breaking with future endpoint changes. Since MAPI/HTTP, like any other web protocol, simply sends an anonymous HTTP request to Exchange and gets back the authentication settings, there is no need for Autodiscover to advertise the authentication settings. This makes it easier to roll out changes in authentication settings for Outlook. The future MAPI/HTTP puts the Exchange team in position to innovate more quickly. It simplifies the architecture removing dependency on the RPC technologies which are no longer evolving as quickly as the customers demand. It provides the path for extensibility of the connection capabilities. A new capability that is on the roadmap for Outlook is to enable multi-factor authentication for users in Office 365. This capability is made possible with the use of MAPI/HTTP and is targeted to be delivered later this year. For a deeper look at this upcoming feature you can review the recent Multi-Factor Authentication for Office 365 blog post. This won’t stop with Office 365 MFA, but provides the extensibility foundation for 3 rd party identity providers. How does MAPI/HTTP work? Let’s walk through the scenario of an Outlook 2013 SP1 client connecting to Exchange Server 2013 SP1 after MAPI/HTTP has been enabled. The Outlook client begins with an Autodiscover POST request. In this request Outlook includes a new attribute that advertises the client is MAPI/HTTP capable with the attribute X-MapiHTTPCapability = 1. The Exchange server sees the request is coming from a MAPI/HTTP capable client and responds with the MAPI/HTTP information including the settings on how to connect to the mailbox using MAPI/HTTP. This assumes the MAPI/HTTP has been configured and enabled on the server. The Outlook client detects the new connection path and prompts the user to restart Outlook to switch to use the new connection. While the restart is pending Outlook will continue using Outlook Anywhere. We recommend you deploy the latest Office client updates to provide the best user experience. The updates remove the prompt and clients are allowed to make the transition at the next unprompted restart of Outlook. After the restart, Outlook now uses MAPI/HTTP to communicate with Exchange. What’s required? So now we have a clear set of advantages you can offer users, let’s review the requirements to enable MAPI/HTTP. Server Requirements: Use of MAPI/HTTP requires all Exchange 2013 Client Access Servers to be updated to Exchange Server 2013 SP1 (or later). The feature is disabled by default in SP1 so you can get the servers updated without anyone noticing any changes. If you are an Office 365 Exchange Online customer you won’t have anything to worry about on the service side of deployment. Client Requirements: Outlook clients must be updated to use MAPI/HTTP. Office 2013 SP1 or Office 365 ProPlus February update (SP1 equivalent for ProPlus) are required for MAPI/HTTP. It is recommend you deploy the May Office 2013 public update or the April update for Office 365 ProPlus to eliminate the restart prompt when MAPI/HTTP is enabled for users. Outlook 2010 was updated to support MAPI/HTTP in the January 2015 Public Update, and additional fixes for it were released in the April 2015 Public Update. Prior version clients will continue to work as-is using Outlook Anywhere. Outlook Anywhere is the supported connection method for those clients. We will not be backporting MAPI/HTTP to Outlook 2007 or earlier versions. How to get ready Part one of getting ready is to get the required updates to your servers and clients as described in the prior section. Part two of getting ready is evaluating potential impacts MAPI/HTTP might have on your on-premises servers. Again if you an Office 365 customer you can ignore this bit. When you implement MAPI/HTTP in your organization, it will have an impact on your Exchange server resources. Before you go any further you need to review the impacts to your server resources. The Exchange 2013 Server Role Requirements Calculatorhas been updated to factor in use of MAPI/HTTP. You need to use the most recent version of the calculator (v6.3 or later) before you proceed. MAPI/HTTP increases the CPU load in the Exchange Client Access Servers. This is a 50% increase over Exchange 2013 RTM, however it is still lower than Exchange 2010 requirements. As you plan be mindful that deploying in a multi-role configuration will minimize the impact to your sizing. Again use the calculator to review potential impacts this may have in your environment. This higher CPU use is due to the higher request rate with several short-lived connections, with each request taking care of authentication and proxying. To provide the best MAPI/HTTP performance you need to install .NET 4.5.1 on your Exchange 2013 servers. Installing .NET 4.5.1 will avoid long wait times for users thanks to a fix to ensure the notification channel remains asynchronous to avoid queued requests. The change in communication between Exchange and Outlook has a small impact on the bytes sent over the wire. The header content in MAPI/HTTP is responsible for an increase in bytes transferred. In a typical message communications we have observed an average packet size increase of 1.2% versus Outlook Anywhere for a 50 KB average packet. In scenarios of data transfers over 10 MB the increase in bytes over the wire is 5-10%. These increases assumes an ideal network where connections are not dropped or resumed. If you consider real world conditions you may actually find MAPI/HTTP data on the wire may be lower than Outlook Anywhere. Outlook Anywhere lacks the ability to resume connections and the cost of re-syncing items can quickly outweigh the increase from the MAPI/HTTP header information. Now deploy MAPI/HTTP Now that you have prepared your servers with SP1, updated your clients, and reviewed potential sizing impacts you are ready to get on with implementing MAPI/HTTP. It is disabled by default in SP1 and you must take explicit actions to configure and enable it. These steps are well covered in the MAPI over HTTPTechNet article. A few important things to remember in your deployment. Namespace: MAPI/HTTP is a new endpoint on CAS and can utilize both an internal namespace and an external namespace. For more information on how to properly plan your namespace design, see Namespace Planning in Exchange 2013 and Load Balancing in Exchange 2013. Certificates: The certificate used in Exchange will need to include both the internal and external MAPI/HTTP virtual directories to avoid any user certificate prompts, thus consider if the names exist on your certificates. Refer to the certificate planning post for additional help planning. MAPI/HTTP Configuration: Enabling MAPI/HTTP is an organizational configuration in Exchange, you won’t have the ability configure this for a subset of servers. If you require more specific control you can control the client behavior with a registry key. NOTE: If you require more specific control you can control the client behavior with a registry key on each client machine. This is not recommended or required but included if your situation demands this level of control. This registry entry prevents Outlook from sending the MAPI/HTTP capable flag to Exchange in the Autodiscover request. When you change the registry key on a client it will not take effect until the next time the client performs an Autodiscover query against Exchange. To disallow MAPI/HTTP and force RPC/HTTP to be used. HKEY_CURRENT_USER\Software\Microsoft\Exchange] “MapiHttpDisabled”=dword:00000001 To allow MAPI/HTTP simply delete the MapiHttpDisabled DWORD, or set it to a value of 0 as below. HKEY_CURRENT_USER\Software\Microsoft\Exchange] “MapiHttpDisabled”=dword:00000000 Connectivity: An important final consideration is to verify load balancers, reverse proxies, and firewalls are configured to allow access to the MAPI/HTTP virtual directories. At this time ForeFront Unified Access Gateway (UAG) 2010 SP4 is not compatible with MAPI/HTTP. The UAG team has committed to deliver support for MAPI/HTTP in a future update. How do I know it is working? There are a few quick ways to verify your configuration is working as expected. 1. Test with the Test-OutlookConnectivity cmdlet Use this command to test MAPI/HTTP connectivity: Test-OutlookConnectivity -RunFromServerId Contoso -ProbeIdentity OutlookMapiHttpSelfTestProbe This test is detailed in the MAPI over HTTPTechNet Article. 2. Inspect MAPI/HTTP server logs Administrators can review the following MAPI/HTTP log files to validate how the configuration is operating: Location Path CAS: %ExchangeInstallPath%Logging\HttpProxy\Mapi\HTTP Mailbox: %ExchangeInstallPath%Logging\MAPI Client Access\ Mailbox: %ExchangeInstallPath%Logging\MAPI Address Book Service\ 3. Check Outlook connection status on clients You can also quickly verify that the client is connected using MAPI/HTTP. The Outlook Connection status dialog can be launch by CTRL-right clicking the Outlook icon in the notification area and selecting Connection Status. Here are the few key fields to quickly confirm the connection is using MAPI/HTTP. Field Value Protocol HTTP (v/s RPC/HTTP for Outlook Anywhere) Proxy Server Empty Server name Actual server name (v/s GUID for Outlook Anywhere connections) Summary MAPI/HTTP provides a simplified transport and resulting architecture for Outlook to connect with Exchange. It enables improved user experiences to allow them faster access to mail and improves the resilience of their Outlook connections. These investments are the foundation for future capabilities such as multi-factor authentication in Outlook. It also helps IT support and troubleshoot client connection issues using standard HTTP protocol tools. As with all things new you must properly plan your implementation. Use the deployment guidanceavailable on TechNet and the updated sizing recommendations in the calculator before you start your deployment. With proper use it will guide you to a smooth deployment of MAPI/HTTP. Special thanks to Brian Day and Abdel Bahgat for extensive contributions to this blog post. Brian Shiers | Technical Product Manager MAPI/HTTP FAQ We collected a number of questions which frequently came up during the development, internal dogfooding, and customer TAPtesting of MAPI/HTTP. We hope these answer most of the questions you may have about MAPI/HTTP. Can MAPI/HTTP be enabled/disabled on a per-server basis? No, it is an organization-wide Exchange setting. The user experience mentioned during database failovers when one server is not yet MAPI/HTTP capable made the functionality to turn MAPI/HTTP on and off per server not a viable solution. Can MAPI/HTTP be enabled/disabled on a per-mailbox basis? No, there is not currently a Set-CasMailbox parameter to enable/disable MAPI/HTTP for a single mailbox. I updated the registry key to disable MAPI/HTTP on a client but the connection didn’t change? The registry entry simply controls what Outlook sends to Exchange about its MAPI/HTTP capability during an Autodiscover request. It does not immediately change the connection method Outlook is using nor will it change it with a simple restart of Outlook. Remember, the Autodiscover response Outlook gets only has MAPI/HTTP or RPC/HTTP settings in it so it has no way to immediately switch types. You must allow Outlook to perform its next Autodiscover request and get a response from Exchange after setting this registry entry before the change will take place. If you must attempt to speed along this process, there are two options. Set the registry entry as you wish, close Outlook, delete the Outlook profile, and then restart Outlook and go through the profile wizard. This should result in an immediate switch, but any settings stored in the Outlook profile are lost. Set the registry entry as you wish, close Outlook, delete the hidden Autodiscover response XML files in %LOCALAPPDATA%\Microsoft\Outlook, and restart Outlook. Restart Outlook once more to complete the switch. What happens if a user’s mailbox database is mounted on a MAPI/HTTP capable server and then a database failover happens to a non-MAPI/HTTP capable server? E.g. When not all mailbox servers in a DAG are MAPI/HTTP capable and MAPI/HTTP has already been enabled in the organization, and then a mailbox failover takes place between the SP1 (or later) and pre-SP1 servers. Additionally this could happen if you move a mailbox from a MAPI/HTTP capable mailbox server to a server that is not MAPI/HTTP capable. In the above example Outlook would fail to connect and when Autodiscover next ran the user would get an Outlook restart notification warning because MAPI/HTTP is no longer a viable connection method due to the mailbox being mounted on a pre-SP1 server. After the client restart the client profile would be back to utilizing RPC/HTTP. Note: While a mix of MAPI/HTTP capable and non-capable mailbox Exchange servers in the same DAG are supported in an environment with MAPI/HTTP enabled, it is very strongly not recommended due to the possible user experience outlined above. It is suggested the entire organization be upgraded to SP1 or later before enabling MAPI/HTTP in the organization. What if a user accesses additional mailboxes where MAPI/HTTP is not yet available? Outlook profiles can continue access additional resources using non-MAPI/HTTP connectivity methods even if the user’s primary mailbox utilizes MAPI/HTTP. For example a user can continue to access Legacy Public Folders or Shared Mailboxes on other Exchange servers not utilizing MAPI/HTTP. During the Autodiscover process Exchange will determine and hand back to Outlook the proper connectivity method necessary for each resource being accessed. If MAPI/HTTP becomes unreachable will a client fallback to RPC/HTTP? No, a user’s profile will never attempt to use RPC/HTTP if MAPI/HTTP becomes unavailable because the original Autodiscover response only contained one connection method to use. There is no fallback from MAPI/HTTP to RPC/HTTP or vice versa. Normal high availability design considerations should ensure the MAPI/HTTP endpoint remain accessible in the event of server or service failures. Is MAPI/HTTP replacing Exchange Web Services (EWS)? No, MAPI/HTTP is not a replacement for EWS and there are no plans to move current EWS clients to MAPI/HTTP. Is RPC/HTTP being deprecated as an available connection method? Over time this may take place as non-MAPI/HTTP capable Outlook versions age out of their product support lifecycle, but there are no immediate plans to remove RPC/HTTP as a valid connection method. What authentication methods does MAPI/HTTP support? A huge architectural improvement by moving to MAPI/HTTP is that MAPI/HTTP is abstracted from authentication. In short authentication is done at the HTTP layer, so whatever HTTP can do, MAPI/HTTP can use. Does moving the MAPI/HTTP negatively affect the Lync client at all? No, the Lync client uses the same profile as configured by Outlook and will connect via whatever connectivity method is in use by Outlook. Is there any kind of SDK/API for third party application usage of MAPI/HTTP? The MAPI/HTTP protocol is publically documented (PDF download) and has the same level of documentation support as RPC/HTTP. There are no plans to update the MAPI CDO library for MAPI/HTTP and third party companies are still encouraged to utilize Exchange Web Services as the long-term protocol for interaction with Exchange as discussed in Exchange 2013 Client Access Server Rolearticle. What are the client requirements for MAPI/HTTP? Use of MAPI/HTTP requires Outlook 2013 clients to obtain Office 2013 Service Pack 1 or the February update for Office 365 ProPlus clients. Outlook 2010 was updated to support MAPI/HTTP in the January 2015 Public Update, and additional fixes for it were released in the April 2015 Public Update. How does this affect publishing Exchange 2013 via ARR, WAP, TMG, UAG, B2M, ABC, BBD … Publishing Exchange 2013 with MAPI/HTTP in use does not change very much. You will need to ensure devices in front of Exchange handling user access to CAS are allowing access to the Default Web Site’s /mapi/ virtual directory. At this time UAG SP3 is not compatible with MAPI/HTTP even with all filtering options disabled. UAG plans to add support for MAPI/HTTP in a future update. Learn More Still want more information? Review the following sessions on this topic from Microsoft Exchange Conference 2014 Outlook Connectivity: Current and FutureOutlook Connectivity: Current and FutureOutlook Connectivity: Current and Future What's New in Outlook 2013 and Beyond What's New in Authentication for Outlook 2013GAL Photos in Exchange 2010 and Outlook 2010
Over the years, displaying recipient photographs in the Global Address List (GAL) has been a frequently-requested feature, high on the wish lists of many Exchange folks. Particularly in large organizations or geographically dispersed teams, it's great to be able to put a face to a name for people you've never met or don't frequently have face time with. Employees are commonly photographed when issuing badges/IDs, and many organizations publish the photos on intranets. There have been questions about workarounds or third-party add-ins for Outlook, and you can also find some sample code on MSDN and elsewhere. A few years ago, an unnamed IT person wrote ASP code to make employee photos show up on the intranet based on the Employee ID attribute in Active Directory - which was imported from the company's LDAP directory. A fun project to satisfy the coder alter-ego of the said IT person. Luckily, you won't need to turn to your alter-ego to do this. Exchange 2010 and Outlook 2010 make this task a snap, with help from Active Directory. Active Directory includes the Picture attribute (we'll refer to it using its ldapDisplayName: thumbnailPhoto) to store thumbnail photos, and you can easily import photos— not the high-res ones from your 20 megapixel digital camera, but small, less-than-10K-ish ones, using Exchange 2010's Import-RecipientDataProperty cmdlet. Photos in Active Directory? Really? The first question most IT folks would want to ask is— What's importing all those photos going to do to the size of my Active Directory database? And how much Active Directory replication traffic will this generate? The thumbnailPhoto attribute can accomodate photos of up to 100K in size, but the Import-RecipientDataProperty cmdlet won't allow you to import a photo that's larger than 10K. (Note, the attribute limit was stated as 10K earlier. This has been updated to state the correct value. -Bharat) The original picture used in this example was 9K, and you can compress it further to a much smaller size - let's say approximately 2K-2.5K, without any noticeable degradation when displayed at the smaller sizes. If you store user certificates in Active Directory, the 10K or smaller size thumbnail pictures are comparable in size. Storing thumbnails for 10,000 users would take close to 100 Mb, and it's data that doesn't change frequently. Note: The recommended thumbnail photo size in pixels is 96x96 pixels. With that out of the way, let's go through the process of adding pictures. A minor schema change First stop, the Active Directory Schema. A minor schema modification is required to flip the thumbnailPhoto attribute to make it replicate to the Global Catalog. Note: If you're on Exchange 2010 SP1, skip this step. The attribute is modified by setup / SchemaPrep. If you haven't registered the Schema MMC snap-in on the server you want to make this change on, go ahead and do so using the following command: Regsvr32 schmmgmt.dll Fire up a MMC console (Start -> Run -> MMC) and add the Schema snap-in In the Active Directory Schema snap-in, expand the Attributes node, and then locate the thumbnailPhoto attribute. (The Schema snap-in lists attributes by its ldapDisplayName). In the Properties page, select Replicate this attribute to the Global Catalog, and click OK. Figure 1: Modifying the thumbnailPhoto attribute to replicate it to Global Catalog Loading pictures into Active Directory Now you can start uploading pictures to Active Directory using the Import-RecipientDataProperty cmdlet, as shown in this example: Import-RecipientDataProperty -Identity "Bharat Suneja" -Picture -FileData ([Byte[]]$(Get-Content -Path "C:\pictures\BharatSuneja.jpg" -Encoding Byte -ReadCount 0)) To perform a bulk operation you can use the Get-Mailbox cmdlet with your choice of filter (or use the Get-DistributionGroupMember cmdlet if you want to do this for members of a distribution group), and pipe the mailboxes to a foreach loop. You can also retrieve the user name and path to the thumbnail picture from a CSV/TXT file. Thumbnails in Outlook 2010 Now, let's fire up Outlook 2010 and take a look what that looks like. In the Address Book/GAL properties for the recipient Figure 2: Thumbnail displayed in a recipient's property pages in the GAL When you receive a message from a user who has the thumbnail populated, it shows up in the message preview. Figure 3: Thumbnail displayed in a message While composing a message, the thumbnail also shows up when you hover the mouse on the recipient's name. Figure 4: Recipient's thumbnail displayed on mouse over when composing a message There are other locations in Outlook where photos are displayed. For example, in the Account Settings section in the Backstage Help view. Update from the Outlook team Our friends from the Outlook team have requested us to point out that the new Outlook Social Connector also displays GAL Photos, as well as photos from Contacts folders and from social networks, as shown in this screenshot. Figure 5: Thumbnail photos displayed in the People Pane in the Outlook Social Connector More details and video in Announcing the Outlook Social Connector on the Outlook team blog. GAL Photos and the Offline Address Book After you've loaded photos in Active Directory, you'll need to update the Offline Address Book (OAB) for Outlook cached mode clients. This example updates the Default Offline Address Book: Update-OfflineAddressBook "Default Offline Address Book" In Exchange 2010, the recipient attributes included in an OAB are specified in the ConfiguredAttributes property of the OAB. ConfiguredAttributes is populated with a default set of attributes. You can modify it using the Set-OfflineAddressBook cmdlet to add/remove attributes as required. By default, thumbnailPhoto is included in the OAB as an Indicator attribute. This means the value of the attribute isn't copied to the OAB— instead, it simply indicates the client should get the value from Active Directory. If an Outlook client (including Outlook Anywhere clients connected to Exchange using HTTPS) can access Active Directory, the thumbnail will be downloaded and displayed. When offline, no thumbnail downloads. Another example of an Indicator attribute is the UmSpokenName. You can list all attributes included in the default OAB using the following command: (Get-OfflineAddressBook "Default Offline Address Book").ConfiguredAttributes For true offline use, you could modify the ConfiguredAttributes of an OAB to make thumbnailPhoto a Value attribute. After this is done and the OAB updated, the photos are added to the OAB (yes, all 20,000 photos you just uploaded...). Depending on the number of users and sizes of thumbnail photos uploaded, this would add significant bulk to the OAB. Test this scenario thoroughly in a lab environment— chances are you may not want to provide the GAL photo bliss to offline clients in this manner. To prevent Outlook cached mode clients from displaying thumbnail photos (remember, the photo is not in the OAB – just a pointer to go fetch it from Active Directory), you can remove the thumbnailPhoto attribute from the ConfiguredAttributes property of an OAB using the following command: $attributes = (Get-OfflineAddressBook "Default Offline Address Book").ConfiguredAttributes $attributes.Remove("thumbnailphoto,Indicator") Set-OfflineAddressBook "Default Offline Address Book" -ConfiguredAttributes $attributes Bharat Suneja Updates: 11/3/2010: Corrected size limit of thumbnailPhoto attribute to 100K. 8/25/2011: Added note to reflect Exchange 2010 SP1 setup / SchemaPrep modifies thumbnailPhoto attribute to replicate to Global Catalog. GAL Photos now has an FAQ. Check out GAL Photos: Frequently Asked Questions. To visit this post again, use the short URL aka.ms/galphotos. To go to the 'GAL Photos: Frequently Asked Questions' post, use aka.ms/galphotosfaq.Improving Security - Together
For many years, client apps have used Basic Authentication to connect to servers, services and endpoints. It is enabled by default on most servers and services and it’s super simple to set up. Simplicity isn’t at all bad in itself, but Basic Authentication makes it easier for attackers to capture user’s credentials and so we’re taking steps to improve data security in Exchange Online.Client Connectivity in an Exchange 2013 Coexistence Environment
This article is part 3 in a series that discusses namespace planning, load balancing principles, client connectivity, and certificate planning. Over the last several months, we have routinely fielded questions on how various clients connect to the infrastructure once Exchange 2013 is deployed. Our goal with this article is to articulate the various connectivity scenarios you may encounter in your designs. To that end, this article will begin with a walk through of a deployment that consists of Exchange 2007 and Exchange 2010 in a multi-site architecture and show how the connectivity changes with the introduction of Exchange 2013. Existing Environment Figure 1: Exchange 2007 & Exchange 2010 Multi-Site Architecture As you can see from the above diagram, this environment contains three Active Directory sites: Internet Facing AD Site (Site1) - This is the main AD site in the environment and has exposure to the Internet. This site also has a mix of Exchange 2007 and Exchange 2010 servers. There are three namespaces associated with this location – mail.contoso.com and autodiscover.contoso.com resolve to the CAS2010 infrastructure and legacy.contoso.com resolves to the CAS2007 infrastructure. Regional Internet Facing AD Site (Site2) - This is an AD site that has exposure to the Internet. This site has Exchange 2010 servers. The primary namespace is mail-region.contoso.com and resolves to the CAS2010 infrastructure located within this AD site. Non-Internet Facing AD Site (Site3) - This is an AD site that does not have exposure to the Internet. This site contains Exchange 2010 infrastructure. Note: The term, Internet Facing AD Site, simply means any Active Directory site containing Client Access servers whose virtual directories have the ExternalURL property populated. Similarly, the term, Non-Internet Facing AD Site, simply means any Active Directory site containing Client Access servers whose virtual directories do not have ExternalURL property populated. To understand the client connectivity before we instantiate Exchange 2013 into the environment, let’s look at the four users. Autodiscover The Autodiscover namespace, autodiscover.contoso.com, as well as, the internal SCP records resolve to the CAS2010 infrastructure located in Site1. Outlook clients and ActiveSync clients (on initial configuration) will submit Autodiscover requests to the CAS2010 infrastructure and retrieve configuration settings based on their mailbox’s location. The Autodiscover service on Exchange 2010 can process Autodiscover requests for both Exchange 2007 and Exchange 2010 mailboxes. Note: The recommended practice is to configure the 2007 and 2010 Client Access server’s AutoDiscoverServiceInternalUri value (which is the property value you use to set the SCP record) to point to autodiscover.contoso.com, assuming split-brain DNS is in place. If split-brain DNS is not configured, then set AutoDiscoverServiceInternalUri to a value that resolves to the load balanced VIP for the 2010 Client Access servers in your environment. For more information on how Autodiscover requests are performed, see the whitepaper, Understanding the Exchange 2010 Autodiscover Service. Internal Outlook Connectivity For internal Outlook clients using RPC/TCP connectivity whose mailboxes exist on Exchange 2010, they will connect to the Exchange 2010 RPC Client Access array endpoint (assuming one exists). Keep in mind the importance of configuring the RPC Client Access array endpoint correctly, as documented in Ambiguous URLs and their effect on Exchange 2010 to Exchange 2013 Migrations. For internal Outlook clients using RPC/TCP connectivity whose mailboxes exist on Exchange 2007, they will connect directly to the Exchange 2007 Mailbox server instance hosting the mailbox. Outlook Anywhere Red User will connect to mail.contoso.com as his RPC proxy endpoint. CAS2010 in Site1 will de-encapsulate the RPC data embedded within the HTTP packet and since the target mailbox database is located within the local site, will retrieve the necessary data from the local Exchange 2010 Mailbox server. Purple User will connect to mail.contoso.com as his RPC proxy endpoint. CAS2010 in Site1 will de-encapsulate the RPC data embedded within the HTTP packet, determine that the mailbox server is located on an Exchange 2007 server, and proxy the Outlook RPC data to the target Exchange 2007 Mailbox server. Blue User will connect to mail.contoso.com as his RPC proxy endpoint. CAS2010 in Site1 will de-encapsulate the RPC data embedded within the HTTP packet, determine that the mailbox server hosting the mailbox is located in another AD site (in this case Site3) and proxy the Outlook RPC data to the target Exchange 2010 Client Access server. Orange User will connect to mail-region.contoso.com as his RPC proxy endpoint. CAS2010 in Site2 will de-encapsulate the RPC data embedded within the HTTP packet and since the target mailbox database is located within the local site, will retrieve the necessary data from the local Exchange 2010 Mailbox server. Note: In addition to the mail and directory connections, Outlook Anywhere clients also utilize Exchange Web Services and an Offline Address Book, which are provided via the Autodiscover response. Outlook Web App For more information, see the article Upgrading Outlook Web App to Exchange 2010. Red User will connect to mail.contoso.com as his namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, determine that the mailbox is located within the local AD site and retrieve the necessary data from the Exchange 2010 Mailbox server. Purple User will connect to mail.contoso.com as his namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within the local AD site on an Exchange 2007 Mailbox server. CAS2010 will initiate a single sign-on silent redirect (assumes FBA is enabled on source and target) to legacy.contoso.com. CAS2007 will then process the request and retrieve the necessary data from the Exchange 2007 Mailbox server. Blue User will connect to mail.contoso.com as his namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within Site3, which does not contain any OWA ExternalURLs. CAS2010 in Site1 will proxy the request to CAS2010 in Site3. CAS2010 in Site3 will retrieve the necessary data from the Exchange 2010 Mailbox server. For the Orange User, there are three possible scenarios depending on what namespace the users enters and how the environment is configured: Orange User will connect to mail-region.contoso.com as his namespace endpoint. CAS2010 in Site2 will authenticate the user, do a service discovery, and determine that the mailbox is located within the local AD site and retrieve the necessary data from the Exchange 2010 Mailbox server. Orange User will connect to mail.contoso.com as his namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within Site2, which does contain an OWA ExternalURL. CAS2010 in Site1 is not configured to do a cross-site silent redirection, therefore, the user is prompted to use the correct URL to access his mailbox data. Orange User will connect to mail.contoso.com as his namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within Site2, which does contain an OWA ExternalURL. CAS2010 in Site1 is configured to do a cross-site silent redirection, therefore, CAS2010 will initiate a single sign-on silent redirect (assumes FBA is enabled on source and target) to mail-region.contoso.com. CAS2010 in Site2 will then facilitate the request and retrieve the necessary data from the Exchange 2010 Mailbox server. Exchange ActiveSync For more information, see the article Upgrading Exchange ActiveSync to Exchange 2010. Red User’s ActiveSync client will connect to mail.contoso.com as the namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, determine that the mailbox is located within the local AD site and retrieve the necessary data from the Exchange 2010 Mailbox server. Purple User’s ActiveSync client will connect to mail.contoso.com as the namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within the local AD site on an Exchange 2007 Mailbox server. CAS2010 proxies the request to CAS2007. Blue User’s ActiveSync client will connect to mail.contoso.com as the namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within Site3, which does not contain any EAS ExternalURLs. CAS2010 in Site1 will issue a cross-site proxy the request to CAS2010 in Site3. CAS2010 in Site3 will retrieve the necessary data from the Exchange 2010 Mailbox server. For the Orange User, there are two possible scenarios: Orange User will connect to mail-region.contoso.com as his namespace endpoint. CAS2010 in Site2 will authenticate the user, do a service discovery, and determine that the mailbox is located within the local AD site and retrieve the necessary data from the Exchange 2010 Mailbox server. Orange User will connect to mail.contoso.com as his namespace endpoint. CAS2010 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within Site2, which does contain an EAS ExternalURL. Assuming the device supports Autodiscover and is on a defined list of devices that supports the 451 redirect response, CAS2010 will issue a 451 response to the device, notifying the device it needs to use a new namespace, mail-region.contoso.com. CAS2010 in Site2 will then facilitate the request and retrieve the necessary data from the Exchange 2010 Mailbox server. If the device is not on the supported list, then CAS2010 in Site1 will proxy the request to CAS2010 in Site2 Exchange Web Services For the Red User, Autodiscover will return the mail.contoso.com namespace for the Exchange Web Service URL. CAS2010 in Site1 will service the request. For the Blue User, Autodiscover will return the mail.contoso.com namespace for the Exchange Web Service URL. CAS2010 will then proxy the request to an Exchange 2010 Client Access server located in Site3. For the Purple User, Autodiscover will provide back the legacy.contoso.com namespace for the Exchange Web Service URL. CAS2007 in Site1 will service the request. For the Orange User, Autodiscover will return the mail-region.contoso.com namespace for the Exchange Web Service URL. CAS2010 in Site2 will service the request. Client Connectivity with Exchange 2013 in Site1 Exchange 2013 has now been deployed in Site1 following the guidance documented within the Exchange Deployment Assistant. As a result, Outlook Anywhere has been enabled on all Client Access servers within the infrastructure and the mail.contoso.com and autodiscover.contoso.com namespaces have been moved to resolve to Exchange 2013 Client Access server infrastructure. Figure 2: Exchange 2013 Coexistence with Exchange 2007 & Exchange 2010 in a Multi-Site Architecture To understand the client connectivity now that Exchange 2013 exists in the environment, let’s look at the four users. Autodiscover The Autodiscover external namespace, autodiscover.contoso.com, as well as, the internal SCP records resolve to the CAS2013 infrastructure located in Site1. Outlook clients and ActiveSync clients (on initial configuration) will submit Autodiscover requests to the CAS2013 infrastructure and depending on the mailbox version, different behaviors occur: For mailboxes that exist on Exchange 2010, CAS2013 will proxy the request to an Exchange 2010 Client Access server that exists within the mailbox’s local site. This means that for Red User, this will be a local proxy to a CAS2010 in Site1. For Blue User and Orange User, this will be a cross-site proxy to a CAS2010 located in the user’s respective site. CAS2010 will then generate the Autodiscover response. For mailboxes that exist on Exchange 2007, CAS2013 will proxy the request to an Exchange 2013 Mailbox server in the local site, which will generate the Autodiscover response. This means for Purple User, the MBX2013 server in Site1 will generate the response. For mailboxes that exist on Exchange 2013, CAS2013 will proxy the request to the Exchange 2013 Mailbox server that is hosting the active copy of the user’s mailbox which will generate the Autodiscover response. Internal Outlook Connectivity For internal Outlook clients using RPC/TCP connectivity whose mailboxes exist on Exchange 2010, they will still connect to the Exchange 2010 RPC Client Access array endpoint. For internal Outlook clients using RPC/TCP connectivity whose mailboxes exist on Exchange 2007, they will still connect directly to the Exchange 2007 Mailbox server instance hosting the mailbox. When you have an Exchange 2013 mailbox you are using Outlook Anywhere, both within the corporate network and outside of the corporate network; RPC/TCP connectivity no longer exists for Exchange 2013 mailboxes. In Exchange 2007/2010, the way Outlook Anywhere was implemented is that you had one namespace you could configure. In Exchange 2013, you have both an internal host name and an external host name. Think of it as having two sets of Outlook Anywhere settings, one for when you are connected to the corporate domain, and another for when you are not. You will see this returned to the Outlook client in the Autodiscover response via what looks like a new provider, ExHTTP. However, ExHTTP isn’t an actual provider, it is a calculated set of values from the EXCH (internal Outlook Anywhere) and EXPR (External Outlook Anywhere) settings. To correctly use these settings, the Outlook client must be patched to the appropriate levels (see the Exchange 2013 System Requirements for more information). Outlook will process the ExHTTP in order – internal first and external second. Important: In the event that you are utilizing a split-brain DNS infrastructure, then you must utilize the same authentication value for both your internal and external Outlook Anywhere settings, or switch to use different names for Outlook Anywhere inside and out. Outlook gives priority to the internal settings over the external settings and since the same namespace is used for both, regardless of whether the client is internal or external, it will utilize only the internal authentication settings. The default Exchange 2013 internal Outlook Anywhere settings don’t require HTTPS. By not requiring SSL, the client should be able to connect and not get a certificate pop-up for the mail and directory connections. However, you will still have to deploy a certificate that is trusted by the client machine for Exchange Web Services and OAB downloads. External Outlook Connectivity In order to support access for Outlook Anywhere clients whose mailboxes are on legacy versions of Exchange, you will need to make some changes to your environment which are documented in the steps within the Exchange Deployment Assistant. Specifically, you will need to enable Outlook Anywhere on your legacy Client Access servers and enable NTLM in addition to basic authentication for the IIS Authentication Method. The Exchange 2013 Client Access server’s RPC proxy component sees the incoming connections, authenticates and chooses which server to route the request to (regardless of version), proxying the HTTP session to the endpoint (legacy CAS or Exchange 2013 Mailbox server). Red User will connect to mail.contoso.com as his RPC proxy endpoint. CAS2013 in Site1 will determine the mailbox version is 2010 and that the mailbox database hosting the user’s mailbox is located in the local site. CAS2013 will proxy the request to CAS2010 in Site1. CAS2010 will de-encapsulate the RPC from the HTTP packet and obtain the data from the Exchange 2010 Mailbox server. Purple User will connect to mail.contoso.com as his RPC proxy endpoint. CAS2013 in Site1 will determine the mailbox version is 2007 and that the mailbox database hosting the user’s mailbox is located in the local site. CAS2013 will proxy the request to CAS2007 in Site1. CAS2007 will de-encapsulate the RPC from the HTTP packet and obtain the data from the Exchange 2007 Mailbox server. Blue User will connect to mail.contoso.com as his RPC proxy endpoint. CAS2013 in Site1 will determine the mailbox version is 2010 and that the mailbox database hosting the user’s mailbox is located in Site3. CAS2013 will proxy the request to CAS2010 in Site3. CAS2010 will de-encapsulate the RPC from the HTTP packet and obtain the data from the Exchange 2010 Mailbox server. Orange User will continue to access his mailbox using the Exchange 2010 regional namespace, mail-region.contoso.com. Outlook Web App For Outlook Web App, the user experience will depend on the mailbox version and where the mailbox is located. Red User will connect to mail.contoso.com as his namespace endpoint. CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox version is 2010 and is located within the local AD site. CAS2013 will proxy the request to an Exchange 2010 Client Access server which will retrieve the necessary data from the Exchange 2010 Mailbox server. Purple User will connect to mail.contoso.com as his namespace endpoint. CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within the local AD site on an Exchange 2007 Mailbox server. CAS2013 will initiate a single sign-on silent redirect (assumes FBA is enabled on source and target) to legacy.contoso.com. CAS2007 will then facilitate the request and retrieve the necessary data from the Exchange 2007 Mailbox server. Blue User will connect to mail.contoso.com as his namespace endpoint. CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within Site3, which does not contain any OWA ExternalURLs. CAS2013 in Site1 will proxy the request to CAS2010 in Site3. CAS2010 in Site3 will retrieve the necessary data from the Exchange 2010 Mailbox server. For the Orange User, there are two possible scenarios: Orange User will connect to mail-region.contoso.com as his namespace endpoint. In this case, CAS2013 is not involved in any fashion. Orange User will connect to mail.contoso.com as his namespace endpoint. CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within Site2, which does contain an OWA ExternalURL. CAS2013 in Site1 will initiate a single sign-on silent redirect (assumes FBA is enabled on source and target) to mail-region.contoso.com. CAS2010 in Site2 will then facilitate the request and retrieve the necessary data from the Exchange 2010 Mailbox server. Note: Let’s assume that Site3 contains Exchange 2007 servers as well. In this scenario, CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox version is 2007 and the mailbox is located in Site3. CAS2013 will issue a single sign-on silent redirect to legacy.contoso.com. CAS2007 in Site1 will authenticate the user and proxy the request to the Exchange 2007 Client Access server infrastructure in Site3. Exchange ActiveSync For Exchange ActiveSync clients, the user experience will depend on the mailbox version and where the mailbox is located. In addition, Exchange 2013 no longer supports the 451 redirect response – Exchange 2013 will always proxy ActiveSync requests. Red User’s ActiveSync client will connect to mail.contoso.com as the namespace endpoint. CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox version is 2010 and the mailbox is located within the local AD site. CAS2013 will proxy the request to an Exchange 2010 Client Access server which will retrieve the necessary data from the Exchange 2010 Mailbox server. Purple User’s ActiveSync client will connect to mail.contoso.com as the namespace endpoint. CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox version is 2007. CAS2013 will proxy the request to a local Exchange 2013 Mailbox server. The Exchange 2013 Mailbox server will send the request to an Exchange 2007 Client Access server that exists within the mailbox’s site (specifically, the InternalURL value of the Microsoft-Server-ActiveSync virtual directory on CAS2007), which will retrieve the data from the Exchange 2007 Mailbox server. Blue User’s ActiveSync client will connect to mail.contoso.com as the namespace endpoint. CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox is located within Site3. CAS2013 in Site1 will issue a cross-site proxy the request to CAS2010 in Site3. CAS2010 in Site3 will retrieve the necessary data from the Exchange 2010 Mailbox server. For the Orange User, there are two possible scenarios depending on how the device is configured: Orange User’s ActiveSync client is configured to connect to mail-region.contoso.com as the namespace endpoint. In this case, CAS2013 is not involved in any fashion. Note that any new device that is configured will automatically be configured to use mail-region.contoso.com. Orange User’s ActiveSync client is configured to connect to mail.contoso.com as the namespace endpoint. CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox version is 2010 and the mailbox is located within another AD site. CAS2013 will issue a cross-site proxy request to an Exchange 2010 Client Access server that resides in the same site as the mailbox. Note: Let’s assume that Site3 contains Exchange 2007 servers as well. In this scenario, CAS2013 in Site1 will authenticate the user, do a service discovery, and determine that the mailbox version is 2007 and the mailbox is located in Site3. CAS2013 will proxy the request to a local Exchange 2013 Mailbox server. The Exchange 2013 Mailbox server will send the request to an Exchange 2007 Client Access server that exists within the mailbox’s site, which will retrieve the data from the Exchange 2007 Mailbox server. Exchange Web Services Coexistence with Exchange Web Services is rather simple. For the Red User, Autodiscover will return the mail.contoso.com namespace for the Exchange Web Service URL. CAS2013 will then proxy the request to an Exchange 2010 Client Access server within the local site. For the Purple User, Autodiscover will return the legacy.contoso.com namespace for the Exchange Web Service URL. For the Blue User, Autodiscover will return the mail.contoso.com namespace for the Exchange Web Service URL. CAS2013 will then proxy the request to an Exchange 2010 Client Access server located in Site3. For the Orange User, Autodiscover will return the mail-region.contoso.com namespace for the Exchange Web Service URL. Note: Let’s assume that Site3 contains Exchange 2007 servers as well. In this scenario, the Autodiscover response will provide back the legacy.contoso.com namespace for the Exchange Service URL. CAS2007 in Site1 will proxy the request to an Exchange 2007 Client Access server that exists within the mailbox’s site, which will retrieve the data from the Exchange 2007 Mailbox server. This is why you cannot remove Exchange 2007 from the Internet-facing Active Directory site as long as Exchange 2007 exists in non-Internet facing Active Directory sites. Offline Address Book Like with Exchange Web Services, Autodiscover will provide the Offline Address Book URL. For the Red User, Autodiscover will return the mail.contoso.com namespace for the Offline Address Book URL. CAS2013 will then proxy the request to any Client Access server within the local site that is a web distribution point for the Offline Address Book in question. For the Purple User, Autodiscover will return the legacy.contoso.com namespace for the Offline Address Book URL. For the Blue User, Autodiscover will return the mail.contoso.com namespace for the Offline Address Book URL. CAS2013 will then proxy the request to any Client Access server within the local site that is a web distribution point for the Offline Address Book in question. For the Orange User, Autodiscover will return the mail-region.contoso.com namespace for the Offline Address Book URL. Note: Let’s assume that Site3 contains Exchange 2007 servers as well. In this scenario, the Autodiscover response will provide back the legacy.contoso.com namespace for the Offline Address Book URL. This is why you cannot remove Exchange 2007 from the Internet-facing Active Directory site as long as Exchange 2007 exists in non-Internet facing Active Directory sites. How CAS2013 Picks a Target Legacy Exchange Server It’s important to understand that when CAS2013 proxies to a legacy Exchange Client Access server, it constructs a URL based on the server FQDN, not a load balanced namespace or the InternalURL value.But how does CAS2013 choose which legacy Client Access server to proxy the connection? When a CAS2013 starts up, it connects to Active Directory and enumerates a topology map to understand all the Client Access servers that exist within the environment. Every 50 seconds, CAS2013 will send a lightweight request to each protocol end point to all the Client Access servers in the topology map; these requests have a user agent string of HttpProxy.ClientAccessServer2010Ping (yes, even Exchange 2007 servers are targeted with that user string). CAS2013 expects a response - a 200/300/400 response series indicates the target server is up for the protocol in question; a 502, 503, or 504 response indicates a failure. If a failure response occurs, CAS2013 immediately retries to determine if the error was a transient error. If this second attempt fails, CAS2013 marks the target CAS as down and excludes it from being a proxy target. At the next interval (50 seconds), CAS2013 will attempt to determine the health state of the down CAS to determine if it is available. The IIS log on a legacy Client Access server will contain the ping events. For example: 2014-03-11 14:00:00 W3SVC1 DF-C14-02 157.54.7.76 HEAD /ecp - 443 - 192.168.1.42 HTTP/1.1 HttpProxy.ClientAccessServer2010Ping - - coe-e14-1.coe.lab 302 0 0 277 170 0 2014-03-11 14:00:00 W3SVC1 DF-C14-02 157.54.7.76 HEAD /PowerShell - 443 - 192.168.1.27 HTTP/1.1 HttpProxy.ClientAccessServer2010Ping - - coe-e14-1.coe.lab 401 0 0 309 177 15 2014-03-11 14:00:00 W3SVC1 DF-C14-02 157.54.7.76 HEAD /EWS - 443 - 192.168.1.134 HTTP/1.1 HttpProxy.ClientAccessServer2010Ping - - coe-e14-1.coe.lab 401 0 0 245 170 0 2014-03-11 14:00:00 W3SVC1 DF-C14-02 157.54.7.76 GET /owa - 443 - 192.168.1.220 HTTP/1.1 HttpProxy.ClientAccessServer2010Ping - - coe-e14-1.coe.lab 301 0 0 213 169 171 2014-03-11 14:00:01 W3SVC1 DF-C14-02 157.54.7.76 HEAD /Microsoft-Server-ActiveSync/default.eas - 443 - 192.168.1.29 HTTP/1.1 HttpProxy.ClientAccessServer2010Ping - - coe-e14-1.coe.lab 401 2 5 293 194 31 2014-03-11 14:00:04 W3SVC1 DF-C14-02 157.54.7.76 HEAD /OAB - 443 - 10.166.18.213 HTTP/1.1 HttpProxy.ClientAccessServer2010Ping - - coe-e14-1.coe.lab 401 2 5 261 170 171 If for some reason, you would like to ensure a particular CAS2010 is never considered a proxy endpoint (or want to remove it for maintenance activities), you can do so by executing the following cmdlet on Exchange 2010 (note that this feature does not exist on Exchange 2007): Set-ClientAccessServer <server> -IsOutofService $True IMAP & POP Coexistence All this discussion about HTTP-based clients is great, but what about POP and IMAP clients? Like the HTTP-based client counterparts, IMAP and POP clients are also proxied from the Exchange 2013 Client Access server to a target server (whether that be an Exchange 2013 Mailbox server or a legacy Client Access server). However, there is one key difference, there is no health-checking on the target IMAP/POP services. When the Exchange 2013 Client Access server receives a POP or IMAP request, it will authenticate the user and perform a service discovery. If the target mailbox is E2010, CAS2013 will enumerate the POP or IMAP InternalConnectionSettings property value for each Exchange 2010 Client Access server within the mailbox’s site. Therefore, it is important to ensure that the InternalConnectionSettings maps to the server's FQDN, and not a load-balanced namespace. If the target mailbox is E2007, CAS2013 will enumerate each Exchange 2007 Client Access server’s server FQDN within the mailbox’s site. CAS2013 will choose a server to proxy the request based on the incoming connection’s configuration. If the incoming connection is over an encrypted channel, CAS2013 will try to locate an SSL proxy target first, TLS next, plaintext lastly. If the incoming connection is over a plaintext channel, CAS2013 will try to locate a plaintext proxy target first, SSL next, TLS lastly. Important: Exchange 2013 Client Access servers do not validate that the POP or IMAP services are actually running on the target Client Access servers. It's important, therefore, to ensure that the services are running on every legacy Client Access server if you have POP or IMAP clients in your environment. Conclusion Hopefully this information dispels some of the myths around proxying and redirection logic for Exchange 2013 when coexisting with either Exchange 2007 or Exchange 2010. Please let us know if you have any questions. Ross Smith IV Principal Program Manager Office 365 Customer Experience Updates 3/14/2014: Added section on IMAP/POP coexistence 3/25/2014: Added detail that MBX2013 proxies to the CAS2007 MSAS InternalURL for 2007 EAS mailbox accessManaging OAB in Exchange Server 2013
The Exchange team blog article OAB in Exchange Server 2013 introduced the new Offline Address Book (OAB) generation and distribution architecture in Exchange Server 2013. Take a few moments to visit the article if you haven’t seen it yet or re-visit it for a quick refresher. The OAB management and administration is different in Exchange 2013 because of architecture changes. Additionally, the new Exchange Admin Center does not currently have options for managing OABs. This means that, at this time, you will need to use Exchange Management Shell for OAB-related tasks. This article takes you through commonly performed tasks in OAB administration and has a couple of real life scenarios to help understand the tasks better. Note: If you are in multi-forest Active Directory domain environment, make sure the Shell session has ViewEntireForest is enabled, else some of the commands in the article won’t return any output. Command to enable ViewEntireForest: Set-ADServerSettings -ViewEntireForest $true Creating a new OAB Creating a new OAB in Exchange 2013 no longer uses the -Server parameter. In order to create a new OAB, you should only specify the address lists to be required. The following example creates OAB for address list named “Global Address List FAB” New-OfflineAddressBook -Name OAB-FAB -AddressLists "Global Address List FAB" Identify the OAB generation server(s) The arbitration mailboxes in Exchange Server 2013 are assigned certain “Persisted capability” that defines the purpose/function of the arbitration mailbox. An arbitration mailbox with Persisted Capability “OrganizationCapabilityOABGen” is responsible for OAB generation. We will refer this mailbox as “Organization Mailbox” throughout the article. Exchange Server 2013 mailbox server hosting the Organization Mailbox will generate all OAB’s defined in the environment. For a non-DAG environment, use following command to identify the OAB Generation servers: Get-Mailbox -Arbitration | where {$_.PersistedCapabilities -like "*oab*"} | ft name,servername For a DAG environment, identifying OAB generation server(s) is a two-step process. Step1: Identify the mailbox database hosting organization mailbox with OAB Gen capability. Use the following command to list the arbitration mailboxes with persisted capability of OABGen and database on which this mailbox is hosted: Get-Mailbox -Arbitration | where {$_.PersistedCapabilities -like "*oab*"} | ft name,database Step2: Identify the mailbox server where the database hosting organization mailbox is mounted Use following command to identify active copy of mailbox database: Get-MailboxDatabaseCopyStatus db1 The server where database status is “mounted” is the current OAB generation server. Change the OAB generation server There are two methods of changing the OAB generation server. Move mailbox Move the organization mailbox to a mailbox database on a server intended to be designated as OAB Generation server. Example: DB1 is a single copy database present on the server Exch1 and hosts the organization mailbox. DB2 is mailbox database present on Exch2. The following command can be used to move the organization mailbox to DB2 and make Exch2 the OAB generation server. Get-Mailbox -Arbitration -database db1| where {$_.PersistedCapabilities –like “*oab*”} | New-MoveRequest -TargetDatabase db2 This method is more suited for environments that have single copy of mailbox database hosting the Organization Mailbox. Activate the mailbox database on another server This method is suited for environments that have multiple copies of the mailbox database hosting Organization Mailbox. Example: DB1 hosts the Organization Mailbox and has copies on servers Exch1 and Exch2. DB1 is currently active on Exch1. The following command can be used to activate DB1 on Exch2 and therefore make it the OAB generation server: Move-ActiveMailboxDatabase DB1 -ActivateOnServer Exch2 Note: Review guidelines mentioned in “Placement of Organization Mailbox” below before changing the OAB Generation server. Creating a new Organization Mailbox Administrators can create additional Organization Mailboxes for fault tolerance or for serving users in a geographically disbursed Exchange deployment. Creating a new Organization Mailbox is a two step process: Step1: Create a new arbitration mailbox New-Mailbox -Arbitration -Name "OAB Seattle" -Database DB2Seattle -UserPrincipalName oabs@contoso.com –DisplayName “OAB Mailbox for Seattle” Step2: Enable OABGen capability Set-Mailbox -Arbitration oabs -OABGen $true -MaxSendSize 1GB Note: Review guidelines mentioned in “Placement of Organization Mailbox” below before creating additional organization mailboxes. Changing the OAB Generation Schedule The OAB Generation till Exchange Server 2010 was based on a “Schedule” set on OAB properties. You might see a “Schedule” defined when viewing properties of Exchange 2013 OAB. But, the Exchange Server 2013 OAB generation does not take place according to the “Schedule” defined on OAB properties: Instead, Exchange Server 2013 OAB Generation takes place according to OABGeneratorWorkCycle and OABGeneratorWorkCycleCheckpoint properties configured at the Mailbox Server. Example: The values in the above screenshot mean OAB is generated once every day. Which Mailbox Server processed the OAB download request? The Exchange Server 2013 CAS role proxies the OAB download request to an appropriate Mailbox role server. The CAS role maintains log of each request it handles in the log files, present in folder %ExchangeInstallPath%\Logging\HttpProxy\OAB\ These log files are an excellent tool to identify which mailbox server the CAS chose to serve the request. Information of some important fields from the log file: Field Description UrlStem Useful to identify the OAB being downloaded and also if this was a full download or incremental download AuthenticatedUser Name of the User requesting the OAB AnchorMailbox DN of Organization Mailbox identified as the closest to serve the OAB request ServerHostName CAS Server Name handling the request HttpStatus Status code for Proxy action ProxyAction Action CAS Server performed for the request, it will be mostly “Proxy” for Exchange 2013 OAB TargetServer Name of Mailbox role server to which request was proxied The log file can be imported in Excel for better readability. Example: Forcing the OAB Generation The Exchange Server 2013 OAB generation can be forced to start immediately by two methods. Method 1: Update-OfflineAddresBook Below command will force OAB generation of an OAB named "Default Offline Address Book" across all organization mailboxes. Update-OfflineAddressBook "default offline address book" Note: This command initiates an RPC request to each mailbox server hosting an active organization mailbox. Method 2: Restart the Mailbox Assistant service. The Microsoft Exchange Mailbox Assistant service on Mailbox Role is responsible for generating OAB. Restarting this service generates all OAB’s defined in the environment on a specific mailbox server, if it’s hosting an active organization mailbox. Placement of Organization Mailbox Exchange Server 2013 CAS role proxies the OAB download request to a “nearest” mailbox server hosting an active Organization Mailbox. It can proxy the request in round robin fashion if it finds more than one organization mailbox active in same AD site. Prior to CU5, this will result in frequent full OAB downloads and is therefore, not recommended. Hence, current guidance is to plan organization mailbox placement such that you will have one organization mailbox active in an AD site. This applies to creating a new organization mailbox as well as to creating copies of mailbox database that hosts an organization mailbox. Prior to CU5, customers should only deploy a single OAB generation mailbox per Exchange organization to prevent users from accessing different OAB generation mailboxes and requiring a full OAB download. With CU5 and later, customers can assign OABs to specific OAB generation mailboxes and not have to worry about accidentally triggering full OAB downloads due to accessing different OAB generation mailboxes. For more information, please see the article, OAB Improvements in Exchange 2013 Cumulative Update 5. Scenarios The following scenarios discuss a real life situation to further explain the new OAB management methods. Scenario 1: Create a new Organization Mailbox Contoso has Exchange Server 2013 Mailbox & CAS role servers deployed at Dallas and Seattle sites. John, the Exchange Admin for Contoso, analyzes the http proxy log files on CAS servers and finds the OAB download request for Seattle users is going to Dallas servers. On further investigation, John finds he has just one Organization Mailbox present in Dallas, hence OAB download requests of all the users are going to Dallas server. He decides to create a new Organization Mailbox at Seattle site with following commands: Step1: Create a new Arbitration Mailbox New-Mailbox -Arbitration -Name "OAB Seattle" -Database DB2Seattle -UserPrincipalName oabs@contoso.com –DisplayName “OAB Mailbox for Seattle” Step2: Enable the Arbitration Mailbox with OABGen capability Set-Mailbox -Arbitration oabs -OABGen $true -MaxSendSize 1GB Scenario 2: Customize OAB Generation Schedule Ben is an administrator of Exchange 2013 deployment at Tail Spin Toys. The default OAB generation schedule does not suit them and they want to generate OAB approximately every fourth hour of the day. Ben will use following command to change properties of the mailbox servers that will be hosting the Organization Mailbox. Set-MailboxServer Exch1 -OABGeneratorWorkCycle 01.00:00:00 -OABGeneratorWorkCycleCheckpoint 04:00:00 After a couple of days, John analyzes Event ID 17002 in application log and makes sure the OAB is generated every four hours. Hopefully, you find this post useful! Let us know your feedback below! Bhalchandra Atre Updates 5/15/14: Updated Placement of Organization Mailbox section to align with current guidance.Announcing Hybrid Modern Authentication for Exchange On-Premises
We’re very happy to announce support for Hybrid Modern Authentication (HMA) with the next set of cumulative updates (CU) for Exchange 2013 and Exchange 2016, that’s CU8 for Exchange Server 2016, and CU19 for Exchange Server 2013. What is HMA? HMA (not HAM, which Word keeps trying to correct it to for me) provides users the ability to access on-premises application using authorization tokens obtained from the cloud. For Exchange (that’s why you’re here right?), this means on-premises mailbox users get the ability to use these tokens (OAuth tokens specifically) for authentication to on-premises Exchange. Sounds thrilling I know, but what exactly are these tokens? And how do users get hold of them? Rather than repeat many things here, I’m going to suggest you take a break and read the How Hybrid Authentication Really Works post, and if you really want to help boost my YouTube viewing numbers, watch this Ignite session recording too. They will respectively give you a pretty solid grounding in OAuth concepts and to help you understand what HMA is really all about. See how much space we saved in this post by sending you somewhere else? If you ignored my advice, the tl’dr version is this: HMA enables Outlook to obtain Access and Refresh OAuth tokens from Azure AD (either directly for password hash sync or Pass-Through Auth identities, or from their own STS for federated identities) and Exchange on-premises will accept them and provide mailbox access. How users get those tokens, what they have to provide for credentials, is entirely up to you and the capabilities of the identity provider (iDP) – it could be simple username and password, or certificates, or phone auth, or fingerprints, blood, eyeball scanning, the ability to recite poetry, whatever your iDP can do. Note that the user’s identity has to be present in AAD for this to work, and there is some configuration required that the Exchange Hybrid Configuration Wizard does for us. That’s why we put the H in HMA, you need to be configured Hybrid with Exchange Online for this feature. It’s also worth knowing that HMA shares many of the same technology as the upcoming Outlook mobile support for Exchange on-premises with Microsoft Enterprise Mobility + Security feature, which as you’ll see from the blog post also requires Hybrid be in place. Once you have that figured out you’ll be able to benefit from both these features with very little additional work. How Does HMA Work? The video linked above goes into detail, but I’ll share some details here for anyone without the time to watch it. Here’s a diagram that explains HMA when the identity is federated. I think that picture is pretty clear, I spent a lot of time making it pretty clear so I don’t think I need to add much to it other than to say, if it’s not clear, you might want to try reading it again. Why Should I Enable HMA? Great question. There are a few good reasons, but mainly this is a security thing. HMA should be considered ‘more secure’ than the authentication methods previously available in Exchange. That’s a nebulous statement if there ever was one (I could have said it’s more ‘Modern’ but I know you weren’t going to fall for that) but there are a few good arguments as to why that’s true. When you enable HMA you are essentially outsourcing user authentication to your iDP, Exchange becomes the consumer of the resulting authorization tokens. You can enforce whatever authentication the iDP can do, rather than teach Exchange how to handle things like text messaged based MFA, blood analysis or retina scanning. If your iDP can do that, Exchange can consume the result. Exchange doesn’t care how you authenticated, only that you did, and came away with a token it can consume. So it’s clearly ‘more secure’ if you choose to enforce authentication types or requirements stronger than those that come free with Exchange, but even if you stick to usernames and passwords it’s also more secure as passwords are no longer being sent from client to server once the user is authenticated (though of course that depends on whether you are using Basic, NTLM or Kerberos). It’s all token based, the tokens have specific lifetimes, and are for specific applications and endpoints. One other interesting and important benefit to all this is that your auth flow is now exactly the same for both your cloud and on-premises users. Any MFA or Conditional Access policies you have configured are applied the same, regardless of the mailbox location. It’s simpler to stay secure. HMA also results in an improved user experience as there will be less authentication prompts. Once the user logs in once to AAD they can access any app that uses AAD tokens – that’s anything in O365 and even Skype for Business on-premises configured for HMA (read more about Skype for Business’s HMA support here). And don’t forget there’s the fact it’s more ‘Modern’. It’s newer and we put the word Modern on it. So it must be better, or at the very least, newer. Excellent, moving on. Will It Cost Me? Not if you just want to use free Azure ID’s or Federated identities and do MFA at your iDP. If you want to take advantage of advanced Azure features, then yes, you’ll have to pay for those. But to set this up the tenant admin needs only an Exchange and an Azure license assigned, to run the tools and enable the config. What do I need to enable HMA? There are some pre-requisites. The following Identity configurations with AAD are supported Federated Identity with AAD with any on-premises STS supported by Office 365 Password Hash Synchronization Pass Through Authentication In all cases, the entire on-premises directory must be synchronized to AAD, and all domains used for logon must be included in the sync configuration. Exchange Server All servers must be Exchange 2013 (CU19+) and/or Exchange 2016 (CU8+) No Exchange 2010 in the environment MAPI over HTTP enabled. It is usually enabled or True for new installs of Exchange 2013 Service Pack 1 and above. OAuth must be enabled on all Virtual Directories used by Outlook (/AutoDiscover, /EWS, /Mapi, /OAB) In the event your environment utilizes a proxy server infrastructure to allow servers to connect to the Internet, be sure all Exchange servers have the proxy server defined in the InternetWebProxy property. You must use clients that support ADAL (the client-side library that allows the client to work with OAuth tokens) to use the Modern Auth enabled features. Outlook 2013 requires the EnableADAL registry key be set, Outlook 2016 has this key set by default, Outlook 2016 for Mac works as it is, support for Outlook mobile (iOS and Android) is coming. Ensure AAD Connect between on-premises AD and the O365 tenant has the “Exchange hybrid deployment” setting enabled in the Optional Features settings of Azure AD Connect. Ensure SSL offloading is not being used between the load balancer and Exchange servers. Ensure all user networks can reach AAD efficiently. Let’s pick a few of those apart. No Exchange 2010 in the environment. That’s right, if you have E2010 you can’t enable HMA. Why? Because worst case is everyone with a mailbox on E2010 will be cut off from email. You don’t want that. It’s because OAuth happens anonymously upon initial connection. We send the user to AAD to get authenticated before we know where their mailbox is – and if that mailbox is on E2010, when they return with a token we’ll refuse to proxy from E2013/16 to E2010. Game over. Please insert coins. So we have drawn a line here and are stating no support for E2010, and the HCW won’t let you enable OAuth if E2010 exists. Don’t try and make it work, remember that scene from Ghostbusters, the whole crossing the streams thing? It’ll be like that, but worse. Next, MAPI/HTTP – you need to be using MAPI/HTTP not RPC/HTTP (Outlook Anywhere). This feature only works with MAPI/HTTP, and anyway, it’s time to get off RPC/HTTP. That’s very old code and as you might know we ended support for its use in O365, so it would be good to switch. It just works. Then there’s the ‘everyone should be in AAD’ thing. That’s because when you enable HMA, it’s Org wide. It affects every user connecting to Exchange. So, all users trying to access Exchange from a client that support Modern Auth will be sent to AAD. If you only have some users represented in AAD, only those users will be able to auth. The rest will come find you at lunch and make your life a misery. Unless you like misery, I wouldn’t recommend that route. Needing clients that support Modern Auth clearly, makes sense. And you need to make sure all the Exchange VDirs have OAuth enabled on them. Sounds obvious, and they are enabled by default, but some admins like to tinker… so it’s worth checking, and I’ll explain how later. SSL offloading works by terminating the SSL/TLS encryption on the load balancer and transmitting the request as HTTP. In the context of OAuth, using SSL offloading has implications because if the audience claim value specifies a HTTPS record, then when Exchange receives the decrypted request over HTTP, the request is considered not valid. By removing SSL offloading, Exchange will not fail the OAuth session due to a change in the audience claim value. Lastly, the ensuring all user networks can reach AAD comment. This change affects all connectivity from supported clients to Exchange, internal and external. When a user tries to connect to Exchange, whether that server is 10 feet away under the new guys desk or in a datacenter on the other side of the planet the HMA flow will kick in. If the user doesn’t have a valid token the traffic will include a trip to AAD. If you are one of those customers with complex networking in place, consider that. How do I Enable HMA? You’ve checked the pre-reqs, and you think you’re good to go. You can do a lot of this up front without impacting clients, I’ll point out where clients begin to see changes, so you can be prepared. We do recommend trying HMA in your test or lab environment if you can before doing it in production. You are changing auth, it’s something you need to be careful doing, as cutting everyone off from email is never a good thing. Here’s what to do. First, we have some Azure Active Directory Configuration to do. You need to register all the URL’s a client might use to connect to on-premises Exchange in AAD, so that AAD can issue tokens for those endpoints. This includes all internal and external namespaces, as AAD will become the default auth method for all connections, internal and external. Here’s a tip – look at the SSL certificates you have on Exchange and make sure all those names are considered for inclusion. Run the following cmdlets to gather the URL’s you need to add/verify are in AAD. Get-MapiVirtualDirectory | FL server,*url* Get-WebServicesVirtualDirectory | FL server,*url* Get-OABVirtualDirectory | FL server,*url*> Now you need to ensure all URL’s clients may connect to are listed as https service principal names (SPN’s): Connect to your AAD tenant using these instructions. For Exchange-related URL’s, execute the following command (note the AppId ends …02): Get-MsolServicePrincipal -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 | select -ExpandProperty ServicePrincipalNames The output will look similar to the following: [PS] C:\WINDOWS\system32> Get-MsolServicePrincipal -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 | select -ExpandProperty ServicePrincipalNames https://autodiscover.contoso.com/ https://mail.contoso.com/ 00000002-0000-0ff1-ce00-000000000000/*.outlook.com 00000002-0000-0ff1-ce00-000000000000/outlook.com 00000002-0000-0ff1-ce00-000000000000/mail.office365.com 00000002-0000-0ff1-ce00-000000000000/outlook.office365.com 00000002-0000-0ff1-ce00-000000000000/contoso.com 00000002-0000-0ff1-ce00-000000000000/autodiscover.contoso.com 00000002-0000-0ff1-ce00-000000000000/contoso.mail.onmicrosoft.com 00000002-0000-0ff1-ce00-000000000000/autodiscover.contoso.mail.onmicrosoft.com 00000002-0000-0ff1-ce00-000000000000/mail.contoso.com 00000002-0000-0ff1-ce00-000000000000 If you do not already have your internal and external MAPI/HTTP, EWS, OAB and AutoDiscover https records listed (i.e., https://mail.contoso.com and https://mail.corp.contoso.com), add them using the following command (replacing the fully qualified domain names with the correct namespaces and/or deleting the appropriate addition line if one of the records already exists): $x= Get-MsolServicePrincipal -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 $x.ServicePrincipalnames.Add("https://mail.corp.contoso.com/") $x.ServicePrincipalnames.Add("https://owa.contoso.com/") Set-MSOLServicePrincipal -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 -ServicePrincipalNames $x.ServicePrincipalNames Repeat step 2 and verify the records were added. We’re looking for https://namespace entries for all the URL’s, not 00000002-0000-0ff1-ce00-000000000000/namespace entries. For example, [PS] C:\WINDOWS\system32> Get-MsolServicePrincipal -AppPrincipalId 00000002-0000-0ff1-ce00-000000000000 | select -ExpandProperty ServicePrincipalNames https://autodiscover.contoso.com/ https://mail.contoso.com/ https://mail.corp.contoso.com/ https://owa.contoso.com/ 00000002-0000-0ff1-ce00-000000000000/*.outlook.com 00000002-0000-0ff1-ce00-000000000000/outlook.com 00000002-0000-0ff1-ce00-000000000000/mail.office365.com 00000002-0000-0ff1-ce00-000000000000/outlook.office365.com 00000002-0000-0ff1-ce00-000000000000/contoso.com 00000002-0000-0ff1-ce00-000000000000/autodiscover.contoso.com 00000002-0000-0ff1-ce00-000000000000/contoso.mail.onmicrosoft.com 00000002-0000-0ff1-ce00-000000000000/autodiscover.contoso.mail.onmicrosoft.com 00000002-0000-0ff1-ce00-000000000000/mail.contoso.com 00000002-0000-0ff1-ce00-000000000000 Then we need to validate the EvoSts authentication provider is present using the Exchange Management Shell (this is created by the Hybrid Configuration Wizard): Get-AuthServer | where {$_.Name -eq "EvoSts"} If it is not present, please download and execute the latest version of the Hybrid Configuration Wizard. Note that this authentication provider is not created if Exchange 2010 (this includes Edge Transport servers) is detected in the environment. Now let’s make sure OAuth is properly enabled in Exchange on all the right virtual directories Outlook might use. Run the following cmdlets (and a tip, don’t use -ADPropertiesOnly as that sometimes tells little white lies, try it and see if you don’t believe me) Get-MapiVirtualDirectory | FL server,*url*,*auth* Get-WebServicesVirtualDirectory | FL server,*url*,*oauth* Get-OABVirtualDirectory | FL server,*url*,*oauth* Get-AutoDiscoverVirtualDirectory | FL server,*oauth* You are looking to make sure OAuth is enabled on each of these VDirs, it will look something like this (and the key things to look at are highlighted); [PS] C:\Windows\system32>Get-MapiVirtualDirectory | fl server,*url*,*auth* Server : EX1 InternalUrl : https://mail.contoso.com/mapi ExternalUrl : https://mail.contoso.com/mapi IISAuthenticationMethods : {Ntlm, OAuth, Negotiate} InternalAuthenticationMethods : {Ntlm, OAuth, Negotiate} ExternalAuthenticationMethods : {Ntlm, OAuth, Negotiate} [PS] C:\Windows\system32> Get-WebServicesVirtualDirectory | fl server,*url*,*auth* Server : EX1 InternalNLBBypassUrl : InternalUrl : https://mail.contoso.com/EWS/Exchange.asmx ExternalUrl : https://mail.contoso.com/EWS/Exchange.asmx CertificateAuthentication : InternalAuthenticationMethods : {Ntlm, WindowsIntegrated, WSSecurity, OAuth} ExternalAuthenticationMethods : {Ntlm, WindowsIntegrated, WSSecurity, OAuth} LiveIdNegotiateAuthentication : WSSecurityAuthentication : True LiveIdBasicAuthentication : False BasicAuthentication : False DigestAuthentication : False WindowsAuthentication : True OAuthAuthentication : True AdfsAuthentication : False [PS] C:\Windows\system32> Get-OabVirtualDirectory | fl server,*url*,*auth* Server : EX1 InternalUrl : https://mail.contoso.com/OAB ExternalUrl : https://mail.contoso.com/OAB BasicAuthentication : False WindowsAuthentication : True OAuthAuthentication : True InternalAuthenticationMethods : {WindowsIntegrated, OAuth} ExternalAuthenticationMethods : {WindowsIntegrated, OAuth} [PS] C:\Windows\system32>Get-AutodiscoverVirtualDirectory | fl server,*auth* Server : EX1 InternalAuthenticationMethods : {Basic, Ntlm, WindowsIntegrated, WSSecurity, OAuth} ExternalAuthenticationMethods : {Basic, Ntlm, WindowsIntegrated, WSSecurity, OAuth} LiveIdNegotiateAuthentication : False WSSecurityAuthentication : True LiveIdBasicAuthentication : False BasicAuthentication : True DigestAuthentication : False WindowsAuthentication : True OAuthAuthentication : True AdfsAuthentication : False Once you have checked these over, you might need to add OAuth here and there. It’s important to make sure all the servers are consistent, there’s really nothing harder to troubleshoot than when one server out of ten is wrong… (Top Nerd Note: I hope you know why we didn’t include *url* in the Get-AutodiscoverVirtualDirectory cmdlet? Answers in the comments section if you do. There are no prizes to be won!) If you need to add an Auth method, here’s a tip. For all except /Mapi, just set the -OAuthAuthentication property to $True. Done. But for /Mapi you need add it explicitly, and not using some fancy @Add PowerShell thing you learned in some online course or from that smart guy in the office who tells everyone he doesn’t use ECP as it’s for kids and dogs. Because I've learned too that sometimes that doesn’t always work the way it should. If you needed to add OAuth to all the Mapi Vdirs in the org, do it like this; Get-MapiVirtualDirectory | Set-MapiVirtualDirectory -IISAuthenticationMethods Ntlm, OAuth, Negotiate Up to this point no clients should have been impacted (unless you messed the Vdir auth up, and if you did, you should only have been adding OAuth, not taking others away…you know that now don’t you). So next we start to impact clients – so this is the bit you want to do out of normal business hours. For career reasons. So, make sure you validate the following: Make sure you have completed the steps above in the Azure AD Configuration section. All the SPN’s you need should be in there. Make sure OAuth is enabled on all virtual directories used by Outlook. Make sure your clients are up to date and HMA capable by validating you have the minimal version as defined in our supportability requirements. Make sure you have communicated what you are doing. Set the EvoSts authentication provider as the default provider (this step affects Outlook 2016 for Mac and native EAS clients that support OAuth right away): Set-AuthServer EvoSTS -IsDefaultAuthorizationEndpoint $true Enable the OAuth client feature for Windows Outlook: Set-OrganizationConfig -OAuth2ClientProfileEnabled $True That’s it. All the prep you did means it comes down to two cmdlets. Wield the power wisely. How do I Know I’m Using HMA? After HMA is enabled, the next time a client needs to authenticate it will use the new auth flow. Just turning on HMA may not immediately trigger a re-auth for any client. To test that HMA is working after you have enabled it, restart Outlook. The client should switch to use the Modern Auth flow. You should see an ADAL generated auth dialog, from Office 365. Once you enter the username you might be redirected to your on-premises IDP, like ADFS (and might not see anything at all if Integrated auth is configured), or you might need to enter a password. You might have to do MFA, it depends on how much stuff you’ve set up in AAD already. Once you get connected (and I hope you do), check Outlook’s Connection Status dialog (Ctrl-Right Click the Outlook tray icon) you will see the word Bearer in the Authn column – which is the sign that it’s using HMA. Well done you. Check everyone else is ok before heading home though, eh? Something Went Wrong. How do I Troubleshoot HMA? Ah, you’re reading this section. It’s panic time, right? I was thinking of not publishing this section until next year, just for giggles. Mine, not yours. But I didn’t. Here’s what to think about if stuff isn’t working like I said it would. Firstly, make sure you did ALL the steps above, not some, not just the ones you understood. We’ve all seen it, 10 steps to make something work, and someone picks the steps they do like it’s a buffet. If you’re sure you’ve done them all, let’s troubleshoot this together. If you need to simply turn this back off then just run the last two cmdlets we ran again, but setting them to False this time. You might need to run IISReset on Exchange more than once, we cache settings all over the place for performance reasons, but those two will put you back to where you were if all hope is lost (hopefully you still have a chance to capture a trace as detailed in a moment before you do this, as it will help identity what went wrong). If you aren’t reverting the settings just yet, you clearly want to troubleshoot this a bit. First thing is – is the client seeing any kind of pop up warning dialog? Are they seeing any certificate errors? Trust or name mismatches, that sort of thing? Anything like that will stop this flow in its tracks. The clients don’t need anything more than trusting the endpoints they need to talk to – Exchange, AAD (login.windows.net and login.microsoftonline.com) and ADFS or your iDP of choice if in use. If they trust the issuer of the certs securing those sites, great. If you have some kind of name translation thing going on somewhere, that might cause a warning, or worse, a silent failure. Here’s an example of this I saw recently. Exchange was published using Web Application Proxy (WAP). You can do that, but only in pass-through mode. The publishing rule for AutoDiscover in this case was using autodiscover.contoso.com to the outside world, but the WAP publishing rule was set up to forward that traffic to mail.contoso.com on the inside. That causes this to fail, as Outlook heads to AAD to get a token for the resource called https://autodiscover.contoso.com and it does. Then it hands that to WAP, who then forwards to Exchange using the https://mail.contoso.com target URI – the uri used in the token isn’t equal to the uri used by WAP… kaboom. So, don’t do that. But I’ll show you later how an error like that shows up and can be discovered. Assuming certificates are good, we need to get deeper. We need to trace the traffic. The tool I prefer to use for this is Fiddler, but there are others out there that can be used. Now, Fiddler or the like can capture everything that happens between client and server – and I mean everything. If you are doing Basic auth, Fiddler will capture those creds. So, don’t run a Fiddler trace capturing everything going on and share it with your buddies or Microsoft. We don’t want your password. Use a test account or learn enough about Fiddler to delete the passwords. I’ll leave it to the Telerik people who create Fiddler to tell you how to install and really use their tool, but I’ll share these few snippets I’ve learned, and how I use it to debug HMA. Once installed and with the Fiddler root certs in the trusted root store (Fiddler acts as a man-in-the-middle proxy) it will capture traffic from whatever clients you choose. You need to enable HTTPS decryption (Tools, Options, HTTPS), as all our traffic is encased in TLS. If you have ADFS you can either choose to configure Fiddler to Skip Decryption for the ADFS url, if you don’t want to see what happens at ADFS, but if you do, you will have to relax the security stance of ADFS a bit to allow the traffic to be properly captured. Only do this while capturing the traffic for debug purposes, then reset it back. Start with bypassing decryption for the iDP first, come back to this if you suspect that is the issue. To set level of extended protection for authentication supported by the federation server to none (off) Set-AdfsProperties -extendedprotectiontokencheck none Then to set it back to the default once you have the capture: Set-AdfsProperties -extendedprotectiontokencheck Allow Read more about all that clever ADFSstuff here. Now you run the capture. Start Fiddler first, then start Outlook. I suggest closing all other apps and browsers, so as not to muddy the Fiddling waters. Keep an eye on Fiddler and Outlook, try and log in using Outlook, or repro the issue, then stop tracing (F12). Now we shall try to figure out what’s going on. I prefer the view where I have the traffic listed in the left hand pane, then on the right the top section is the request, and hte lower right in the response. But you do whatever works for you. But Fiddler shows each frame, then splits each into the Request, and the Response. That’s how you need to orient yourself. So the flow you’ll see will be something like this; Client connects to Exchange, sending an empty ‘Bearer‘ header. This is the hint to tell Exchange it can do OAuth but does not yet have a token. If it sends Bearer and a string of gobbledygook, that’s your token. Here are two examples of this. The header section to look at is Security. This is using Fiddler’s Header view. Do you see how the Security header says just Bearer on the left, but shows Bearer + Token on the right. Exchange responds with (lower pane of the same packet in Fiddler, raw view), here’s where you can get a token (link to AAD). If you scroll all the way to the right you’ll see the authorization_uri (AAD) Normally, Outlook goes to that location, does Auth, gets a token, comes back to Exchange, and then tries to connect using Bearer + Token as above. If it’s accepted, it’s 200’s and beers all round and we’re done. Where could it go wrong? Client Failure Firstly, the client doesn’t send the empty Bearer header. That means isn’t even trying to do Bearer. This could be a few things. It could be that you are testing with Outlook 2010 which doesn’t support Bearer (so stop trying and upgrade). Maybe you are using Outlook 2013 but forgot to set the EnableADAL reg keys set? See the link below for those. But what if this is Outlook 2016, which has EnableADAL set by default and it is still not sending the Header…. Huh? Most likely cause, someone has been tinkering around in the registry or with GPO’s to set registry keys. I knew a guy who edited the registry once and three days later crashed his car. So, do not tell me you were not warned. You need to make sure keys are set as per https://support.office.com/en-us/article/Enable-Modern-Authentication-for-Office-2013-on-Windows-devices-7dc1c01a-090f-4971-9677-f1b192d6c910 Outlook2016 for Mac can also have MA disabled (though it’s enabled by default). You can set it back to the default by running this from Terminal: defaults write com.microsoft.Outlook DisableModernAuth -bool NO That’s how we deal with the client not sending the Header. Check again and see the Header in all its Header glory. Auth_URI Failures Next thing that might happen is the server doesn’t respond with the authorization-uri, or it’s the wrong one. If there’s no authorization_uri at all then the EvoSts AuthServer does not have IsDefaultAuthorizationEndpoint set to $true. Recheck you ran Set-AuthServer EvoSts -IsDefaultAuthorizationEndpoint $true If it comes back, but with some other value than expected, make sure the right AuthServer is set as default, we only support you using AAD for this flow. If you think setting this to your on-premises ADFS endpoint will make this work without AAD… you’re wrong, as you discovered when you tried. If you are thinking of trying it, don’t bother. That’s an Exchange 2019 thing. Oh, did I just let that out of the bag? If HMA is enabled at the org level, but connections still don’t elicit the authorization_uri you expect it’s likely OAuth isn’t enabled on the Virtual Directory Outlook is trying to connect to. You need to simply make sure you have OAuth enabled on all VDirs, on all servers. Go back to the How Do I Enable section and check those VDirs again. Now, sometimes that all comes back ok but the client still doesn’t take the bait. If so, check for the following in the response; HTTP/1.1 401 Unauthorized Content-Length: 0 Server: Microsoft-IIS/8.5 Microsoft-HTTPAPI/2.0 request-id: a8e9dfb4-cb06-4b18-80a0-b110220177e1 Www-Authenticate: Negotiate Www-Authenticate: NTLM Www-Authenticate: Basic realm="autodiscover.contoso.com" X-FEServer: CONTOSOEX16 x-ms-diagnostics: 4000000;reason="Flighting is not enabled for domain 'gregt@contoso.com'.";error_category="oauth_not_available" X-Powered-By: ASP.NET WWW-Authenticate: Bearer client_id="00000002-0000-0ff1-ce00-000000000000", trusted_issuers="00000001-0000-0000-c000-000000000000@f31f3647-5d87-4b69-a0b6-73f62aeab14c", token_types="app_asserted_user_v1 service_asserted_app_v1", authorization_uri="https://login.windows.net/common/oauth2/authorize" Date: Thu, 13 Jul 2017 18:22:13 GMT Proxy-Support: Session-Based-Authentication Now this response is interesting because it says, go get a token (www-authenticate), but in x-ms-diagnostics it says, no, don’t. Is Exchange unsure? This means OAuth is enabled, but not for Outlook for Windows. So, you ran one of the two commands above (or you ran them both but not enough time has passed for them to kick in) Verify that the OAuth2ClientProfileEnabled property is set to $true by checking; (Get-OrganizationConfig).OAuth2ClientProfileEnabled Other Failures We have a token, we know OAuth is enabled at the Org level in Exchange, we know all the Vdirs are good. But it still won’t connect. Dang, what now? Now you’ll have to start to dig into server responses more closely, and start looking for things that look like errors. The errors you’ll see are usually in plain English, though of course that doesn’t mean they make sense. But here are some examples. Missing SPNs Client goes to AAD to get a token and get this: Location: urn:ietf:wg:oauth:2.0:oob?error=invalid_resource&error_description=AADSTS50001%3a+The+application+named+https%3a%2f%2fmail.contoso.com%2f+was+not+found+in+the+tenant+named+contoso.com.++This+can+happen+if+the+application+has+not+been+installed+by+the+administrator+of+the+tenant+or+consented+to+by+any+user+in+the+tenant.++You+might+have+sent+your+authentication+request+to+the+wrong+tenant.%0d%0aTrace+ID%3a+cf03a6bd-610b-47d5-bf0b-90e59d0e0100%0d%0aCorrelation+ID%3a+87a777b4-fb7b-4d22-a82b-b97fcc2c67d4%0d%0aTimestamp%3a+2017-11-17+23%3a31%3a02Z Name Mismatches Here’s one I mentioned earlier. There’s some device between client and server changing the names being used. Tokens are issued for specific uri’s, so when you change the names… HTTP/1.1 401 Unauthorized Content-Length: 0 WWW-Authenticate: Bearer client_id="00000002-0000-0ff1-ce00-000000000000", trusted_issuers="00000001-0000-0000-c000-000000000000@8da56bec-0d27-4cac-ab06-52ee2c40ea22,00000004-0000-0ff1-ce00-000000000000@contoso.com,00000003-0000-0ff1-ce00-000000000000@8da56bec-0d27-4cac-ab06-52ee2c40ea22", token_types="app_asserted_user_v1 service_asserted_app_v1", authorization_uri="https://login.windows.net/common/oauth2/authorize", error="invalid_token" Server: Microsoft-IIS/8.5 Microsoft-HTTPAPI/2.0 request-id: 5fdfec03-2389-42b9-bab9-c787a49d09ca Www-Authenticate: Negotiate Www-Authenticate: NTLM Www-Authenticate: Basic realm="mail.contoso.com" X-FEServer: RGBMSX02 x-ms-diagnostics: 2000003;reason="The hostname component of the audience claim value 'https://autodiscover.contoso.com' is invalid";error_category="invalid_resource" X-Powered-By: ASP.NET Date: Thu, 16 Nov 2017 20:37:48 GMT SSL Offloading As mentioned in the previous section, tokens are issued for a specific uri and that value includes the protocol value ("https://"). When the load balancer offloads the SSL, the request Exchange will receives comes in via HTTP, resulting in a claim mismatch due to the protocol value being "http://": Content-Length →0 Date →Thu, 30 Nov 2017 07:52:52 GMT Server →Microsoft-IIS/8.5 WWW-Authenticate →Bearer client_id="00000002-0000-0ff1-ce00-000000000000", trusted_issuers="00000001-0000-0000-c000-000000000000@00c118a9-2de9-41d3-b39a-81648a7a5e4d", authorization_uri="https://login.windows.net/common/oauth2/authorize", error="invalid_token" WWW-Authenticate →Basic realm="mail.contoso.com" X-FEServer →RGBMSX02 X-Powered-By →ASP.NET request-id →2323088f-8838-4f97-a88d-559bfcf92866 x-ms-diagnostics →2000003;reason="The hostname component of the audience claim value is invalid. Expected 'https://mail.contoso.com'. Actual 'http://mail.contoso.com'.";error_category="invalid_resource" Who’s This? Perhaps you ignored my advice about syncing all your users to AAD? HTTP/1.1 401 Unauthorized Cache-Control: private Server: Microsoft-IIS/7.5 request-id: 63b3e26c-e7fe-4c4e-a0fb-26feddcb1a33 Set-Cookie: ClientId=E9459F787DAA4FA880A70B0941F02AC3; expires=Wed, 25-Oct-2017 11:59:16 GMT; path=/; HttpOnly X-CalculatedBETarget: ex1.contoso.com WWW-Authenticate: Bearer client_id="00000002-0000-0ff1-ce00-000000000000", trusted_issuers="00000001-0000-0000-c000-000000000000@cc2e9d54-565d-4b36-b7f0-9866c19f9b17" x-ms-diagnostics: 2000005;reason="The user specified by the user-context in the token does not exist.";error_category="invalid_user" X-AspNet-Version: 4.0.30319 WWW-Authenticate: Basic realm="mail.contoso.com" WWW-Authenticate: Negotiate WWW-Authenticate: NTLM X-Powered-By: ASP.NET X-FEServer: E15 Date: Tue, 25 Oct 2016 11:59:16 GMT Content-Length: 0 Password Changed? When the user changes their password they must re-authenticate to get a new Refresh/Access token pair. HTTP/1.1 400 Bad Request Cache-Control: no-cache, no-store Pragma: no-cache Content-Type: application/json; charset=utf-8 Expires: -1 Server: Microsoft-IIS/8.5 Strict-Transport-Security: max-age=31536000; includeSubDomains X-Content-Type-Options: nosniff x-ms-request-id: f840b3e7-8740-4698-b252-d759825e0300 P3P: CP="DSP CUR OTPi IND OTRi ONL FIN" Set-Cookie: esctx=AQABAAAAAABHh4kmS_aKT5XrjzxRAtHz3lyJfwgypqTMzLvXD-deUmtaub0aqU_17uPZe3xCZbgKz8Ws99KNxVJSM0AglTVLUEtzTz8y8wTTavHlEG6on2cOjXqRtbgr2DLezsw_OZ7JP4M42qZfMd1mR0BlTLWI3dSllBFpS9Epvh5Yi0Of5eQkOHL7x97IDk_o1EWB7lEgAA; domain=.login.windows.net; path=/; secure; HttpOnly Set-Cookie: x-ms-gateway-slice=008; path=/; secure; HttpOnly Set-Cookie: stsservicecookie=ests; path=/; secure; HttpOnly X-Powered-By: ASP.NET Date: Thu, 16 Nov 2017 20:36:16 GMT Content-Length: 605 {"error":"invalid_grant","error_description":"AADSTS50173: The provided grant has expired due to it being revoked. The user might have changed or reset their password. The grant was issued on '2017-10-28T17:20:13.2960000Z' and the TokensValidFrom date for this user is '2017-11-16T20:27:45.0000000Z'\r\nTrace ID: f840b3e7-8740-4698-b252-d759825e0300\r\nCorrelation ID: f3fc8b2f-7cf1-4ce8-b34d-5dd41aba0a02\r\nTimestamp: 2017-11-16 20:36:16Z","error_codes":[50173],"timestamp":"2017-11-16 20:36:16Z","trace_id":"f840b3e7-8740-4698-b252-d759825e0300","correlation_id":"f3fc8b2f-7cf1-4ce8-b34d-5dd41aba0a02"} Unicorn Rampage? When a Unicorn Rampage has taken place and all tokens are invalidated you’ll see this. HTTP/1.1 400 Bad Unicorn Cache-Control: no-cache, no-store, not-bloody-safe Pragma: no-cache Content-Type: application/json; charset=utf-8 Expires: -1 Server: Microsoft-IIS/8.5 Strict-Transport-Security: max-age=31536000; includeSubDomains X-Content-Type-Options: nosniff x-ms-request-id: f840b3e7-8740-4698-b252-d759825e0300 P3P: CP="DSP CUR OTPi IND OTRi ONL FIN" Set-Cookie: esctx=AQABAAAAAABHh4kmS_aKT5XrjzxRAtHz3lyJfwgypqTMzLvXD-deUmtaub0aqU_17uPZe3xCZbgKz8Ws99KNxVJSM0AglTVLUEtzTz8y8wTTavHlEG6on2cOjXqRtbgr2DLezsw_OZ7JP4M42qZfMd1mR0BlTLWI3dSllBFpS9Epvh5Yi0Of5eQkOHL7x97IDk_o1EWB7lEgAA; domain=.login.windows.net; path=/; secure; HttpOnly Set-Cookie: x-ms-gateway-slice=008; path=/; secure; HttpOnly Set-Cookie: stsservicecookie=ests; path=/; secure; HttpOnly X-Powered-By: ASP.NET Date: Thu, 16 Nov 2017 20:36:16 GMT Content-Length: 605 {"error":"unicorn_rampage","error_description":"The Unicorns are on a rampage. It’s time go home” '2017-11-16T20:27:45.0000000Z'\r\nTrace ID: f840b3e7-8740-4698-b252-d759825e0300\r\nCorrelation ID: f3fc8b2f-7cf1-4ce8-b34d-5dd41aba0a02\r\nTimestamp: 2017-11-16 20:36:16Z","error_codes":[50173],"timestamp":"2017-11-16 20:36:16Z","trace_id":"f840b3e7-8740-4698-b252-d759825e0300","correlation_id":"f3fc8b2f-7cf1-4ce8-b34d-5dd41aba0a02"} And so on. You can see there are a few things that can go wrong, but Fiddler is your friend, so use it to debug and look closely and often the answer is staring you right there in the face. Viewing Tokens Lastly, and just for fun, if you want to see what an actual, real life Access token looks like, I’ll show you how… calm down, it’s not that exciting. In Fiddler, in the Request (upper pane), where you see Header + Value (begins ey…), you can right click the value and choose Send to Text Wizard, and set Transform to ‘From Base64’. Or you can copy the entire value and use a web site such as https://jwt.io to transform them into a readable format like this. { "aud": "https://autodiscover.contoso.com/", "iss": "https://sts.windows.net/f31f3647-5d87-4b69-a0b6-73f62aeab14c/", "acr": "1", "aio": "ASQA2/8DAAAAn27t2aiyI+heHYucfj0pMmQhcEEYkgRP6+2ox9akUsM=", "amr": [ "pwd" ], "appid": "d3590ed6-52b3-4102-aeff-aad2292ab01c", "appidacr": "0", "e_exp": 262800, "enfpolids": [], "family_name": "Taylor", "given_name": "Greg", "ipaddr": “100.100.100.100", "name": "Greg Taylor (sounds like a cool guy)", "oid": "7f199a96-50b1-4675-9db0-57b362c5d564", "onprem_sid": "S-1-5-21-2366433183-230171048-1893555995-1654", "platf": "3", "puid": "1003BFFD9ACA40EE", "scp": "Calendars.ReadWrite Contacts.ReadWrite Files.ReadWrite.All Group.ReadWrite.All Mail.ReadWrite Mail.Send Privilege.ELT Signals-Internal.Read Signals-Internal.ReadWrite Tags.ReadWrite user_impersonation", "sub": "32Q7MW8A7kNX5dPed4_XkHP4YwuC6rA8yBwnoROnSlU", "tid": "f31f3647-5d87-4b69-a0b6-73f62aeab14c", "unique_name": "GregT@contoso.com", "upn": "GregT@contoso.com", "ver": "1.0" } Fun times, eh? I was just relieved to see my enfpolids claim was empty when I saw that line, that sounds quite worrying and something I was going to ask my doctor about. Summary We’ve covered why HMA is great, why it’s more secure, how to get ready for it and how to enable it. And even how to troubleshoot it. Like all changes it requires careful planning and execution, and particularly when messing with auth, be super careful, please. If people can’t connect, that’s bad. We’ve been running like this for months inside Microsoft, and we too missed an SPN when we first did it, so it can happen. But if you take your time and do it right, stronger, better and heck, a more Modern auth can be yours. Good luck Greg Taylor Principal PM Manager Office 365 Customer Experience
