My recent task was to expand our existing Lync environment (Lync Server 2013 Standard) with an Edge and a Reverse proxy server. (This guide probably works for Skype for Business as well). Our old Lync environment had been in test usage for a while (with a rather small test-user group), but with more and more Lync users adding up it was time to expand. As a matter of fact, a Lync environment without Edge and reverse proxy servers is rather useless – you are unable to organize external meetings.
First, let me start off by saying that there are A LOT of “moving parts” involved in configuring a reverse proxy and an Edge server. You must plan for IP addresses, DMZ settings, DNS settings, certificates, firewall settings and so forth. To get a grasp of the whole picture I’m suggesting that you read/watch the following:
https://technet.microsoft.com/en-us/library/dn594589.aspx – A very good protocol poster (Skype for Business Server 2015 Protocol Workloads) that helps with the overall picture. It’s also very good for checking firewall requirements/port configurations. Yes, it’s a bit overwhelming but very good in the end 🙂
https://technet.microsoft.com/en-us/library/jj204756%28v=ocs.15%29.aspx – Explains (firewall) ports.
A good place to continue after this would be http://blog.schertz.name/2012/07/lync-edge-server-best-practices/. It’s a very nice all-around document about Edge and Reverse proxy. Pay close attention to the chapter about Best Practices. Our goal in the end was to get something that resembles this picture:
Fig 1. Lync Front-end in combination with Edge and Reverse proxy – Simple Topology. (Pic source: http://blog.schertz.name/2012/07/lync-edge-server-best-practices/)
Even after a lot of reading, It’s hard to know where to start (in this blog post). There are soooooo many different things going on and a lot of stuff to remember. A good place to start could be certificate planning, which also means that you have to decide which IPs/hostnames you’ll be using in your own environment. Then again, I think the best place to start is planning network infrastructure/topology. First, consider whether you are going with a simple topology (Front-end, Edge server, Reverse proxy – we’re using this) OR a complex topology (multiple Front-ends, Edge servers, Reverse proxies). Second, It’s very important to have a working DMZ, and you should also know if you’ll be using (only) Public IP addresses or public IPs in combination with NATed ones. After you’ve got an answer to these questions it will be much easier planning for the other requirements. With this in mind, I’ll start off with the networking part. I’ll then move over to areas like DNS, certificates, actual Edge server installation, IIS ARR installation and finally some words about mobility and federation.
But first off, here’s a short explanation of what the Edge and Reverse proxy servers bring to the table:
Includes 4 modules:
Access Edge service. The Access Edge service provides a single, trusted connection point for both outbound and inbound Session Initiation Protocol (SIP) traffic.
Web Conferencing Edge service. The Web Conferencing Edge service enables external users to join meetings that are hosted on your internal Lync Server 2013 deployment.
A/V Edge service. The A/V Edge service makes audio, video, application sharing, and file transfer available to external users. Your users can add audio and video to meetings that include external participants, and they can communicate using audio and/or video directly with an external user in point-to-point sessions. The A/V Edge service also provides support for desktop sharing and file transfer.
XMPP Proxy service. The XMPP Proxy service accepts and sends extensible messaging and presence protocol (XMPP) messages to and from configured XMPP Federated partners.
The reverse proxy is required for the following:
To allow users to connect to meetings or dial-in conferences using simple URLs
To enable external users to download meeting content
To enable external users to expand distribution groups
To allow the user to obtain a user-based certificate for client certificate based authentication
To enable remote users to download files from the Address Book Server or to submit queries to the Address Book Web Query service
To enable remote users to obtain updates to client and device software
To enable mobile devices to automatically discover Front End Servers offering mobility services
To enable push notifications to mobile devices from the Office 365 or Apple push notification services
Networking / Network interfaces
I’m now assuming that you have:
A working Lync Server Standard/Enterprise 2013 (or Skype for Business) Front-end
A soon-to-become (Lync) Reverse Proxy server (Windows Server 2012 R2)
A soon-to-become Lync Edge server (Windows Server 2012 R2)
Talked to your network guys about the network infrastructure (IPs/DMZ). Hardware (F5) load balancers can be a whole different story for example.
Talked to your firewall guys about opening ports. I myself sat down with the Skype for Business Server 2015 Protocol Workloads printout and had a long discussion with a firewall guy. We/he got the job done without any hiccups. (It’s still working fine today 🙂 )
On the Reverse Proxy:
Assign one IP for the internal network adapter and one for the external network adapter. Internal and External should be in different subnets. One interface is communicating with the internet and the other one is communicating with your internal network/AD. Have a look at https://technet.microsoft.com/en-us/library/gg429707%28v=ocs.15%29.aspx or https://technet.microsoft.com/en-us/library/jj204691%28v=ocs.15%29.aspx for examples. (I’m not going into much DNS details (yet), but you could name these new IPs lyncproxy-int.domain.com and lyncproxy-ext.domain.com for example).
Set the default gateway on the external network adapter only
Assign static routes. From my experience, the information regarding this can be a bit difficult to understand. Let me copy/paste the information from the above link:
“Important: Similar to the Edge Servers, you set the default gateway on the external network adapter only. The default gateway will be the IP address of the router or external facing firewall that directs traffic to the Internet. For traffic that is destined from the reverse proxy to the internal facing network adaptor, you must use persistent static routes (such as the route command in Windows Server) for all subnets containing servers referenced by the web publishing rules. Setting a persistent route does not cause the computer to become a router. If IP forwarding is not enabled, the computer is acting only to direct specific traffic destined for another network to the appropriate interface. This is essentially setting two gateways – one as the default pointing to the external networks, and one for traffic destined to the internal interface and on to a router or other network.
However, creating persistent routes for all subnets may not be necessary if your network’s routers are configured to summarize routes. Create a persistent route to the network where the router is defined and use the router as the default gateway. If you are not sure how your network is configured and need guidance on what persistent routes need to be created, consult with your company’s Network Engineers.
The reverse proxy must be able to resolve the DNS host (A) records for the internal Director or Front End Server and next hop pool FQDNs used in the web publishing rules. As with the Edge Servers, for security reasons, we recommend that you do not configure a reverse proxy to use a DNS server located in the internal network. This means you either need DNS servers in the perimeter, or you need HOSTS file entries on the reverse proxy that resolves each of these FQDNs to the internal IP address of the servers”.
In plain English this means that you configure the external interface “normally”, as you would with any other external network interface in your infrastructure. You should define the gateway as the “IP address of the router or external facing firewall that directs traffic to the Internet”. Your network guys can help you with this if unsure (also see the next chapter). The internal side on the other hand should not have a default gateway – instead you configure static routes. I’ll try to explain this:
Example network subnets (defined by your network administrators):
External DMZ (16 addresses, all are not needed but room for expansion)
Internal DMZ (16 addresses, all are not needed but room for expansion)
Example external network adapter configuration on the server:
Example Internal network adapter configuration on the server:
GW: no gw
We’re using split-brain DNS so the internal and external DNS names are the same. All IP’s are from a Class B chunk, and they’re all public IP’s that are defined as internal or external in the firewall/DNS. With the above configuration in place, you should now add a route to the internal interface on the reverse proxy server. This is done with the route add command (The –p switch make the changes persistent). Here’s an example using the above IP schema:
C:\>route add -p 126.96.36.199 mask 255.255.240.0 188.8.131.52
C:\>route add -p 184.108.40.206 mask 255.255.240.0 220.127.116.11
C:\>route add -p 18.104.22.168 mask 255.255.240.0 22.214.171.124
C:\>route add -p 126.96.36.199 mask 255.255.240.0 188.8.131.52
C:\>route add -p 184.108.40.206 mask 255.255.224.0 220.127.116.11
The above command example would make all of the above IP ranges take the route against the internal interface. All other IPs would take the external route. The above ranges are also defined as internal in DNS/firewall. Do the same for all of your internal IP ranges. This method is different when using NAT and/or non-split-brain configurations. (In case of NAT, your internal IPs are in the 192.168.x.x, 10.x.x.x, or 172.16.x.x. range). Perhaps a picture will tell more than words:
Fig 2. Internal and external overview. (Picture source: https://technet.microsoft.com/en-us/library/jj204756%28v=ocs.15%29.aspx)
This should be it for the networking part on the Reverse proxy server. Now we do the same on the Edge server.
On the Edge server:
The network configuration on the Edge server follow the same pattern as the Reverse proxy. I’m using three external IPs and one internal IP. This is by best practice design (https://technet.microsoft.com/en-us/library/gg412847%28v=ocs.15%29.aspx). If you are in a limited-budget-external-IP-dilemma, you can also make it work with one external IP (not including that option in this text however).
- Assign three external IPs
- one for SIP traffic
- one for AV traffic
- one for Web Conferencing
- Assign one internal IP
I’m not going much into DNS details now either, but you could name these new IPs lyncedge-int.domain.com, lyncedgesip-ext.domain.com, lyncedgeav-ext.domain.com and lyncedgewc-ext.domain.com for example. There’s nothing much to add here. Follow the same procedure as for the reverse proxy when configuring your internal and external network interfaces:
Example Internal network adapter configuration on the server:
GW: no gw
Example external network adapter configuration on the server:
Now add the same routes as you did on the reverse proxy. There you have it, we can now move over to the DNS part.
I assume that you by now have figured out your topology and configured networking on the involved servers. Good, that’s one step in the right direction. You might have noticed that I haven’t talked much about host names, only IP addresses. This is mostly because you can configure the networking part this far without knowing (almost) any host names. (Of course you most certainly will ask for a hostname at the same time you get an IP address, but anyways).
I have to say that DNS was one of the most confusing/difficult/challenging/painful parts in this whole configuration/deployment. There were tons and tons of misleading/wrong information, and it required countless hours of testing. Anyways, I’ll spare you the DNS-pain and tell you about our configuration in a while. But before I do, I make you read some homework. Here are a couple of interesting links (with or without errors):
Let me start off by saying that I like the jackstromberg article. All my testing was actually based on the DNS table from that article. However those records were also a bit confusing, and some even unnecessary. Here are our DNS records with comments:
Fig 3. Internal DNS
No other records are required for our specific configuration/environment (at the moment). SRV records are a thing of the past and only needed when working with Lync 2010 clients. See http://blog.schertz.name/2012/12/lync-2013-client-autodiscover/ for more information. If you are going to use federation however (which we probably are in the future), you SHOULD set up SRV records (though not needed if manually entering servers). See: http://windowsitpro.com/skype-business/understanding-main-types-federation-skype-and-lync-environments. As you can see, Allowed Partner Server (Direct Federation) works without SRV records but specifying the records when you federate will probably still make your life easier.
I will now also make a statement about the lyncdiscover.mydomain.com record in the Internal DNS. You can read on many, MANY places on the Internet that you should have this record present in the internal DNS so that mobility works. I can confirm that our users mobile phones (WP, iOS, Android) work just FINE without this record. The key is to have the external Web Services record present in the internal DNS (webext.mydomain.com) and point it to the reverse proxy. If you DO use lyncdiscover in the internal DNS, ALL traffic will go through the proxy. This is probably not a desirable configuration. Yes, I’ve seen this “live” in our environment so I know what I’m talking about. The “problem” went away after we removed the lyncdiscover.mydomain.com record from the internal DNS. Good info about this:
Read the above links CAREFULLY and you’ll have a MUCH better understanding, believe me 🙂 Again, this setup works FOR US. I’m not saying that the lyncdiscover.mydomain.com record should be removed from every internal DNS configuration out there.
Fig 4. External DNS.
External DNS was much more straight forward. Comments are included in the picture.
The DNS records are (as you can see) a bit different for the external network/outside world compared to the internal network. All external traffic goes through the reverse proxy, which in turn use URL rewrites to connect to the corresponding URLs on the inside network. (I’ll leave the URL rewrite / ISS/ARR discussion for a later chapter).
You have probably noticed that the webext.mydomain.com record is present in BOTH the internal and external DNS. The reason for this is mobile devices. Mobile devices need to access the mobility service, and they do that ONLY from the outside. I’m yet again referring to the Mobility service flow using AutoDiscover (picture) at https://technet.microsoft.com/en-us/library/hh690030%28v=ocs.15%29.aspx. Lyncs own autodiscover feature will know if the client is on the internal or external network based on the lyncdiscover/lyncdiscoverinternal record. However, it’s a whole different story with webext.mydomain.com. If this URL is accessed from either the inside or outside network, the client is unable to know it’s final destination. This is because you’re pointing the client (in both cases) to the reverse proxy, which in turn point to the same URL internally and externally. This means that you’ll end up in an endless loop. To solve this you’ll edit the hosts-file.
You also have to add a local DNS record for lyncdiscover, otherwise this record will remain unresolvable as it’s not present in the internal DNS. This was all a big mystery for me, as the documentation seldom mentioned this dilemma. I got an idea after hours of googling though – the holy hosts-file. Thanks to http://kressmark.blogspot.fi/2015/06/windows-2012-r2-iis-85-and-arr-30-as.html for the idea. This was by no means a big surprise, but you’ll get lost (in DNS) after hours and hours of testing. Believe me.
This means that you’ll have to add local DNS records on the reverse proxy. Fire up notepad and edit the C:\Windows\System32\drivers\etc\hosts –file on the reverse proxy. Add the following:
18.104.22.168 webext.mydomain.com, where 22.214.171.124 is the IP of your Front-end server.
126.96.36.199 lyncdiscover.mydomain.com, where 188.8.131.52 is the IP of your Front-end server.
(Meet and dialin are already resolvable by internal DNS and correctly points to the Front-end).
Now when a client resolves webext (internally or externally), it always gets sent to the reverse proxy. The reverse proxy in turn resolves webext to the front-end via the hosts-file. Lyncdiscover in turn won’t be resolvable internally after it reaches the reverse proxy if no hosts file-record is added. There you have it – all your DNS problems solved 🙂
Certificates and DNS go somewhat hand-in-hand as you need to know which hostnames you’ll be using in the certificates. I’d probably start off by reading again. Some information to get you started:
Take your time to read and plan – this way you’ll be rewarded in the end. The above links discuss both internal and external certificates. They also discuss the differences between the reverse proxy and the Lync Edge certificates.
We’re not completely going by best practice regarding the certificates. We’re using external certificates on the internal front-end server. This is due to the fact that we already had an external certificate installed on the Front-end. It doesn’t do much harm either, and at least for us it’s not an extra expense.
Without further stories I’ll present our “certificate solution”. We’re using one certificate per interface on the edge server, but you could also use just one certificate will all hosts included. This would be more expensive due to the fact that you’ll have to pay extra for additional SAN-names. (You DON’T need a certificate for the AV-interface on the Edge server).
Fig 5. Edge and Reverse proxy certificate chart.
To add to this list, our Front-end needed to get its public certificate renewed with the added host webext.domain.com. Before the renewal, it included public certs for lyncdiscover, lyncdiscoverinternal, meet, dialin and sip. Now it includes all those + webext. Webext is needed for the external Web Services (externally accessing the front-end). More about that later on.
The following chapters has information about how to install the certificates on the servers.
Configuring the Lync Front-end Server for Edge / Installing the Edge Server / Edge Server Certificate Installation
Congratulations If you’ve had the energy to read this far. It’s now finally time to install the Edge server 🙂 Much of the Edge installation/configuration is actually tied to the Front-end server however. You’ll start by making changes to your current topology and then export/publish the topology on the Edge server. Like you’ve probably noticed before, it’s the prep work that takes most of the time. (Internet is full of articles on how to install an Edge server). That said, I happen to like the post at http://www.sherweb.com/blog/installing-lync-2013-edge-server/ and our installation is based on this article. However, there are some differences. In Step 6, the article tells you to request internal and external certificates from the setup itself. We didn’t do it this way because our internal CA isn’t an “online certification authority” (it doesn’t respond to online web requests due to security reasons). Instead we made an offline request and signed it manually on the CA. A little bit more hassle, but worked just fine in the end.
We didn’t use the certificate request wizard for the external certificate either, as those gets created by our “certificate guy”. He uses his own methods and just delivers a fully working certificate. I won’t go into the details, but it works. So in the end, whatever floats your boat and can get you the correct certificates is fine 🙂
If using this “manual method” (alternative to Step 6 in the guide), you must manually install the certificate(s) in the certificate store(s) before continuing. This is by no means difficult. For the internal/external certificates do the following:
- Fire up “mmc.exe” on the Edge server
- Add the certificates Snap-in. Select computer account –> local computer and click OK.
- Right click on the Personal –> Certificates folder.
- Select All Tasks… Import
- Imported certificate will show up in this location.
- Move the different certificates to their corresponding places (Personal, Trusted Root Certification Authorities, Intermediate Certification Authorities). See screenshots/figures below.
Fig 6. Personal certificate (computer)
Fig 7. Trusted Root CA certificate
Fig 8. Intermediate CA certificate
- Done. External certificates are also showing in this list as I have imported them in the same manner.
Now, back to the installation post at http://www.sherweb.com/blog/installing-lync-2013-edge-server/.
Some of my own notes:
- Followed the guide.
- Network specifications were OK.
- Software specifications were OK. Didn’t (need to) install Windows Identity Foundation on the Front-end. (It should be installed as a pre-requirement on the Edge server however).
- Fired up Topology Builder on the front-end and followed the guide.
- FQDN of the new edge pool should match the FQDN (CN) on the internal certificate: lyncedge-int.domain.com from certificate chart
- Access Edge service should match the FQDN (CN) on the external certificate: lyncedgesip-ext.domain.com
- Web Conferencing Edge service should match the FQDN (CN) on the external certificate: lyncedgewc-ext.domain.com
- A/V Edge service should match the FQDN (CN) on the external certificate: lyncedgeav-ext.domain.com
- NOTE: This is a “DNS/certificate-thing”. Whatever certificate CN-record you created for the above services should be used. You can swap CN and SAN records for a more “clean” name, i.e. lyncedgeav-ext.domain.com vs. av.domain.com. (See above DNS/certificate chart and the note about switching places between CN and SAN).
- Enabled federation (not xmpp yet though)
- Defined internal and external IP addresses
- Defined Next hop pool: our front-end
- Changed the External web services in the Topology builder to match the one we have defined in DNS and in the certificate (would be webext.domain.com in the example).
Fig 9. External web services
- Published the topology
- Exported the configuration (step 4 in guide).
- Moved over to the Edge server itself
- Installed Lync server and imported the configuration (step 5 in guide)
- The certificates were already installed in the certificate store so no need to request certificates (step 6)
- Defined the existing certificates
- Started services
- Done! (no need for step 8 and 9, yet)
Installing and configuring the Reverse Proxy server / Reverse Proxy server Certificate Installation
I’m now assuming that you have a working Edge server. You can install the reverse proxy server without a working Edge server also, but installing the Edge server first makes it easier to test the reverse proxy functionality right after the installation. First some homework/reading:
I decided to go with a combination of the Microsoft guide and the jackstromberg one this time. In the end, it worked perfectly. I had lots of problems and headaches down the line, but this time it had nothing to do with the guides (rather it had to do with typos and a non-working ARR that had to be reinstalled).
Before following the (Microsoft) guide however, we have to install the certificates the same way we did on the Edge server. The reverse proxy certificates are of course different, but I’m assuming that you have requested them at the same time as the Edge certificates. Just follow the Edge steps (fire up mmc.exe and so on) and you’re good to go. One different step is that you have to bind the certificates in IIS, otherwise they won’t be used when clients connect via the reverse proxy. It’s rather easy, let me show you some screenshots:
Fig 10. Adding https bindings in IIS.
Fig 11. Adding https bindings in IIS. Remember to add both the internal and the external interface, with their own certificate.
Now continue following the guides, or use another guide of your choice.
I used the following simple URLs:
All good, tested and working! 🙂
A note from my own experiences: ARR is VERY STRICT regarding the URL rewrite rules. If something isn’t working, be sure to double check the rules!
I wasn’t quite sure if there was a need for a separate mobility chapter as I’ve covered this area quite well in the DNS and certificate chapters. I guess a couple of lines won’t do no harm however. I’ll once again start by giving you a nice list of homework/reading:
http://windowsitpro.com/lync/decoding-lyncs-endpointconfigurationcache-file (deleting cache)
Much of the mobility stuff has to do with the fact that the mobility service isn’t working properly via lyncdiscoverinternal. Instead you configure the mobile devices to go the external way, via the reverse proxy. See the following picture:
Fig 12. Lync mobility (source: http://www.ucprimer.com/tech-blog/5-essential-tips-for-deploying-lync2013-mobility-service)
The mobility bit was a big headache, but in the end we got it working in a desirable way. The secret was to remove lyncdiscover.domain.com from the internal DNS (against many recommendations). See the DNS chapter for more information.
We’re definitely interested in federation, but we haven’t federated with any partners yet. It’s no harm reading about federation though, and in the end it will be much easier setting it up once you’ve done your homework. I’ve done my homework, so why wouldn’t you 🙂 Here you go:
The Edge server is already enabled for federation, but the front-end is not. This is easily fixed in the Topology builder once we/you decide to federate:
Fig 13. Enable federation on the Front-end.
In addition to this, it’s also recommended that you add a DNS SRV record (_sipfederationtls._tcp.domain.com).
And finally here’s a screenshot I took sometime in the middle of the whole deployment. As I’ve stated before, there were lots and lots of googling and homework to be done 🙂
Fig 14. Google is your friend. Don’t believe everything you read though…
That’s Firefox with Tab Mix Plus and Multirow Bookmarks Toolbar Plus Extensions, btw.
This quite much summarizes the Lync Edge and Reverse Proxy server deployment. Hope you’ve enjoyed reading 🙂