Update for Conferencing Server fails to install on Skype for Business Server 2015

I finally had some time to install and play around with Skype for Business Server 2015 and Exchange Server 2016 in my test environment. One of the features I was looking forward to in SfB Server 2015 was the ability to use offline messaging. I followed lots and lots of different articles, and this feature should be enabled out of the box from CU3(?) onwards. You can enable it manually on older CU’s also, following


for example. There are many other guides as well, but they’re all about the same. However, I learned the hard way that this feature is not yet enabled in the on-premise version of SfB Server 2015. Correct me if I’m wrong though. Some info regarding this (without an answer):


After some serious Googling I was still unable to enable offline messaging though. This basically means that all of my updating was for “nothing”. Well, not completely. At least I learned how to force-update SfB Server 2015 🙂


First I installed the newest CU for SfB Server 2015, as this should’ve “fixed” lots of offline messaging stuff. Well, installing this CU wasn’t so straight forward after all. I then downloaded the latest CU from https://support.microsoft.com/en-us/help/3061064/updates-for-skype-for-business-server-2015 (in my case February 2017 Cumulative Server Update Installer). I followed the installation method to apply the update to a back end server or Standard Edition server. This is where the problems started.

The first problem was regarding the Url rewrite module 2.0 for IIS on Windows Server 2016. It wouldn’t install and you’ll receive an error while running the SfB Server 2015 Update Installer. Luckily I found a solution at http://stackoverflow.com/questions/35319942/url-rewrite-does-not-install-on-windows-10-iis-10. (You should edit the windows registry before running the Update Installer). Well, the problems didn’t stop here. After this problem was fixed I was greeted with:



and after a while the Update installer told me that the Conferencing Server update failed:


(never mind the versions in the screenshot, they are wrong as I had to “borrow” this picture from the ucsorted.com link below).


If you read the log it’ll tell you “ERROR 1603: OCSMCU.msp had errors installing”. I did some googling and found these:


The TechNet forum post suggest uninstalling some updates which I couldn’t even find on my system. This made the whole “fix” rather useless. The other link also talks about problematic updates, or the fact that you shouldn’t let SfB Server update itself at all before installing a new CU. This seemed very weird and the solution to reinstall everything from scratch also seemed very far fetched. There had to be a better solution.


Well, using my own common sense I started thinking about alternatives. After a while I figured why not uninstall the Conferencing Server component from the SfB Server 2015 installation and replace it with the stock one from the installation media? That said, I uninstalled the problematic component (even though SfB tells you it’s “dangerous”):


After uninstallation of the Conferencing Server component, I copied the OCSMCU.msi file from the original install media (X:\Setup\amd64\Setup) and ran it. It installed without problems. It was now time to run the Skype for Business Server 2015 Update Installer again:


Success 🙂


And just in case I ran the Skype for Business Server 2015 Update Installer once again to see the result:


All dots are green, success indeed 🙂


I then continued following the installation instructions and ran:

net start w3svc

followed by:

Install-CsDatabase -Update –LocalDatabases:



All good and everything seems to be working 🙂

I do wonder why there always seem to be problems with every update for Lync/SfB Server… oh well.

Lync Server 2013 Health Checking

I thought I’d write a tiny follow up to my previous blog post Health Checking / Monitoring Exchange Server 2013/2016. I got exited about all the health checking stuff, so naturally I had to do some research about Lync/SfB Server as well. That  said, I found a very comprehensive script written by Steve Moore. You can get the script from:


I’ve tested the script on both Lync Server 2013 and SfB Server 2015 with success. To put it briefly, it’s awesome! 🙂

As with the Exchange scripts, I wanted to use a scheduled task for running the script. I tried the same methods as with Exchange, but it failed. It turned out that this script wouldn’t accept the $true or $false parameters when running from a scheduled task. From the instructions:

Running the Script

To run the script you can use the following command line:

.\SystemStatus.ps1 -Action status -ServerFile c:\scripts\systemstatus\servers.txt -PoolName lync-pool.contoso.org -SendEmail $true -Environment Contoso -EventData $True


However, running the script from cmd (not PowerShell) or task scheduler you’ll get an error:

“Cannot convert value “System.String” to type “System.Boolean”, parameters of this type only accept booleans or numbers, use $true, $false, 1 or 0 instead”

My solution was to hard-code the parameters in the script itself, like so:


There was no need to edit PoolName but I did it anyways – it’s one less parameter to define when running the script. Same goes for Environment. The interesting parameter however was [bool]$SendEmail=$False. I changed this to $True so I didn’t have to specify it as a parameter when running the script. As a bonus I also removed CC and BCC from

Send-MailMessage -To $Recipients -From $FromAddress -CC $CC -BCC $BCC -Subject $strMailSubject -BodyAsHtml:$True -SmtpServer $SMTPHost -Body $strBody #Send the email

as I have no need to send CC’s or BCC’s. In other words:

Send-MailMessage -To $Recipients -From $FromAddress -Subject $strMailSubject -BodyAsHtml:$True -SmtpServer $SMTPHost -Body $strBody #Send the email


With these changes in place, it was easy setting up Task Scheduler:

  • General: Run whether user is logged on or not. Run with highest privileges
  • Trigger: Weekly, at 00:00 every Wednesday of every week.
  • Action: Start a program
    • Program/script: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
    • Add arguments (optional): -NoProfile -ExecutionPolicy Bypass -File “c:\software\scripts\SystemStatus.ps1” -Action status -ServerFile c:\software\scripts\servers.txt

servers.txt include the name of our front-end server.

And that’s it. You’ll then get a sexy report emailed back to you every Wednesday looking something like this:


Just some non-critical warnings and nothing major to worry about. All good!

Happy monitoring to you all! 🙂

Health Checking / Monitoring Exchange Server 2013/2016

I‘ve never wrote about monitoring / health checking before so here we go. There are maaaaany different ways of monitoring servers, so I’ll just present my way of monitoring things (in the Exchange environment). If you’re using SCOM or Nagios, you’re already halfway there. The basic checks in SCOM or Nagios will warn you about low disk space and high CPU load and so forth. But what if a Exchange service or a DAG renders errors for example? Exchange is a massive beast to master, so in the end you’ll need decent monitoring tools to make your life easier (before disaster strikes).

We’re using Nagios for basic monitoring which is working great. That said, from time to time I’ve noticed some small problems that Nagios won’t report. I have then resorted to PowerShell commands or the windows event log. These problems would probably have been noticed (in time) if we had decent/additional Exchange-specific monitoring in place. There are Exchange-plugins available for Nagios (I’ve tried a few), but they aren’t as sophisticated as custom PowerShell scripts made by “Exchange experts”. It’s also much easier running a script from Task Scheduler than configuring Nagios. At least that’s my opinion.

Anyhow, our monitoring/health checking consist of three scripts, namely:

add-pssnapin *exchange* -erroraction SilentlyContinue
$body=Get-HealthReport -Server “yourserver” | where {$_.alertvalue -ne “Healthy” -and $_.AlertValue -ne “Disabled”} | Format-Table -Wrap -AutoSize; Send-MailMessage -To “me@ourdomain.com” -From “HealthSetReport@yourserver” -Subject “HealthSetReport, yourserver” -Body ($body | out-string ) -SmtpServer yoursmtpserver.domain.com


I have all of these scripts set up as scheduled tasks. You’d think that setting up a scheduled task is easy. Well, not in my opinion. I had to try many different techniques but at least it’s working now.

For Paul’s script I’m using the following settings:


  • For “Triggers” I’m using 02:00 daily.
  • For “Actions” I’m using:
    • Start a program: powershell.exe
    • Add arguments (optional): -NoProfile -ExecutionPolicy Bypass -File “G:\software\scripts\Test-ExchangeServerHealth.ps1” –SendEmail
    • Start in (optional): G:\software\scripts


The same method wouldn’t work for Steve’s script though. I used the same “Run with highest privileges” setting, but running the PowerShell command similar to the above wouldn’t work. (This was easily tested running from a cmd promt manually (instead of powershell)). My solution:

  • Triggers: 01:00 every Saturday (yeah, I don’t feel the need to run this every night. Paul’s script will report the most important things anyways).
  • Actions:
    • Start a program: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
    • Add arguments (optional): -NonInteractive -WindowStyle Hidden -command “. ‘C:\Program Files\Microsoft\Exchange Server\V15\bin\RemoteExchange.ps1’; Connect-ExchangeServer -auto; G:\software\scripts\Get-ExchangeEnvironmentReport.ps1 -HTMLReport G:\software\scripts\environreport.html -SendMail:$true -MailFrom:environreport@yourserver -MailTo:me@ourdomain.com -MailServer:yoursmtpserver.domain.com
    • Start in (optional): (empty)


My own script:

  • Triggers: 00:00 every Sunday (yeah, I don’t feel the need to run this every night. Paul’s script will report the most important things anyways).
  • Actions:
    • Start a program: powershell.exe
    • Add arguments (optional): -NoProfile -ExecutionPolicy Bypass -File “G:\software\scripts\Get-HealthSetReport.ps1”
    • Start in (optional): G:\software\scripts



http://practical365.com/exchange-server/powershell-script-exchange-server-health-check-report/ (Paul’s script)
http://www.stevieg.org/2011/06/exchange-environment-report/ (Steve’s script)



Exchange also checks its own health. Let me copy/paste some information:

One of the interesting features of Exchange Server 2013 is the way that Managed Availability communicates the health of individual Client Access protocols (eg OWA, ActiveSync, EWS) by rendering a healthcheck.htm file in each CAS virtual directory. When the protocol is healthy you can see it yourself by navigating to a URL such as https://mail.exchangeserverpro.net/owa/healthcheck.htm.
When the protocol is unhealthy the page is unavailable, and instead of the HTTP 200 result above you will see a “Page not found” or HTTP 404 result instead.

Source: http://practical365.com/exchange-server/testing-exchange-server-2013-client-access-server-health-with-powershell/

Further reading: https://blogs.technet.microsoft.com/exchange/2014/03/05/load-balancing-in-exchange-2013/ and the chapter about Health Probe Checking

We have no need to implement this at the “Exchange server-level” though, as these checks are already done in our Zen Load Balancer (described in this blog post). I guess you could call this the “fourth script” for checking/monitoring server health.



So, what all this adds up to are some nice reports. I’ll get a daily mail (generated at 02:00) looking like this (and hopefully always will 🙂 ):


Daily report generated from Test-ExchangeServerHealth.ps1 script



Weekly report generated from Get-ExchangeEnvironmentReport.ps1 script



Weekly report generated from Get-HealthSetReport.ps1 script (The problem was already taken care of 🙂 )


As you can see from the screenshots above, these checks are all reporting different aspects of the Exchange server health.

This quite much covers all the necessary monitoring in my opinion and with these checks in place you can probably also sleep better during the nights 🙂 Even though these checks are comprehensive, I’m still planning on even more checks. My next step will be an attempt at Real-time event log monitoring using NSClient++ / Nagios. Actually the Nagios thought was buried and I will instead focus on the ELK stack which is a VERY comprehensive logging solution.

Using FarmGuardian to enable HA on Back-ends in Zen Load Balancer

We’ve been using the Zen Load Balancer Community Edition in production for almost a year now and it has been working great. I previously wrote a blog post about installing and configuring Zen, and now it was time to look at the HA aspect of the back-end servers defined in various Zen farms. Zen itself is quite easy to set up in HA-mode. You just configure two separate Zen servers in HA-mode according to Zen’s own documentation. Well, this is very nice and all, and it’s also working as it should. The thing that confused me the most however (until now), is the HA aspect of the back-ends. I somehow thought that If you specify two back-ends in Zen and one of them fail, Zen automatically uses the backend which is working and marked as green (status dot). Well, this isn’t the case. I don’t know if I should blame myself or the poor documentation – or both. Anyways, an example is probably better. Here’s an example of L4xNAT-farms for Exchange (with two back-ends):


I guess it’s quite self-explanatory; we’re Load Balancing the “normal” port 443 + imap and smtp. (All the smtp-ports aren’t open to the Internet though, just against our 3rd party smtp server). The http-farm is used for http to https redirection for OWA.

Furthermore, expanding the Exchange-OWAandAutodiscover-farm:



and the monitoring part of the same farm:



This clearly shows that the “Load Balancing-part” of Zen is working – the load is evenly distributed. You can also see that the status is green on both back-ends. Fine. Now one would THINK that the status turns RED if a back-end is down and that all traffic would flow through the other server if this happens. Nope. Not happening. I was living in this illusion though 😦 As I said before, this is probably a combination of my own lack of knowledge and poor documentation. Also, afaik there are no clear “rules” for the farm type you should use when building farms. Zen itself (documentation) seem to like l4xnat for almost “everything”. However, if you’re using HTTP-farms, you get HA on the back-ends out-of-the box. (You can specify back-end response timeouts and checks for resurrected back-ends for example). Then again, you’ll also have to use SSL-offloading with the http-farm which is a whole different chapter/challenge when used with Exchange. If you’re using l4xnat you will NOT have HA enabled on the back-ends out-of-the-box and you’ll have to use FarmGuardian instead. Yet another not-so-well-documented feature of Zen.

FarmGuardian “documentation” is available at https://www.zenloadbalancer.com/farmguardian-quick-start/. Have a look for yourself and tell me if it’s obvious how to use FarmGuardian after reading.

Luckily I found a few hits on Google (not that many) that were trying to achieve something similar:


These gave me some ideas. Well, I’ll spare you the pain of googling and instead I’ll present our (working) solution:


First off, you’ll NEED a working script or command for the check-part. Our solution is actually a script that checks that every virtual directory is up and running on each exchange back-end. If NOT, the “broken” back-end will be put in down-mode and all traffic will instead flow through the other (working) one. I chose 60 sec for the check time, as Outlook times out after one minute by default (if a connection to the exchange server can’t be established). Here’s the script, which is based on a script found at https://gist.github.com/phunehehe/5564090:


Big thanks to the original script writer and to my workmate which helped me modify the script. Sorry, only available in “screenshot form”.

You can manually test the script by running ./check_multi_utl.sh “yourexchangeserverIP”  from a Zen terminal:


The (default) scripts in Zen are located in /usr/local/zenloadbalancer/app/libexec btw. This is a good place to stash your own scripts also.


You can find the logs in /usr/local/zenloadbalancer/logs. Here’s a screenshot from our log (with everything working):



And lastly I’ll present a couple of screenshots illustrating how it looks when something is NOT OK:

(These screenshots are from my own virtual test environment, I don’t like taking down production servers just for fun 🙂 )


FarmGuardian will react and present a red status-symbol. In this test, I took down the owa virtual directory on ex2. When the problem is fixed, status will return to normal (green dot).


and in the log:


The log will tell you that the host is down.


Oh, as a bonus for those of you wondering how to do a http to https redirect in Zen:


Create new HTTP-farm and leave everything as default. Add a new service (name it whatever you want) and then just add the rules for redirection. Yes, it’s actually this simple. At least after you find the documentation 🙂

And there you have it. Both the Zen servers AND the back-ends working in HA-mode. Yay 🙂

Exchange 2007 to 2013 migration

This time around I’ll write about all the steps I used for a successful Exchange 2007 to 2013 migration in a small size company (about 40 users). I’m not quite sure (yet) if we will do the upgrade to 2016 directly afterwards, however I’ll make a new 2013 to 2016 migration blog post if we decide to do so. At this point I can also state that it is impossible/not supported to do a migration from Exchange 2007 to Exchange 2016 directly, at least without 3rd party software. You can have a look at the “migration chart” at https://technet.microsoft.com/en-us/library/ms.exch.setupreadiness.e16e12coexistenceminversionrequirement%28v=exchg.160%29.aspx for example. If you want to pay extra money and go directly from 2007 to 2016 it should be possible with CodeTwo Exchange Migration, http://www.codetwo.com/blog/migrate-legacy-2003-or-2007-exchange-to-exchange-2016/ for example.

Anyways, onto the migration stuff itself. As always you should start with the homework. This time I don’t have that many sources for you – instead some quality ones which gets the job done properly. Start off by reading:

The second link is awesome! Read it slowly and carefully. You’ll be a lot smarter in the end. Lots of stuff to think of, but very nicely written.

I didn’t follow the guide exactly to the word (as usual), but I couldn’t have done the job without it. Some changes for our environment:

  • We do not use TMG. All TMG steps were bypassed and were replaced by similar steps according to our own firewall policies.
  • We have a Linux postfix server that handles all incoming email. It also handles antivirus and spam checking of emails. After these checks are done, it forwards email to the Exchange server.
  • Storage configuration / Hard drive partitions / Databases weren’t created the same way as in the guide.
  • Certificates were renewed by our “certificate guy”. No need for complicated requests etc.
  • No stress tests and/or analyses were done. No need.
  • Configured recipient filtering (there’s a chapter about it).
  • A script which deletes old IIS and Exchange logs was introduced (there’s a part written about this also).


My own steps for the migration process:

On the old server:

  • Patched Exchange Server 2007. Also installed the latest Update Rollup (21). You should have a fully patched (old)server before installing/introducing a new Exchange Server in the domain.


  • Took screenshots of all current configurations (just in case). Most of the settings will migrate however. Stuff to backup are nicely documented in the above homework-link.
    • Namespace
    • Receive connectors
    • Send connectors
    • Quotas
    • Outlook Anywhere, OWA, OAB, EWS, ActiveSync settings
    • Accepted domains
    • Etc.. etc. that would be of use
  • Got a new certificate which included the new host legacy.domain.com
  • Installed the new certificate (on Exchange 2007 at first):



On the new server:

  • Installed a new server, Windows Server 2012 R2.


      • Moving on to the other prerequisites:



Moving on to the actual Exchange installation

  • Had a look at my partition table, just to check that everything looked OK. (it did):


  • The partition layout should be quite self-explanatory so I won’t comment on that. I will however tell setup to use the existing partitions. I actually resized the partitions a bit after this screenshot…
  • Once again following information from the excellent guide, I used the latest CU as installation source (NOT the installation DVD/ISO).




  • Actual installation (note paths for DB and Logs):


  • Done. Moving over to post-installation steps


Post-installation steps

  • Checking and changing the SCP. This should be done asap after the installation.


          Checking SCP.


          Changing SCP.

  • Everything looks good!
  • Next, we’ll install the new certificate on the Exchange 2013 server:


           A simple “import” will do the job.

  • Also have a look at the certificate in IIS (and change to the new one if necessary):



Following the guide you should change the authentication to NTLM:

“As Outlook Anywhere is the protocol Outlook clients will use to communicate with Exchange Server 2013, replacing MAPI/RPC within the LAN, it’s important that these settings are correct – even if you are not publishing Outlook Anywhere externally. During co-existence it’s also important to ensure that the default Authentication Method, Negotiate, is updated to NTLM to ensure client compatibility when Exchange 2013 proxies Outlook Anywhere connections to the Exchange 2007 server”.

  • Moving over to the send and receive connectors.
    • The send connector automatically “migrated” from the old server.
    • The receive connector did NOT migrate from the old server. This is because Exchange 2013 use different roles for transportation compared to 2007. 2007 included only Hub Transport, but Exchange 2013 use both Hub Transport and Frontend Transport. For those of you interested in this change, read http://exchangeserverpro.com/exchange-2013-mail-flow/ and http://exchangeserverpro.com/exchange-2013-configure-smtp-relay-connector/ for example.
    • The CAS receives mail on port 25 and forwards it to the “backend” mailboxes that listens on port 2525.
    • I left the “Default Frontend servername” with its default settings:


    • …and configured a new SMTP relay-connector which has “our settings”. This connector has to be “Frontend Transport”. You cannot create a new connector as Hub Transport. You’ll be greeted by an error message if you try:


Information about this can be found at:


If you want to create a new receive connector that listen on port 25, you can do this but you have to create it using the Frontend Transport role if you have either an Exchange 2016 server or an Exchange 2013 server with both the CAS and MBX roles installed on the same server”.

All our University email (and this specific company’s email) is received via a Linux postfix server. This server handles all spam filtering and antivirus. After these checks are done, the mail is delivered/forwarded to Exchange.




After these steps were done, I continued with:

  • Configuring mailbox quotas to match those on the old server.
  • Configuring the Offline Address Book to be stored on the new server.
  • Checking the log locations – should the transport logs be moved to another location or left at the default location? I changed them so they will go to the log-partition. In the end, this is just a small percentage of all logs generated. All other non-transport logs gets filled under C:\Program Files\Microsoft\Exchange Server\V15\Logging. I’m using a PowerShell script to delete all logs older than 30 days, and the same goes for the IIS logs in C:\inetpub\logs. The script looks like this, and is run via Task Scheduler daily:

$DateToDelete = 30
$StartFolder = “C:\Program Files\Microsoft\Exchange Server\V15\Logging”
$Year = (Get-Date).Year
$Day = Get-Date
Get-ChildItem $StartFolder -Recurse -Force -ea 0 | where{!$_.PsIsContainer -and $_.LastWriteTime -lt (Get-Date).AddDays(-$DateToDelete)} | ForEach{Add-Content -Path “Delete Log $Year.log” -Value ” $_.FullName”; Remove-Item -Path $_.FullName }
$DateToDelete = 30
$StartFolder = “e:\Logs”
$Year = (Get-Date).Year
$Day = Get-Date
Get-ChildItem $StartFolder -Recurse -Force -ea 0 | where{!$_.PsIsContainer -and $_.LastWriteTime -lt (Get-Date).AddDays(-$DateToDelete)} | ForEach{Add-Content -Path “Delete Log $Year.log” -Value ” $_.FullName”; Remove-Item -Path $_.FullName }
$DateToDelete = 30
$StartFolder = “c:\inetpub\logs”
$Year = (Get-Date).Year
$Day = Get-Date
Get-ChildItem $StartFolder -Recurse -Force -ea 0 | where{!$_.PsIsContainer -and $_.LastWriteTime -lt (Get-Date).AddDays(-$DateToDelete)} | ForEach{Add-Content -Path “Delete Log $Year.log” -Value ” $_.FullName”; Remove-Item -Path $_.FullName }

And the command to run from task scheduler:

  • PowerShell.exe -NoProfile -ExecutionPolicy Bypass -Command “& ‘D:\pathtoyour\scripts\clearlogging.ps1′”
    • Runs daily at 03:00

As you’ve probably noticed from my Exchange installation screenshots, I already pointed the Transaction logs to a different partition in the installation phase (E:\Databases\DB1). These logs don’t need manual deletion however, they get deleted via the backup solution automatically (Veeam). The key here is that the backup software has to be Exchange aware. The other logs at e:\ are the Transport logs (E:\Logs), which are only a tiny part of the whole logging structure (C:\Program Files\Microsoft\Exchange Server\V15\Logging) in Exchange. You could leave the Transport logs in their default location though, as the above script will go through that directory also…


Recipient filtering / Stopping backscatter

As a nice bonus, Exchange 2013 can now handle recipient filtering (filter out non-existent users) properly. For more information about recipient filtering read:


The filtering CAN be done without an Exchange Edge server even though Internet will tell you otherwise. We enabled it on our postfix server following tips found on https://www.roaringpenguin.com/recipient-verification-exchange-2013. Installation on the Exchange-side on the other hand looked like this:



I also enabled Anonymous users on the “Default receive connector”:


Happy days! We can now filter out non-existent users on Exchange rather than manually on the postfix server.

I also checked that recipient filtering was active and working:


Yes, it was 🙂

With all this done I now moved forward with the configuration. Again, following http://www.msexchange.org/articles-tutorials/exchange-server-2013/migration-deployment/planning-and-migrating-small-organization-exchange-2007-2013-part13.html


Getting ready for coexistence

I’ll start off by copy/pasting some text.

“With our database settings in place and ready to go, we can start thinking about co-existence – before we do though, it’s time to make sure things work within Exchange 2013! So far we’ve got our new server up and running, but we’ve still not logged in and checked everything works as expected”. Source: http://www.msexchange.org/articles-tutorials/exchange-server-2013/migration-deployment/planning-and-migrating-small-organization-exchange-2007-2013-part13.html

With this information in mind, I started testing according to the above link. The chapter of interest was “Testing base functionality”. All tests passed. Very nice 🙂

With all tests done, and all users aware of the migration, I did the following after work hours:

    • Asked the “DNS guy” to make a CNAME record for legacy.domain.com pointing to the old server.
    • Changed all virtual directories on the old server to use the name “legacy”.
      • Things to remember:
        • No external url for Microsoft-Server-ActiveSync.
        • Autodiscover Internal URL / SCP record on both Exchange 2007 and Exchange 2013 server should point to the new server.
    • DNS records are changed to point to the new server.
      • autodiscover and the namespace –record
    • Had a look at the send connector. Everything seemed OK. (Settings were migrated from the old server). However, minor change:
      • Removed the old server from the “source servers” and added the new server. New mail should be sent from the new server (and not from the old one anymore):


    • postfix was also configured to route mail to the new server instead of the old one.
    • Done. Next in line is moving/migrating mailboxes to the new server. Yay.


Migrating mailboxes

I started out by following the guide at http://www.msexchange.org/articles-tutorials/exchange-server-2013/migration-deployment/planning-and-migrating-small-organization-exchange-2007-2013-part15.html , more specifically the part about “Pre-Migration Test Migrations”. I moved a couple of test users and after that I sent and received mail to/from these users via Outlook and OWA. No errors were noticed, so I moved over to the real deal and started moving “real” mailboxes. Again, nothing special, I continued following the information at http://www.msexchange.org/articles-tutorials/exchange-server-2013/migration-deployment/planning-and-migrating-small-organization-exchange-2007-2013-part16.html. I did a batch of 10 users at first (users A to E) and all of them were successfully migrated:


(The remaining mailboxes were also successfully migrated).


Upgrading Exchange AD Objects

Now it was time to upgrade the AD Objects following information from http://www.msexchange.org/articles-tutorials/exchange-server-2013/migration-deployment/planning-and-migrating-small-organization-exchange-2007-2013-part16.html.



The first two objects didn’t need an upgrade, they were apparently already automatically upgraded during the migration process. The distribution group in the screenshot that needed an upgrade is a mailing list/distribution group.


Public Folders

The old environment didn’t use public folders so luckily there were no need to migrate these. I did run into some problems with Public Folders however. More information in the chapter below.



  • Everything seemed fine, BUT after a couple of days one user didn’t see any new mail in a delegated mailbox she had. She also got the dreaded password prompt every time she started Outlook.
    • Later I heard that also other users were prompted for password
  • This got me thinking about authentication methods. I’ve seen this before. A couple hours of googling still had my thoughts in the same direction, authentication methods.
  • I still wonder why all of this happened though, knowing that ALL mailboxes were now hosted on the new Exchange 2013 server. Why on earth would someone’s Outlook even check for things on the old server? Maybe some old Public Folder references etc. perhaps? Don’t know, the only thing I do know is that it had to be fixed.

Some links about the same dilemma (almost, at least):

http://blogs.microsoft.co.il/yuval14/2014/08/09/the-ultimate-guide-exchange-2013-and-outlook-password-prompt-mystery/ (L. Authentication Issue)

The thing is, I had authentication set to “NTLM” on the new Exchange 2013 server during the coexistence, following the very same guide as with almost everything else in this post. The NTLM setting should be “enough” afaik. One thing that wasn’t mentioned in the guide however, was how the old server was/should be configured. I’m quite sure there are many best practices for Exchange 2007 also, but I myself hadn’t installed that server in the past. Well, hours later, comparing different authentication methods, I finally think I got it right. Here’s the before and after:


Before: old server IISAuthenticationMethods were only Basic.


Solution: Adding NTLM to IISAuthenticationMethods (on the legacy server)


After: NTLM added

I also removed the “Allow SSL offloading” from the new server for consistency. Not that I know if it helped fixing the problem or not.


You get kinda tired from all testing and googling, but hey, at least its working as it should and users aren’t complaining anymore! 🙂


  • Shared mailbox dilemma. When you send a message from the shared mailbox, the sent message goes into your own Sent Items folder instead of the shared mailbox sent items.

If the shared mailbox is on Exchange 2010 and only has the Exchange 2010 sent items behavior configured, the settings are not converted to the equivalent Exchange 2013 settings during migration. You will need to manually apply the Exchange 2013 sent items configuration. It is probably best to do that before moving the mailbox. The Exchange 2010 settings are retained though”. Source: http://exchangeserverpro.com/managing-shared-mailbox-sent-items-behaviour-in-exchange-server-2013-and-office-365/

    • Well, no wonder my settings didn’t stick when migrating from 2007 to 2013. I configured the correct settings again:
      • Get-Mailbox mysharedmailbox | Set-Mailbox -MessageCopyForSentAsEnabled $true -MessageCopyForSendOnBehalfEnabled $true


Decommission Exchange 2007

Still following the guide, it was now time to decommission the old Exchange 2007 server. First off I left the server turned OFF for a week. No problems were encountered, so I decided to move on with the real decommissioning work.

  • Didn’t need to touch any TMG rules (obviously, since we don’t use TMG)
  • Removed unused Offline Address Books (OAB)
  • Removed old Databases
    • Mailbox Database removal was OK.
    • Public Folders were a whole different story. What a headache. I followed almost every guide/instruction out there. Did NOT WORK. I got the “nice” message: “The public folder database “ExchangeServer\Storage Group\Public Folder Database” contains folder replicas. Before deleting the public folder database, remove the folders or move the replica to another public folder database”. God dammit. We’ve never ever been using Public Folders. Well, luckily I found some useful “fixes” after a while. Some “fixes” that MS won’t mention. Solutions:
    • Removed CN=Configuration,CN=Services, CN=Microsoft Exchange, CN={organisation name i.e First Organisation}, CN=Administrative Groups, CN={Administrative Group name}, CN=Servers, CN={servername}, CN=Information Store, CN={Storage Group Name}, CN={Public Folder Database Name} with ADSIEdit (after I had backed up the key with help from http://www.mysysadmintips.com/windows/active-directory/266-export-active-directory-objects-with-ldifde-before-performing-changes-with-adsi-edit for example).
    • Ran the Get-MailboxDatabase | fl name,pub* –command again, but to my surprise the damn Public Folder Database wasn’t gone. Instead it was in the AD “Deleted Objects”. FFS, it CAN’T be this hard removing the PF Database (reference).
    • Trying to get rid of the deleted object with ldp didn’t work either: “The specified object does not exist”. I was getting even more frustrated.
    • Well, at least now according to EMC I have no active Mailbox Databases. That’s good news, so I can now remove the Storage Groups even though this annoying PF DB reference still exist in AD. I can live with it for now, and hopefully when the Tombstone Lifetime expires, so will this PF DB reference. (That wasn’t the case however, continue reading)
  • Removed Storage Groups, FINALLY:



      • System mailboxes are already on the new server. Good.
  • Uninstalled Exchange 2007 from control panel.
    • At least I tried. Of course there were problems. Again.


Got some tips from https://social.technet.microsoft.com/Forums/exchange/en-US/6469264a-dc33-4b07-8a7c-e681a0f9248f/exchange-setup-error-there-was-a-problem-accessing-the-registry-on-this-computer?forum=exchangesvradminlegacy. Solution was simply to start the Remote Registry service. It now uninstalled nicely.


  • Removed legacy DNS entries
  • Firewall guy was informed that the server was decommissioned and all its firewall rules could be removed.
  • Turned off the server and archived it.
  • Happy days. No more Exchange 2007.


Security hardening

I always aim to keep my servers secure. This one was no exception, so I was aiming for at least a grade A on the Qualys SSL Labs test, https://www.ssllabs.com/ssltest/. I followed the guide from https://scotthelme.co.uk/getting-an-a-on-the-qualys-ssl-test-windows-edition/ and voilà, grade A was achieved  🙂 I left the HTTP Strict Transport Security policy alone for now however, it will need some more testing.


TeamViewer on Raspberry Pi 3

During the summer we moved from an apartment to a house. Among many other things, this meant changes to the Internet connectivity. Our old place had VDSL, but now I got to choose what I want. Fiber optics would’ve been an option (fast and reliable), but it’s rather expensive compared to the other alternatives. In the end, I ended up with a 50/50 Mbit/s 4G mobile broadband connection which is shared through a 4G router (which I already owned). This was by far the most bang for the buck. We also happen to live quite near a 4G base station, so regular speeds reach about 40Mbit/s+. Not bad. Not bad at all. I’m not a gamer either, so no need for low ping times. On the negative side, all Finnish 4G mobile broadband connections are NAT:ed on the ISP side (exception is enterprise/business subscribers). This dilemma is also the subject of this post, in other words how to “bypass” dual NAT using TeamViewer. (I had to ditch my fancy old VPN stuff  😦 )

Well, moving along to the subject, TeamViewer. TeamViewer is by no means a new technology, but the possibility to use it on the RPi is limited. When googling you’ll most certainly find a solution named ExaGear Desktop. ExaGear Desktop enables x86 programs to be run (or “emulated”) on the RPi. While this is a good idea, it’ll most certainly be slow. A RPi running Raspbian is already quite slow, so doing x86 emulation on top of that is even slower. You’ll also find people telling you it’s impossible to run TeamViewer on the RPi altogether (which is untrue).

Why would you need TeamViewer on the RPi you ask? Isn’t there already SSH and port forwarding and such? Yes, BUT port forwarding won’t work with (double) NAT:ed 3G/4G connections. (NAT at the ISP level and NAT behind the router). As stated before, my main Internet connection at home is a 4G mobile broadband connection (without a public IP) connected to a 4G router. This combination doesn’t allow me to remote connect to my LAN using the “normal methods” (VPN, port forwarding and so on). The reason for this is that you can’t configure the ISP NAT rules (obviously). Some similar information about this dilemma can be found here for example:

https://elisa.fi/asiakaspalvelu/aihe/mobiililaajakaista/ohje/ip-osoitteet/ (in Finnish)


Luckily there’s a solution to this dilemma (and luckily I’m here to minimize your googling):

The TeamViewer IoT contest: https://www.teamviewer.com/iotcontest  🙂

Just download and install it in Raspbian (with a simple double click). After that have a look at the Frequently Asked Questions and the General Questions & Answers + Troubleshooting Questions and Answers on the same download page. The only settings I changed were:

  • Sat an unattended fixed password and disabled random password (via the GUI).
  • Edited /boot/config.txt and commented out framebuffer_width and framebuffer_height lines. I left them at 1280×720 which worked OK for me. (I did this because I’m running the RPi headless and the resolution would therefore be screwed up (too tiny) if connecting without a monitor attached to the RPi).
  • Added the RPi to the “My Computers”-list in the TeamViewer client (from Windows)
  • Voilà!


…and some screenshots:


Fig 1. Connected, yey  🙂



Fig 2. Having a look at the doggycam remotely



Alternative solutions

  • Pay a slightly more expensive monthly fee for your mobile broadband connection, and in return get a public IP address. This is true for almost all ISP’s here in Finland at least.
  • Some ISP’s provide public IP’s on 4G mobile broadband connections using IPv6. This only works with a couple of mobile broadband sticks/routers however (they have to fully support IPv6). I wouldn’t buy a new stick (or router) only for this feature. (They’re rather expensive after all).
  • Use TeamViewer on a x86 computer instead of a RPi. Yes, why not. Then again it consumes much more energy than a RPi when in use 24/7. It’s also noisy. And hey, everything is always much more cool and hip with a RPi 🙂


That’s it! This was just a short little post that hopefully will make your life a whole lot easier when dealing with the evil NAT:ed 4G connections 🙂

ownCloud 9 on Raspberry Pi 2 with mounted Buffalo NAS

The Linux nerd inside me was screaming for a new RPi project. What to build? What to do? You can’t read any modern IT literature nowadays without stumbling upon the word “cloud”. Well, cloud it is. Owncloud in my case. I guess almost everyone is familiar with this open source cloud software, but for those of you that aren’t you can find information at:


The idea was to access my trusty old Buffalo NAS remotely, without having the need to map network drives etc. Buffalo is actually offering some sort of cloud solution also, but hey, it’s much more fun configuring your own stuff 🙂 The idea is quite simple – The RPi is a front-end for the NAS. Clients are connecting to the RPi, which in turn mount network storage from the NAS. Here’s the setup:


Fig 1. RPi + Buffalo NAS


Initial questions and ideas

  • Should Raspberry Pi / ownCloud be visible on the Internet? If so, how to secure it properly?
      • Port forwarding with restrictions / reverse proxy?
  • If not visible on the Internet, how should one connect from the outside world?
      • VPN?

I actually decided to go with option 2, not visible on the Internet. My decision is based on the fact that I’m already running a VPN server. It’s one more extra step before getting/synchronizing the files, but I think it’s worth it in the end. Besides, all my other services are behind VPN also.

That said, I STILL configured ownCloud to be “future-proof” even if the server won’t be Internet-facing (with port forwarding etc.) right now. (See the securing ownCloud chapter). Better safe than sorry 🙂



As with almost every project, I usually follow an existing guide. ownCloud is quite a mainstream product, so there are tons and tons of documentation available. The guide that I used as my baseline this time was: http://www.htpcguides.com/install-owncloud-8-x-raspberry-pi-for-personal-dropbox/ . Thanks to the author 🙂 Steps:

  • Followed the guide down to “Now make your ownCloud directory adjust your path as necessary to your mounted hard drive folder..”. As I’ll be using a NAS, it was time for another guide:
    • http://sharadchhetri.com/2013/10/23/how-to-mount-nas-storage-in-owncloud-to-keep-all-users-data/
    • created a share on the NAS (named owncloud). Gave the share read/write access for a user also named “owncloud”.
    • mounted the share on the RPi. The guide uses backslash, but it should be forward slash:
      • e.g. mount -t cifs // /mnt -o username=owncloud,password=owncloud
    • Did not follow step 3 completely because there was no data-directory created during the installation (yet). The installer GUI will look for a data directory though, so this is the time to create and mount it properly.
    • Got the uid of www-data or apache user by using id command:
      • root@owncloud:~# id www-data
        uid=33(www-data) gid=33(www-data) groups=33(www-data),1000(pi)
      • OK. ID is 33
    • Created a local data-directory which will mount the owncloud share (from the NAS).
    • mkdir -p /var/www/owncloud/data
      • changed ownership and permission on the data directory;
        • chmod -R 770 /var/www/owncloud/data ; chown -R www-data:www-data /var/www/owncloud/data
      • Added the following line to /etc/fstab (bottom of file) to make the data directory available in owncloud (setup):
        • // /var/www/owncloud/data cifs user,uid=33,rw,suid,username=owncloud,password=owncloud,file_mode=0770,dir_mode=0770,noperm 0 0
    • Ran mount –a and checked if the NAS got properly mounted. For me it did. In other words the “local” /var/www/owncloud/data was now actually living on the NAS.
    • Finished the configuration via ownClouds own GUI setup. Everything went fine…
    • …however, after a reboot the share was NOT auto mounted 😦
    • I got an error when trying to access owncloud over the web interface: Please check that the data directory contains a file “.ocdata” in its root
      • Scratched my head and wondered what the hell went wrong. I was quite sure it had to do with permissions. Turned out I was right. Short version:
      • http://htyp.org/Please_check_that_the_data_directory_contains_a_file_%22.ocdata%22_in_its_root
        • created an empty .ocdata –file (after I had manually mounted /var/www/owncloud/data directory from the NAS).
        • chmodded that file and the data-directoy with “new rights” ;
          • chmod 777 .ocdata ; chmod 777  /var/www/owncloud/data
          • success, the NAS now got automounted after a RPi-reboot 🙂
    • Everything worked, so moving over to the (optional) security part.


Securing ownCloud

Owncloud’s own Security & setup warnings will warn you about things to fix. Here’s a screenshot with almost no security measurements taken. (Actual screenshot is not mine, it’s “borrowed” from the Internet):


Fig 2. Before fixing Security & Setup warnings. I’ll write about memory cache in the next chapter (Optimizing ownCloud).


… and here’s a screenshot with fixed security (also memcached fixed):


Fig 3. No security & setup warnings 🙂


Basic security

The initial setup guide I followed had already done some basic security measurements (luckily):

  • Redirected all unencrypted traffic to HTTPS (in /etc/nginx/sites-available/owncloud):
      server {
        listen 80;
        server_name htpcguides.crabdance.com;
        return 301 https://$server_name$request_uri;  # enforce https
  • Used SSL certificates for https (self-signed):
      ssl_certificate /etc/nginx/ssl/owncloud.crt;
      ssl_certificate_key /etc/nginx/ssl/owncloud.key;
  • Created a virtual host for owncloud, not using “default”.
  • Protected the data directory and files from the internet (outside world):
       # Protecting sensitive files from the evil outside world
        location ~ ^/owncloud/(data|config|\.ht|db_structure.xml|README) {
                 deny all;
  • After this, there were still some things to take care of. Although not visible in Fig 2 above, I also got a warning saying that HTTP Strict Transport Security wasn’t used. Well, a quick googling fixed this. All that was needed was a parameter in the same configuration file as above (/etc/nginx/sites-available/owncloud):

More information about security can be found in ownClouds own Hardening and Security Guidance:



Advanced security

If you are going to deploy a server that’s facing the Internet you have to think about security. The basic security measurements are a must, but what if you want to secure it even more? You certainly want your site protected against DDOS and brute-force attacks, don’t you? Well, here’s where one of my favorites come into play – fail2ban. If you have no idea what I’m talking about I suggest that you read at least the following:

https://www.techandme.se/fail2ban-owncloud/ (<- Specifically THIS link)

If you’re lazy, just follow the above guides and setup accordingly. However, use common sense and double check that everything seems to be in order before going production. I myself created the following jail-files and configuration files, and restarted fail2ban. Everything was working as expected 🙂 My configuration files:

root@owncloud:/etc/fail2ban/filter.d#  ls -la ngin*.*
-rw-r–r– 1 root root 345 May 23 09:28 nginx-auth.conf
-rw-r–r– 1 root root 422 Mar 15  2014 nginx-http-auth.conf
-rw-r–r– 1 root root 280 May 23 09:29 nginx-login.conf
-rw-r–r– 1 root root 300 May 23 09:28 nginx-noscript.conf
-rw-r–r– 1 root root 230 May 23 09:28 nginx-proxy.conf
-rw-r–r– 1 root root 282 May 24 09:53 nginx-req-limit.conf


root@owncloud:/etc/fail2ban/filter.d# ls -la own*
-rw-r–r– 1 root root 146 May 23 09:08 owncloud.conf

(Contents of these files can be found in the links above)

…and jail-files:

root@owncloud:/etc/fail2ban# cat jail.local

enabled = true
filter  = owncloud
port    = https
bantime  = 3000
findtime = 600
maxretry = 3
logpath = /var/www/owncloud/data/owncloud.log

enabled = true
filter = nginx-auth
action = iptables-multiport[name=NoAuthFailures, port=”http,https”]
logpath = /var/log/nginx*/*error*.log
bantime = 600 # 10 minutes
maxretry = 6

enabled = true
filter = nginx-login
action = iptables-multiport[name=NoLoginFailures, port=”http,https”]
logpath = /var/log/nginx*/*access*.log
bantime = 600 # 10 minutes
maxretry = 6

enabled  = true
filter = apache-badbots
action = iptables-multiport[name=BadBots, port=”http,https”]
logpath = /var/log/nginx*/*access*.log
bantime = 86400 # 1 day
maxretry = 1

enabled = false
action = iptables-multiport[name=NoScript, port=”http,https”]
filter = nginx-noscript
logpath = /var/log/nginx*/*access*.log
maxretry = 6
bantime  = 86400 # 1 day

enabled = true
action = iptables-multiport[name=NoProxy, port=”http,https”]
filter = nginx-proxy
logpath = /var/log/nginx*/*access*.log
maxretry = 0
bantime  = 86400 # 1 day

enabled = true
filter = nginx-req-limit
action = iptables-multiport[name=ReqLimit, port=”http,https”, protocol=tcp]
logpath = /var/log/nginx/*error*.log
findtime = 600
bantime = 7200
maxretry = 10


After you’ve created the jails and the configuration files you should restart the fail2ban service, “sudo service fail2ban restart”. You can then have a look in the log file, /var/log/fail2ban.log to see if everything looks ok. For me it did:

2016-05-24 09:55:53,094 fail2ban.jail   [4778]: INFO    Jail ‘ssh’ started
2016-05-24 09:55:53,136 fail2ban.jail   [4778]: INFO    Jail ‘owncloud’ started
2016-05-24 09:55:53,162 fail2ban.jail   [4778]: INFO    Jail ‘nginx-auth’ started
2016-05-24 09:55:53,190 fail2ban.jail   [4778]: INFO    Jail ‘nginx-login’ started
2016-05-24 09:55:53,223 fail2ban.jail   [4778]: INFO    Jail ‘nginx-badbots’ started
2016-05-23 10:28:13,243 fail2ban.jail   [1350]: INFO    Jail ‘nginx-noscript’ started
2016-05-24 09:55:53,249 fail2ban.jail   [4778]: INFO    Jail ‘nginx-proxy’ started
2016-05-24 09:55:53,281 fail2ban.jail   [4778]: INFO    Jail ‘nginx-req-limit’ started

All this configuration is a bit overkill for me as I’m not going to expose the ownCloud server on the Internet. Instead I’m using VPN + ownCloud. This is however a great opportunity to learn about ownCloud and it’s security so it would be a shame NOT to configure it as secure as possible 🙂 (The ssh-jail is a very nice bonus if you’re also forwarding that port towards the Internet).


Optimizing ownCloud

After all the security hardening stuff it was time to look at optimization. The initial guide includes some optimization and it installs all the PHP modules needed for memory caching. I’m quite sure I could optimize ownCloud much more, but there’s no need to overdo it in such a small home environment. In other words, memory caching is enough in my case. More info about memory caching can be found in ownClouds own documentation: https://doc.owncloud.org/server/8.1/admin_manual/configuration_server/caching_configuration.html

Even though this topic is already covered at the bottom in the initial guide (http://www.htpcguides.com/install-owncloud-8-x-raspberry-pi-for-personal-dropbox/), I’ll write a summary here:

  • Edit /var/www/owncloud/config/config.php
  • At the bottom of the file, add:
    'memcache.local' => '\OC\Memcache\Memcached',
    'memcache.distributed' => '\OC\Memcache\Memcached',
    'memcached_servers' => 
    array (     0 => 
    array (     0 => '',     1 => 11211,     ),     ),
  • Check that memcached is running:

      root@owncloud:# netstat -nap | grep memcached
      tcp        0      0*               LISTEN      457/memcached
      udp        0      0*                           457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7961     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7955     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7753     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7967     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7960     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7954     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7966     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7964     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7958     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7963     457/memcached
      unix  3      [ ]         STREAM     CONNECTED     7957     457/memcached

      Source: http://stackoverflow.com/questions/1690882/how-do-i-see-if-memcached-is-already-running-on-my-chosen-port

  • Done. You should now have a pretty safe (and optimized) environment to play with 🙂


ownCloud works pretty much the same way as Dropbox and others. Here’s a screenshot from a Mac client and one test-file synchronized:


Fig 4. ownCloud + Mac.


Useful configuration paths

I’ll finish this blog post with a short summary of useful configuration paths. Here you go:

/var/www/owncloud/config/config.php (https://doc.owncloud.org/server/8.1/admin_manual/configuration_server/config_sample_php_parameters.html)