Health Checking / Monitoring SharePoint 2016

I’ve previously written about health checking/monitoring in the blog posts Health Checking / Monitoring Exchange Server 2013/2016 and Lync Server 2013 Health Checking. These posts are very popular, so I decided to write a similar post about Health Checking / Monitoring SharePoint 2016.

This time however, there weren’t that many useful scripts available. Almost all of the scripts I found were too basic, including just a few PowerShell commands. I on the other hand was looking for something similar to the Exchange health report, including a weekly email and so forth. That said, I finally managed to find a a couple of nice sources:

http://abhayajoshi.blogspot.com/2016/05/sharepoint-server-daily-check.html
https://gallery.technet.microsoft.com/SharePoint-Server-Check-V10-f5e205b4

My own script is a modified version of the one found in the first link, with “borrowed” features from the one found in the second link. The end result looks like this, and is just what I was looking for:

(WordPress is not that cooperative with large pictures, use your browsers zoom feature on this embedded picture instead).

sharepoint_server_report_merged

You’ll get all the important stuff from the email report – ranging from basic information such as memory and disk usage to more advanced stuff like core SharePoint services and application reports including upgrade needs. I don’t mind sharing either, so here’s my modified PowerShell script (with $servers and email addresses censored):


$head='<style>body{font-family:Calibri;font-size:10pt;}th{background-color:black;color:white;}td{background-color:#19fff0;color:black;}h4{margin-right: 0px; margin-bottom: 0px; margin-left: 0px;}</style>'

#Add SharePoint PowerShell Snap-In
Add-PSSnapin Microsoft.SharePoint.PowerShell -erroraction SilentlyContinue

#SharePoint Servers, use comma to separate.
$servers = @("xxxxx")

#===============#
# Server Report #
#===============#

#Memory Ustilization
#Modified to display GB instead of byte, source:
#https://www.petri.com/display-memory-usage-powershell

#I'm skipping the free percentage stuff //JS

#$Memory = Get-WmiObject -Class Win32_OperatingSystem -ComputerName $servers| Select CsName , FreePhysicalMemory  , TotalVisibleMemorySize , Status | ConvertTo-Html -Fragment

$Memory = Get-Ciminstance Win32_OperatingSystem | Select @{Name = "Total(GB)";Expression = {[int]($_.TotalVisibleMemorySize/1mb)}},
@{Name = "Free(GB)";Expression = {[math]::Round($_.FreePhysicalMemory/1mb,2)}} | ConvertTo-Html -Fragment

#Disk Report
$disk = Get-WmiObject -Class Win32_LogicalDisk -Filter DriveType=3 -ComputerName $servers |
Select DeviceID , @{Name="Size(GB)";Expression={"{0:N1}" -f($_.size/1gb)}}, @{Name="Free space(GB)";Expression={"{0:N1}" -f($_.freespace/1gb)}} | ConvertTo-Html -Fragment

#Server UpTime
$FarmUpTime = Get-WmiObject -class Win32_OperatingSystem -ComputerName $servers |
Select-Object __SERVER,@{label='LastRestart';expression={$_.ConvertToDateTime($_.LastBootUpTime)}} | ConvertTo-Html -Fragment

#Core SharePoint services
#Borrowed and modified from https://gallery.technet.microsoft.com/SharePoint-Server-Check-V10-f5e205b4 //JS
$coreservices = Get-WmiObject -Class Win32_Service -ComputerName $servers | ? {($_.Name -eq "AppFabricCachingService") -or ($_.Name -eq "c2wts") -or ($_.Name -eq "FIMService") `
-or ($_.Name -eq "FIMSynchronizationService") -or ($_.Name -eq "Service Bus Gateway") -or ($_.Name -eq "Service Bus Message Broker") -or ($_.Name -eq "SPAdminV4") `
-or ($_.Name -eq "SPSearchHostController") -or ($_.Name -eq "OSearch15") -or ($_.Name -eq "SPTimerV4") -or ($_.Name -eq "SPTraceV4") -or ($_.Name -eq "SPUserCodeV4") `
-or ($_.Name -eq "SPWriterV4") -or ($_.Name -eq "FabricHostSvc") -or ($_.Name -eq "WorkflowServiceBackend") -or  ($_.Name -eq "W3SVC") -or ($_.Name -eq "NetPipeActivator")} `
| Select-Object DisplayName, StartName, StartMode, State, Status| ConvertTo-Html -Fragment

#====================#
# Application Report #
#====================#

#SharePoint Farm Status
$SPFarm = Get-SPFarm | select Name , NeedsUpgrade , Status , BuildVersion |ConvertTo-Html –Fragment

#Web Application Pool Status
#Source: http://stevemannspath.blogspot.com/2013/06/sharepoint-2013-listing-out-existing.html
$WAppPool = [Microsoft.SharePoint.Administration.SPWebService]::ContentService.ApplicationPools | select Name, Username,Status | ConvertTo-Html -Fragment

#Web Application Status Check
$WebApplication = Get-SPWebApplication | Select Name , Url , ContentDatabases , NeedsUpgrade , Status | ConvertTo-Html -Fragment

#IIS Total number of current connections
$IIS1 = Get-Counter -ComputerName $servers -Counter "\web service(_total)\Current Connections" |
Select Timestamp , Readings | ConvertTo-Html -Fragment

#Service Application Pool Status
#Returns the specified Internet Information Services (IIS) application pool. NOTE, these are the SERVICE app pools, NOT WEB app pools.
$SAppPool = Get-SPServiceApplicationPool | Select Name , ProcessAccountName , status | ConvertTo-Html -Fragment

#Service Application Status
$ServiceAppplication = Get-SPServiceApplication | Select DisplayName , ApplicationVersion , Status , NeedsUpgrade | Convertto-Html -Fragment

#Service Application Proxy Status
$ApplicationProxy = Get-SPServiceApplicationProxy | Select TypeName , Status , NeedsUpgrade | ConvertTo-Html -Fragment

#Search Administration Component Status
$SPSearchAdminComponent = Get-SPEnterpriseSearchAdministrationComponent -SearchApplication "Search Service Application" | Select Servername , Initialized | ConvertTo-Html -Fragment

#Search Scope Status
$SearchScope = Get-SPEnterpriseSearchQueryScope -SearchApplication "Search Service Application" | Select Name , Count | ConvertTo-Html -Fragment

# WARNING: The command 'Get-SPEnterpriseSearchQueryScope' is obsolete. Search Scopes are obsolete, use Result Sources instead.
# The below code is maybe not an exact alternative to search scope, but at least something...
# Source: https://docs.microsoft.com/en-us/powershell/module/sharepoint-server/get-spenterprisesearchresultsource?view=sharepoint-ps
# Still using Get-SPEnterpriseSearchQueryScope though, as the output from Get-SPEnterpriseSearchServiceApplication/Get-SPEnterpriseSearchResultSource is not exactly what I want.

#$ssa = Get-SPEnterpriseSearchServiceApplication -Identity 'Search Service Application'
#$owner = Get-SPEnterpriseSearchOwner -Level SSA

#$SearchScope = Get-SPEnterpriseSearchResultSource -SearchApplication $ssa -Owner $owner | select Name, Description, Active | ConvertTo-Html -Fragment

#Search Query Topology Status
#Using Get-SPEnterpriseSearchTopology instead of Get-SPEnterpriseSearchQueryTopology
$SPQueryTopology = Get-SPEnterpriseSearchTopology -SearchApplication "Search Service Application" | select TopologyId , State | ConvertTo-Html -Fragment

# $SPQueryTopology = Get-SPEnterpriseSearchQueryTopology -SearchApplication "Search Service Application" | select ID, State | ConvertTo-Html -Fragment
# The term 'Get-SPEnterpriseSearchQueryTopology' is not recognized as the name of a cmdlet, function, script file, or operable program.
# ID not in use, using TopologyId instead (with Get-SPEnterpriseSearchTopology).

#Content Sources Status with Crawl Log Counts
$SPContentSource = Get-SPEnterpriseSearchCrawlContentSource -SearchApplication "Search Service Application" |
Select Name , SuccessCount , CrawlStatus, LevelHighErrorCount, ErrorCount, DeleteCount, WarningCount | ConvertTo-Html -Fragment

#Crawl Component Status
#$SPCrawlComponent = Get-SPEnterpriseSearchCrawlComponent -SearchApplication "Search Service Application" -CrawlTopology "0e49b522-4b7b-44d5-bc77-99b53e1c9f03"| Select ServerName , State , DesiredState , IndexLocation | ConvertTo-Html -Fragment
$SPCrawlComponent = Get-SPEnterpriseSearchStatus -SearchApplication "Search Service Application" | Select Name , State | ConvertTo-Html -Fragment
# OLD COMMAND: Get-SPEnterpriseSearchServiceApplication | Get-SPEnterpriseSearchTopology
# Not entirely same results as for SharePoint 2010, but it'll do.

#Content Database Status
$DBCheck = Get-SPDatabase | Select Name , Status , NeedsUpgrade , @{Name="size(GB)";Expression={"{0:N1}" -f($_.Disksizerequired/1gb)}} | ConvertTo-Html -Fragment

#Failed Timer Jobs, Last 7 days
$f = Get-SPFarm
$ts = $f.TimerService
$jobs = $ts.JobHistoryEntries | ?{$_.Status -eq "Failed" -and $_.StartTime -gt ((get-date).AddDays(-7))} 
#$failedJobs=$jobs.Count      //.Count seems to do nothing. This one does something on the other hand:
$failedjobs = $jobs | select StartTime,JobDefinitionTitle,Status,ErrorMessage | ConvertTo-Html -Fragment

#SharePoint Solution Status
#$SPSolutions = Get-SPSolution | Select Name , Deployed , Status | ConvertTo-Html -Fragment

#=======================#
# Table for html output #
#=======================#

$legendStat="The checks are considered 'Green' in terms of RAG Status when status values are one of the following: <br>
<b>OK, Online, Ready, Idle, CrawlStarting, Active, True</b>"

ConvertTo-Html -Body "
<font color = brown><H2><B><br>Server Report:</B></H2></font>
<font color = blue><H4><B>Memory Usage</B></H4></font>$Memory
<font color = blue><H4><B>Disk Usage</B></H4></font>$disk
<font color = blue><H4><B>Uptime</B></H4></font>$Farmuptime
<font color = blue><H4><B>SharePoint Server Core Services Status</B></H4></font>$coreservices

<font color = brown><H2><B><br>Application Report:</B></H2></font>
<font color = blue><H4><B>Note :</B></H4></font>$legendStat
<font color = blue><H4><B>Farm Status</B></H4></font>$SPFarm
<font color = blue><H4><B>Web Application POOL Status</B></H4>$WAppPool
<font color = blue><H4><B>Web Application Status</B></H4>$WebApplication
<font color = blue><H4><B>IIS: Current number of active connections</B></H4></font>$IIS1
<font color = blue><H4><B>Service Application POOL Status</B></H4>$SAppPool
<font color = blue><H4><B>Service Application Status</B></H4>$ServiceAppplication
<font color = blue><H4><B>Service Application Proxy Status</B></H4>$ApplicationProxy
<font color = blue><H4><B>Search Administration Component Status</B></H4></font>$SPSearchAdminComponent
<font color = blue><H4><B>Search Scope Status</B></H4></font>$SearchScope
<font color = blue><H4><B>Search Query Topology Status</B></H4></font>$SPQueryTopology
<font color = blue><H4><B>Content Sources Status with Crawl Log Counts</B></H4></font>$SPContentSource
<font color = blue><H4><B>Crawl Component Status</B></H4>$SPCrawlComponent
<font color = blue><H4><B>Content Database Status</B></H4></font>$DBCheck
<font color = blue><H4><B>Failed Timer Jobs, Last 7 days</B></H4></font>$failedJobs
" -Title "SharePoint Farm Health Check Report" -head $head | Out-File C:\software\scripts\ServerAndApplicationReport.html

#Document Inventory Module
#Document Inventory Report
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
function Get-DocumentLibraryInventory([string]$siteUrl) {
$site = New-Object Microsoft.SharePoint.SPSite $siteUrl
foreach ($web in $site.AllWebs) {
foreach ($list in $web.Lists) {
if ($list.BaseType -ne "DocumentLibrary") {continue}
foreach ($item in $list.Items) {
$data = @{
                                "Site" = $site.Url
                                "Web" = $web.Url
                                "list" = $list.Title
                                "Item ID" = $item.ID
                                "Item URL" = $item.Url
                                "Item Title" = $item.Title
                                "Item Created" = $item["Created"]
                                "Item Modified" = $item["Modified"]
                                "File Size" = $item.File.Length/1KB
}
New-Object PSObject -Property $data
}
}
$web.Dispose();
}
$site.Dispose()
}

#=================================#
# Send email to SharePoint Admins #
#=================================#

Function logstamp {
$now=get-Date
$yr=$now.Year.ToString()
$mo=$now.Month.ToString()
$dy=$now.Day.ToString()
$hr=$now.Hour.ToString()
$mi=$now.Minute.ToString()
if ($mo.length -lt 2) {
$mo="0"+$mo #pad single digit months with leading zero
}
if ($dy.length -lt 2) {
$dy="0"+$dy #pad single digit day with leading zero
}
if ($hr.length -lt 2) {
$hr="0"+$hr #pad single digit hour with leading zero
}
if ($mi.length -lt 2) {
$mi="0"+$mi #pad single digit minute with leading zero
}
echo $dy-$mo-$yr
}
$mailDate=logstamp

$To =  "xxxx@xxxx.fi"
$Cc = "xxxxx.xxxxx@xxxx.fi"
$From = "xxxxxxxxxxxx@xxxx.fi"
$SMTP = "xxxx.xxxx.fi"

$Subject = "[SharePoint] Server and Application Report - $mailDate"
$Report = Get-Content C:\software\scripts\ServerAndApplicationReport.html

Send-MailMessage -To $To -SmtpServer $SMTP -From $From -Subject $Subject -BodyAsHtml "$Report" -Cc $Cc

 

 

The script itself is located in c:\software\scripts on the server, and is ran from task scheduler once a week. It produces the report shown in the first screenshot.

The weekly run-command is available in the the screenshot below.

sharepoint_server_report_task_scheduler

 

Side note:

You can also get SharePoint’s own Health Analyzer Alerts from PowerShell (should you have the need to):

http://itgroove.net/mccalec/2015/03/13/getting-current-health-analyzer-alerts-with-powershell/

 

And that’s all there is to it – pretty short and effective post this time around 🙂 Enjoy!

Content Database upgrade from SharePoint 2013 to SharePoint 2016 including tweaks and problem solving

As promised, here’s the follow-up blog post to Installing and configuring SharePoint 2016 on-prem (with a combination of PowerShell and Configuration Wizards). Now that SharePoint is up ‘n running with the needed services, it’s time to think about the content. The team site created in the previous blog post had no content, so now we need to import the old content (database) from SharePoint 2013. This is done by upgrading the content database from SharePoint 2013 to SharePoint 2016. Part of the upgrade process happen on the SQL server, as you have to move the existing content database (file) from the old SQL server to the new one. At least this was the case for us, as we also deployed a brand new SQL server (2016) together with SharePoint 2016.

After the content database upgrade process is done, I’ll discuss how to check the status of the upgrade and how to remove old MySites. I’ll also cover Health Analyzer/Timer Job/Event Viewer problem solving, but for now, let’s focus on the content database upgrade. I’m yet again a huge fan of Shane Young’s videos, and in this specific case the video Content database upgrade from SharePoint 2013 to SharePoint 2016.

If you prefer printed text instead of videos, here are a couple of links to get you going:

https://docs.microsoft.com/en-us/SharePoint/upgrade-and-update/overview-of-the-upgrade-process
https://docs.microsoft.com/en-us/SharePoint/upgrade-and-update/create-the-sharepoint-server-2016-farm-for-a-database-attach-upgrade
https://docs.microsoft.com/en-us/SharePoint/administration/move-all-databases
https://www.advaiya.com/blog/how-to-migrate-sharepoint-server-2013-to-sharepoint-server-2016/

 

Here are my own (video)steps recapped:

  • Had a look at our current (old) SharePoint Server to check the content database in use (CA –> Application Management –> Manage content databases):

          sp2016_cdb_view_current_cdbs

  • Went to the old SQL server and looked up the same database (note the ugly DB names btw 🙂 )
    • Right clicked the DB and chose “Tasks –> Back Up…”:

                sp2016_cdb_view_current_cdbs_from_old_sql_management_studio

 

  • On the NEW SQL server, I right clicked on Databases -> Restore Database.
    • I chose the previously backed up database, which was located in c:\backups_from_vasasql:

                sp2016_cdb_restore_cdb_in_sql_management_studio

    • I renamed the database during restore:

                sp2016_cdb_restore_cdb_in_sql_management_studio_choose_name_to_restore

    • I also checked the “Files” tab
      • Nothing to change in my case. The file names were already in the correct format (wss_content_2013.mdf and wss_content_2013_log.LDF).
      • Clicked OK and waited for the restore.
      • Done!

 

  • After a successful restore, I had a look at the database (note the nice DB names btw 🙂 ):

          sp2016_cdb_restored_on_the_new_sql_server

  • I gave the SP_Install2016 account permissions to this newly imported database:
    • Security –> Logins
    • SP_Install2016, Properties
    • Went to the page/tab “User Mapping”
      • put a tick in “WSS_Content2013” and “db_owner”

                sp2016_cdb_access_rights_to_imported_cdb

    • Now moving over to the SharePoint server(s).

 

On the old SharePoint server…

  • checked which managed paths are defined.
    • SP CA –> Manage Web Applications –> Managed Paths:

              sp2016_cdb_check_current_managed_paths

    • my and my/personal are the managed paths that were created from the old “next, next, next-installation”. Not nice. However, I did create these same managed paths on the new server so the content database will have same “layout” after the migration.

 

On the new SharePoint server…

  • I added the above paths on the new server.
  • We now have to delete the current (empty) content database to avoid conflicts when importing the old one.
    • Go to SP CA –> Application Management –> Databases –> Manage content databases
    • Click on the database name (and check that the portal web application is selected).
    • Scroll down until you find “remove content database”

             sp2016_cdb_remove_content_db

    • Click OK to remove it.
  • The content database list is now empty:

            sp2016_cdb_content_databases_overview_empty_list_after_deletion

 

  • It’s now time to import the migrated content database.
    • Shane will actually test the DB before importing it. However, during my tests, I found it easier to first import the database, and do the testing afterwards.Otherwise there are no sites to test against.
    • I actually knew that I had some missing components (intentionally not installed on the new server), and these were easier to deal with after the import.
  • So, we start off by importing/mounting the DB.
    • Start SharePoint 2016 Management Shell as an administrator. Run the following PowerShell command (which I will yet again only provide as a picture):

                sp2016_cdb_mount-spcontentdatabase_powershell

  • It takes some time to complete. In my case it took about 10 minutes. The end result will look something like this:

           sp2016_cdb_mount-spcontentdatabase_powershell_complete

  • I’ll now test the content database for errors. As suspected, the following errors were thrown at me:

           sp2016_cdb_test-spcontentdatabase_powershell_error_overview

                      sp2016_cdb_change_site_collection_administrators

      • another way of adding access to a web application is by changing the policy.
        • SP CA –> Manage web applications –> User policy

                         sp2016_cdb_policy_for_web_application

        • here we give the SP_Install2016 account full permission (which is needed when entering PowerShell commands etc.)
        • I’m not going to copy/paste every single command I ran, so just have a look at the above links and try it yourself.
  • We’re now ready to have a look at the upgrade status.

 

 

(Content database) Upgrade Status

Case: Everything seemed to be working fine when/after you upgraded the content database, and you saw no obvious errors during the upgrade. Still, when looking in SP CA –> Upgrade and Migration –> Check upgrade status, you noticed that something went wrong/wasn’t upgraded. This was at least the case for us:

sp2016_cdb_content_databases_upgrade_status

There’s both a Failed and a Succeeded run in the screenshot, as I managed to fix the problem. The problem was luckily quite easily fixed with hints from:

http://www.sharepointdiary.com/2014/02/upgrade-available-server-status-after-sharepoint-patching.html (just hints, not the actual solution)
https://support.microsoft.com/en-us/help/3157397/upgrade-available-status-in-new-sharepoint-server-2016-farm (solution)

All “symptoms” were true in the second link, and here’s just the first symptom checked:

SP CA –> System Settings –> Servers –> Manage servers in this farm:

sp2016_cdb_configuration_database_upgrade_status

Yes, an upgrade was available.

 

I double checked the same thing using PowerShell:

Get-SPDatabase | select Name, Needs*

The output looked something like this (I’ve cut out the names that didn’t need an upgrade):

Name                                                 NeedsUpgrade        NeedsUpgradeIncludeChildren
——-                                                 ——————         ————————————–
Secure_Store_Service_DB                True                        True
SharePoint_Admin_Content_DB        True                        True
Managed_Metadata_Service_DB       True                        True

 

I also ran stsadm -o localupgradestatus  to get a summary::

[3] content database(s) encountered.
[1] content database(s) still need upgrade or cannot be upgraded.
[20] site collection(s) are contained in the content databases.
[0] site collection(s) still need upgrade.
[32] other objects encountered, [2] of them still need upgrade or cannot be upgraded.

 

As said, all symptoms in the second link were true, so I applied the “fix”:

To resolve this issue, use one of the following methods:

1. Run the following in the SharePoint 2016 Management Shell to upgrade the compoments in the Central Administration database:

Get-SPWebApplication <Central Administration URL> | Get-SPContentDatabase | Upgrade-SPContentDatabase

sp2016_cdb_upgrade_central_admin_components_powershell1

It took a couple of minutes to complete.

2. Run the SharePoint 2016 Products Configuration Wizard from the command line:

PSConfig.exe -cmd upgrade -inplace b2b -wait -force -cmd applicationcontent -install -cmd installfeatures -cmd secureresources”

sp2016_cdb_upgrade_central_admin_components_powershell2

sp2016_cdb_upgrade_central_admin_components_powershell3

Well, that didn’t go exactly as planned. It was mostly my own mistake though, as I ignored the error I got from the Get-SPFarm | Get-SPPendingUpgradeActions –recursive command 🙂 That said, it was easily fixed with information found at https://docs.microsoft.com/en-us/SharePoint/upgrade-and-update/troubleshoot-database-upgrade-issues-in-sharepoint-2013. The SP_Install2016 account once again needed more permissions on the SQL server:

sp2016_cdb_upgrade_central_admin_components_settings_sql_rights

 

I then ran the above PSConfig.exe command again:

sp2016_cdb_upgrade_central_admin_components_powershell_upgrade_success

Yes, happy days 🙂

 

Further checks:

SharePoint CA -> System Settings -> Manage servers in this farm:

sp2016_cdb_configuration_database_upgrade_status_no_action_required

 

SharePoint CA -> Upgrade and Migration-> Review database status:

sp2016_cdb_manage_databases_upgrade_status_ok

 

via PowerShell:

sp2016_cdb_get-spdatabase_needsupgrade_yes_no_powershell

Everything looks OK!

…and as seen in the first screenshot in this chapter, the “Upgrade Status” is now also “Succeeded”. Happy days, everything is working! I’ll now move over to some tweaking and problem solving instead.

 

 

Removing old MySite Site Collections

During the content database migration/import, a bunch of MySites were also imported. To my knowledge, these were in minimal or no usage at all (and had been auto created). My solution was to remove all the unnecessary MySite site collections from the content database. Overall, it was a quite easy procedure based on information found at:

https://sharepoint.stackexchange.com/questions/111263/powershell-delete-remove-all-site-collection-under-a-spesific-managed-path

The command Get-SPSite “https://host.yourdomain.com/my*” –Limit ALL will list all current MySites (which are located under the default web application). This is the default location for MySites when using setup wizards to install SharePoint. Let me copy/paste some information regarding this dilemma:

“At first  having My Site host configured on the default web application may not be a big issue but in the medium to large farm deployment scenarios, you would like to configure the My Site host on the dedicated web application. This would allow organizations to configure the My Sites taxonomy, topology, security, and storage allocations on the dedicated web application running in its own IIS application pool under dedicated service account”.

Source: https://nikpatel.net/2011/06/23/why-you-shouldnt-use-the-farm-configuration-wizard-to-build-production-sharepoint-2010-farm/

Here’s an example output from the above Get-SPSite command:

sp2016_cdb_get-spsite_mysites

As seen from the screenshot, MySites are scattered all over the default web application. Not cool – it gets very cramped in the default web application. Instead, we want to have more control and put the MySite host on a separate web application, my.domain.com for example. This won’t bloat your default web application, and allows for more flexibility and control. I won’t go through the configuration though, as it’s already covered in the post Installing and configuring SharePoint 2016 on-prem (with a combination of PowerShell and Configuration Wizards). Well, back to removing the MySites from the default web application. It’s also easily done with one PowerShell command. BEFORE running this command however, we need to take a backup of the old MySites, JUST IN CASE. Information regarding backup/restore can be found at:

https://social.msdn.microsoft.com/Forums/sharepoint/en-US/de4c8066-035b-4bd6-b3b4-ef0816db4b7e/move-mysites-to-its-own-web-application?forum=sharepointcustomization
https://community.spiceworks.com/scripts/show/1824-export-spsite-export-a-sharepoint-site-to-file (Remember to change the $RootSite parameter)
https://community.spiceworks.com/scripts/show/1837-import-spsite-import-sharepoint-site (Remember to change the $RootSite parameter)
https://sharepoint.stackexchange.com/questions/116422/moving-site-collection-to-a-different-web-application (wouldn’t work for me…)

I had to modify the script in the first link to get it working as intended:

sp2016_cdb_exporting_mysites_for_backup_script

There’s no parameter named “-Path”, instead you should use “-ExportPath”. The script wouldn’t work as intended with the “+ bak” –thing either, so I just removed it. I had no use for .bak extensions on the mysite backups. (If the .bak-part is left unmodified, the script will just create a new directory named “.bak” inside the c:\backup directory, and store all the separate mysite directories inside that directory. This is not the desired effect).

A quick check in c:\backup shows that everything seem ok:

sp2016_cdb_mysites_backup_overview_in_explorer

All MySites are now backed up. The filenames are in the form “$mysite.Name”, which would be domain_username in our case. Even though everything seemed ok, it wasn’t. The log files inside the directories had errors suggesting that my account needed more permissions. Even with the current account added as a Farm administrator and an administrator on the server itself, it wouldn’t work. After much googling I also tried https://social.technet.microsoft.com/Forums/office/en-US/e70019ef-7b18-4fbb-8825-004db9b7deb8/error-while-exporting-subsite-in-share-point-2013-object-reference-not-set-to-an-instance-of-an?forum=sharepointgeneral without success. I then decided that the current backup data was “enough”. Fingers crossed.

 

Now that we have a backup in place, let’s return to the original PowerShell command:

Get-SPSite “https://host.yourdomain.com/my*” –Limit ALL | Remove-SPSite –Confirm:$false

Be warned! The above command will delete all MySite site collections (from the content database). A quick look from the Central Admin confirm that all MySites are gone:

sp2016_site_collection_list_mysites_removed

Yep, all host.domain.com/my/personal/xxxxxx site collections are gone (even though safely backed up).

All NEW MySites will instead be created in their own web application (my.domain.com), because we configured it this way in my previous blog post. The my.domain.com web application doesn’t include the my and my/personal managed paths, because they aren’t needed (and weren’t manually created). Instead, the my.domain.com web application just include the standard managed paths (/personal and /sites). All new MySites will now automatically be created in my.domain.com/personal/username. Good! If I ever want/need to migrate stuff from the old MySites (I doubt it), it’s doable with the backup/restore explained in the links earlier. This is good enough for me. This also quite much summarizes the MySite-part of this blog post. Now let’s look at some other things instead.

 

 

Health Analyzer, Timer Job and Event Viewer problem solving

Health Analyzer problems

Now that you have upgraded the content database and (think) you have a working environment, it’s time to look at the things that aren’t working 🙂 Start by going to SP CA –> Monitoring –> Health Analyzer –> Review problems and solutions. (Your results may vary, and here I’ll present our “health problems”):

sp2016_cdb_health_analyzer_problems

I have the missing server side dependencies problem “under control”, or lets say that at least I know what the problem is. As discussed earlier, the problem is that some features that were enabled on the old server aren’t enabled on this new server (and won’t be). The solution is to manually delete these features etc. from the content database, following information found at:

http://get-spscripts.com/2011/06/removing-features-from-content-database.html
http://get-spscripts.com/2011/06/diagnosing-missingsetupfile-issues-from.html
http://get-spscripts.com/2011/08/diagnose-missingwebpart-and.html

The next “issue” regarding the server farm account is actually a false positive:

sp2016_cdb_health_analyzer_problems2

I’m however following best practices regarding the accounts so I have nothing to worry about. Some more information regarding this false positive:

“Do not change Search Host Controller. For the other services, use one service account for all Service Applications (Instances). There’s no reason to use multiple and will just waste memory/slow down startup time. – Trevor Seward Feb 19 ’17 at 15:06

Thank you for your quick reply, Last query please, Why Host Controller is listed in this case ? and could you please tell me, can I create one service account to run UPS and search? – Mohamed El-Qassas MVP Feb 19 ’17 at 21:35

Because the health rule has no intelligence. It is one of those you will eventually just disable, similar to the 5x RAM free disk space one. Health analyzer rules aren’t hard and fast “best practice”. They’re inflexible and Microsoft isn’t updating them as the product improves; e.g. it will still tell you that 200GB is the limit for content databases, which is far from the truth. – Trevor Seward Feb 19 ’17 at 21:37 “

Source: https://sharepoint.stackexchange.com/questions/208483/the-server-farm-account-should-not-be-used-for-other-services-issue

Furthermore:

SharePoint 2013 Distributed Cache recommendations

The AppFabric “Windows” service should not be touched at all, meaning, don’t change the account manually, nor start/stop the service or change the password for the account. Only exception would be that someone changed the account manually before, and you only want to revert back to the one AppFabric is aware of, but even then it might be better to simply remove and add the server as a Cache Host instead.”

Source: https://blogs.technet.microsoft.com/filipbosmans/2015/10/16/sharepoint-2013-distributed-cache-recommendations/

So, the solution would be to ignore the warning 🙂

 

Timer Job errors

Go to SP CA –> Monitoring –> Timer Jobs –> Check job status –> Job History (menu on the left). Choose “View: failed jobs” in the upper right corner:

sp2016_cdb_timer_job_history_view_errors

Here you’ll find all the failed timer jobs. I’ve been lucky since I’ve got only one failed job 🙂 (Well, I did have a couple more, but they sorted themselves out after a server reboot and some other stuff I can’t remember right now).

Clicking the “Failed” text will get you more information:

sp2016_cdb_timer_job_history_view_errors_in_detail

Unexpected exception in FeedCacheService.IsRepopulationNeeded: Cache cluster is down, restart the cache cluster and Retry

 

This is actually the same information that you’ll find in event viewer:

sp2016_cdb_timer_job_error_in_event_viewer

 

I googled yet again and stumbled upon the following links:

https://blogs.technet.microsoft.com/saantil/2013/03/31/distributed-cache-in-sharepoint-2013-unexpected-exception-in-feedcacheservice-isrepopulationneeded-cache-cluster-is-down-restart-the-cache-cluster-and-retry/
https://social.technet.microsoft.com/Forums/lync/en-US/32e367df-8d79-46a0-9bfe-914f7e8b6492/cache-cluster-is-down-restart-the-cache-cluster-and-retry?forum=sharepointadminprevious
https://ahmedmadany.wordpress.com/2016/02/26/distributed-cache-service-exception-unexpected-exception-in-feedcacheservice-isrepopulationneeded-cache-cluster-is-down-restart-the-cache-cluster-and-retry/
https://guidesharepoint.wordpress.com/2015/05/19/distributed-cache-in-sharepoint-2013-unexpected-exception-in-feedcacheservice-isrepopulationneeded-cache-cluster-is-down-restart-the-cache-cluster-and-retry/

The solution to remove and re-add the Distributed Cache service seemed a bit far fetched. I also checked the third link that mention permissions and accounts, and that part was already OK. I did not try the solution in the fourth link. However, the comment in the second link, “If you can browse your mysite or create a new my site for users it means it’s working. Also check if news feed is updating” made most sense to me. And yes, everything SEEMS to be working so I’ll be ignoring this error for now. Time will tell.

 

Another test would be to run the Troubleshooting Distributed Cache for SharePoint 2013 On Premise script, found at:

https://gallery.technet.microsoft.com/Distributed-Cache-6e8ace1a

and the “manual” for the script found at:

https://blogs.technet.microsoft.com/filipbosmans/2015/09/07/how-to-check-for-issues-with-distributed-cache-and-the-script/

So, just to be sure, I ran the script:

sp2016_cdb_distributed_cache_check_script

After inspecting the output logs (using information from the second link), I found absolutely no errors. This now confirms my theory “time will tell” (if there’s a problem or not).

 

Event Viewer errors

The event viewer didn’t actually contain that much critical stuff of relevance. However, one reoccurring error kept popping up: Distributed COM – Event ID 10016. This is not a critical one, it’s more of a cosmetic fix. Some information regarding this error can be found at:

https://jwcooney.com/2017/08/27/dealing-with-event-id-10016-system-error-logs-with-a-fresh-sharepoint-installation/
https://social.technet.microsoft.com/Forums/windowsserver/en-US/dc4c2b99-72f5-40d8-8b65-6dcdfdb9c084/distributedcom-error-10016-sharepoint-server?forum=winserversecurity
http://www.wictorwilen.se/Post/Fix-the-SharePoint-DCOM-10016-error-on-Windows-Server-2008-R2.aspx
https://soerennielsen.wordpress.com/2007/04/16/fixing-those-pesky-dcom-event-log-error-10016-in-a-sharepoint-farm-environment/
https://blogs.technet.microsoft.com/sharepointcomic/2008/10/28/sharepoint-causes-dcom-errors-event-id-10016/

I myself chose to ignore this error for now. Yes, I’m a naughty boy.

Btw, same thing goes for the following errors:

Another one to ignore is “Session “SharePointDiagnostics” failed to start with the following error: 0xC0000035” – Event ID 2. According to Google, this has to do with the fact that I don’t have the Web Analytics Service installed. See:

https://sharepoint.stackexchange.com/questions/20609/no-health-reports-available-though-features-enabled
http://www.sharepointpals.com/post/SharePoint-2010-Site-Collection-and-Sites-Health-Analyzer-through-SharePoint-Web-Analytics-Services
http://iedaddy.com/2011/05/sharepoint-2010using-web-analytics/

 

Just ignore ‘em all, or at least I did. Do scratch your head even more if you have the energy, or focus on something more interesting/important instead 🙂

 

 

HTTP to HTTPS redirect

As a last tweak, I’ll explain how to configure http to https redirect for the main site. (No one likes to type https in front of url name anyways). This can be done either with SharePoint itself (AAM), or with IIS URL Rewrite Module 2.1. (Both methods require that port 80 is open to the Internet, and obviously that a binding for port 80 exist in IIS). I prefer the URL Rewrite method, so go ahead and grab the latest IIS URL Rewrite Module 2.1 from:

https://www.iis.net/downloads/microsoft/url-rewrite

There are many guides available for redirect, and here’s one random:

http://www.sharepointdiary.com/2015/02/redirect-http-to-https-in-sharepoint-2013.html

For more information about SharePoint AAM vs. IIS Redirect, have a look at:

https://social.technet.microsoft.com/Forums/en-US/c02b2d46-2cd1-43b2-b122-d8324790a753/np-sharepoint-http-to-https-redirect-better-to-do-with-url-rewrite-in-iis-or-aam-in-sp-safe-to?forum=sharepointgeneral

As I said before, I prefer the IIS method and a “clean” AAM.

Test Lab Guide: Windows Server 2016 with Integrated Exchange 2016, SfB Server 2015 and SharePoint 2016

WARNING! This is a pretty long and detailed blog post 🙂

I decided to upgrade (or actually reinstall) my test lab with the most recent version of Windows Server (including the most recent versions of Exchange, SfB Server and SharePoint). All my server virtual machines are built from a clonedGolden Windows Server Image” in VMware workstation, and I also use the same principle for my clients. This way you can deploy new servers/clients very fast, and they will take up much less disk space compared to installing from scratch.

This “custom” TLG is based on:

Windows Server 2012 R2 Test Lab Guide (including the Basic PKI add-on from here) and
Test Lab Guide: Configure an Integrated Exchange, Lync, and SharePoint Test Lab

with my own Exchange add-ons including:

  • A script for configuring the virtual directories
  • Certificate from domain CA
  • Zevenet Load Balancer (formerly known as Zen)
  • A second server (EX2)
  • Another script for copying the virtual directories from an existing server to a new one
  • Database Availability Group (DAG) between EX1 and EX2
  • Moving a user from one database to another

More about these later on.

I’ll start with an overview of the whole TLG, including my own add-ons:

tlg2016_overview

Fig 1. Test Lab overview – a modified picture from the TLG. (I also configured the Internet subnet (131.107.0.0/24), even though not visible in this picture).

 

The whole project started by following the Windows Server 2012 R2 Test Lab Guide. I then added the Basic PKI infrastructure. These Test Lab Guides were actually “translatable” straight from Windows Server 2012 R2 to Windows Server 2016. I got a “Duplicate IP address error” on one of the servers however, but it was easily solved by following: http://support.huawei.com/enterprise/en/knowledge/KB1000068724. (I have no idea why I got this error (hasn’t happened before), but then again it doesn’t matter now that it was solved).

I then moved over to the Test Lab Guide: Configure an Integrated Exchange, Lync, and SharePoint Test Lab. Step 1 was already done so I moved over to Step 2 and 3 – Installing and configuring a new server named SQL1. Step 3 includes a link to a separate SQL Server 2012 Test Lab Guide, and this TLG also happen to be more or less translatable straight to SQL Server 2016. So yeah, I actually have no further comments about the installation. Step 4 guides you through the Client2-installation, but there’s really nothing to comment about this installation either (pretty basic stuff).

 

Exchange 2016, EX1

It was now time for the Exchange server, EX1. Note to self: Use at least 6GB ram for the VM or the memory will run out. This installation also has a separate guide:

https://social.technet.microsoft.com/wiki/contents/articles/24277.test-lab-guide-install-exchange-server-2013-on-the-windows-2012-r2-base-configuration.aspx.

It’s fine for the most part, however instead of downloading the evaluation version of Exchange I suggest you download the newest Exchange Server 2016 CU instead. This way you’ll get the newest updates from scratch. And yes, all setup files are included in the CU so you can use it as a “clean install”. The prerequisites for Exchange 2016 (on Windows Server 2016) are a bit different compared to Exchange 2013 (on Windows Server 2012 R2) also. The only thing you need to download and install “separately” is Microsoft Unified Communications Managed API 4.0, Core Runtime 64-bit. There’s no need for Microsoft Knowledge Base article KB3206632 if you have a recent/patched version of Windows Server 2016. After that just copy/paste the PowerShell command from the prerequisites page:

Install-WindowsFeature NET-Framework-45-Features, RPC-over-HTTP-proxy, RSAT-Clustering, RSAT-Clustering-CmdInterface, RSAT-Clustering-Mgmt, RSAT-Clustering-PowerShell, Web-Mgmt-Console, WAS-Process-Model, Web-Asp-Net45, Web-Basic-Auth, Web-Client-Auth, Web-Digest-Auth, Web-Dir-Browsing, Web-Dyn-Compression, Web-Http-Errors, Web-Http-Logging, Web-Http-Redirect, Web-Http-Tracing, Web-ISAPI-Ext, Web-ISAPI-Filter, Web-Lgcy-Mgmt-Console, Web-Metabase, Web-Mgmt-Console, Web-Mgmt-Service, Web-Net-Ext45, Web-Request-Monitor, Web-Server, Web-Stat-Compression, Web-Static-Content, Web-Windows-Auth, Web-WMI, Windows-Identity-Foundation, RSAT-ADDS

Then run setup.exe and install Exchange. Use the default options. After completion, follow Step 6: Demonstrate EX1 as an email server in the Exchange TLG. I did NOT try to send an email message from Chris to Janet at this stage though, as I wanted to try this after the Load Balancer was installed. But for now, happy days, Exchange installed!

 

Script for configuring the virtual directories

Usually when installing an Exchange server you change the virtual directories/namespace to something other than the server hostname (default). This namespace is the same name that should be included in the certificate. (I didn’t like the fact that this TLG use self-signed certificates so I added my own subchapter about getting a certificate from a domain CA, see next chapter). In a production environment you should plan the namespace and certificate prior to installation, but in this TLG it doesn’t matter that much. I decided to go with the namespace “exchange.corp.contoso.com”. (Autodiscover (DNS url) should also be included in the certificate (request), which it is by default). Anyhow, I first added the mentioned A records (exchange and autodiscover) to DNS. I pointed them to 10.0.0.11 at this stage (but that will change after the Load Balancer installation). I then changed the virtual directories according to the above plan. For this I used a nice script found from:

https://gallery.technet.microsoft.com/office/Set-all-virtual-directories-f4ec71d3

This script is very nice. The only thing that got me worried was the fact that it tried to change the PowerShell virtual directory. Afaik you shouldn’t change that. Anyway, no big deal, I just answered “no” (seen in screenshot) when the script asked me to change this. Here are a couple of screenshots from the script in action:

tlg2016_set_allvdirs_script1

Fig 2. set-allvdirs.ps1 script

tlg2016_set_allvdirs_script2

Fig 3. set-allvdirs.ps1 script, continued

 

Certificate from domain CA

After all the virtual directories were set, it was time to get a new certificate which reflect the above changes. I headed over to trusty practical 365 to refresh my memory. This time I used EAC when requesting a new certificate. I changed the domains to reflect my newly configured environment. I added exchange.corp.contoso.com and autodiscover.corp.contoso.com and removed all the other hostnames. The other options were pretty basic so nothing special there. I then saved the certificate request and copied it over to my domain CA. However, when I tried to process the certificate request on the CA I was greeted with an error message:

tlg2016_cert_req_error_from_ca

Fig 4. Certificate Request Processor error

A bit of investigation led me to the following url: http://mytechweblog.blogspot.fi/2012/11/the-request-contains-no-certificate.html, which had a solution:

“certreq -submit -attrib “CertificateTemplate: WebServer” WebServerCertReq.txt”

tlg2016_cert_req_manual_submit

Fig 5. Certification request with manual submit.

This solution worked for me, nice! I then saved the .crt file and imported it into Exchange from the same place in EAC where I made the request. However, shortly after this I noticed that EAC and OWA still gave certificate errors. This was strange, but then again nothing new. I had a look in IIS/Bindings, and surely the wrong certificate had been assigned. I corrected this so the newly requested certificate was in use:

tlg2016_cert_assignment_from_IIS_on_ex1

Fig 6. Exchange certificate from domain CA.

 

Zevenet Load Balancer

It was now time to install the Zevenet Load Balancer. The reason for installing the Load Balancer at this stage had to do with the fact that I had now preconfigured all the Exchange virtual directories + autodiscover in DNS. This also meant that it’ll be very easy to point DNS at the Load Balancer instead of the Exchange server/CAS further down the road.

I headed over to https://www.zevenet.com/products/community/ and downloaded the newest version. I installed it following my own old blog post. The main difference this time was that I didn’t bother to use clustered servers. (I already know that it works and we’re using clustered LB’s in production). After installation I did the initial configuration:

tlg2016_zen_new_virtual_network_interface

Fig 7. New virtual network interface. The new VIP was set to 10.0.0.61 (the server IP is 10.0.0.60).

 

I then created a new farm, which listens on port 443 on the newly created virtual network interface IP (VIP):

tlg2016_zen_new_farm

Fig 8. New Farm

 

After this I edited the farm and configured the “real IP”:

tlg2016_zen_edit_real_ip_servers_configuration

Fig 9. Real IP’s. In my case, 10.0.0.11 is the “real” IP for EX1.

 

I then converted the Exchange certificate (in a Linux VM) for use within Zevenet LB:

openssl pkcs12 -in file.pfx -out file.pem -nodes

Source: https://stackoverflow.com/questions/15413646/converting-pfx-to-pem-using-openssl

 

It was then time to import it into Zevenet LB:

tlg2016_zen_manage_certificates

Fig 10. Certificate imported

 

After this I made changes in DNS so that all traffic would go through the Load Balancer:

tlg2016_zen_dns_edited

Fig 11. Editing DNS.

 

Now it was finally time to check that Outlook was working correctly from client1 (or 2):

tlg2016_outlook_connection_status_through_zen

Fig 12. Outlook Connection Status

Well yes, it was. Perfect! 🙂

That quite much summarizes the Load Balancer part. Now moving over to the installation of the second Exchange server, EX2.

 

 

Exchange 2016 – second server, EX2

Now that everything was working as it should with EX1, it was time to add another exchange server to the environment. There were no special notes about this installation, I just followed the same guide as with the first one. One thing that was different however, was the script. I now used a script that could automatically copy the virtual directories from an existing Exchange server during deployment. The script can be found at:

http://www.expta.com/2016/07/new-set-autodiscoverscp-v2-script-is-on.html

I’ll copy/paste some information:

The script is designed to be run during installation. Normally, you would run this script from an existing Exchange server of the same version while the new server is being installed.

That sounded almost too good to be true and I had to try it. That said, I had a test-run from EX1 while EX2 was installing:

tlg2016_set_autodiscover_scp_script1

Fig 13. Set-Autodiscover.ps1 script. Looks promising…

…but it wasn’t:

tlg2016_set_autodiscover_scp_script2

Fig 14. Can’t set the virtual directories.

I had the script running during the whole installation of EX2, but no luck. I suspected that it would be better running the script immediately after the installation instead. That said, I had a go just after the finished installation of EX2:

tlg2016_set_autodiscover_scp_script3

Fig 15. Running the script immediately after the EX2 installation.

Yes, much better this time. All the virt dirs were set within a couple of seconds, and I’d say this “lag” would be fine for a production environment as well. I would also like to “thank” the script for reminding me to install a certificate on this second server. That said, I opened up EAC and chose the new EX2 server from the pull-down menu under certificates. I then chose “import” and used the same certificate I made for EX1. It got imported nicely:

tlg2016_imported_cert_ex2

Fig 16. Imported domain certificate

 

Be sure to enable all needed services on the newly imported certificate also:

tlg2016_setting_services_on_imported_certificate_ex2

Fig 17. Overwriting existing SMTP certificate. At the same time I also chose to enable the IIS, POP and IMAP services.

 

Checking certificates from EMS:

tlg2016_checking_certs_from_ems

Fig 18. Checking active certificates. Looks good!

 

…and while you’re at it, check that OWA won’t give you certificate errors:

tlg2016_checking_that_cert_ís_ok_from_owa

Fig 19. OWA

It doesn’t. All good! (The new certificate wasn’t yet active In Fig 16, therefore the status url bar is red).

Only thing left to do now was to add this second IP (10.0.0.12) to “real servers” in Zevenet Load Balancer. After this change, the “dual exchange server setup” was ready for use.

 

DAG

With both EX1 and EX2 up ‘n running, it was time to configure a Database Availability Group (DAG) between the servers. I’ve done this many times before, and I’ve always used the same guide whether it’s for Exchange 2013 or Exchange 2016. The guide I’ve used is:

https://practical365.com/exchange-server/installing-an-exchange-server-2013-database-availability-group/

It’s very straight forward without any extra bs. Some notes:

  • I’m using the “SP1 (APP1)” server as the witness server.
  • I pre-staged a computer account in AD named “EXDAG”
  • I did not configure a dedicated replication network. (Overkill for a test lab).
  • I did not move the Default Mailbox Databases from the default folder path onto storage volumes dedicated to databases and transaction log files. (Again, a little overkill for a test lab).

tlg2016_manage_db_availability_group_membership

Fig 20. Manage Database Availability Membership

 

After this step was done I configured database copies following yet another good (and familiar) follow-up guide from the same series:

https://practical365.com/exchange-server/exchange-2013-dag-database-copies/

I’ve got no additional comments about the database copies, they work just as intended/written in the guide 🙂 Below you’ll find some related screenshots:

tlg2016_add_mailbox_db_copy

Fig 21. Add Mailbox Database Copy

 

tlg2016_databases_overview_after_db_copy_and_dag

Fig 22. Database overview with database copies.

 

Moving users

I moved the user “Janet” from the original database on EX1 over to the database on EX2. This way I “spread out” my (two 🙂 ) users so their mailboxes are situated on different servers. This is good for failover testing and so forth.

tlg2016_move_user_janet_to_ex2

Fig 23. Moving Janet to another database (server).

 

tlg2016_move_user_janet_to_ex2_2

Fig 24. Moving Janet to another database (server), continued.

 

The above steps now completes the whole Exchange-part of the TLG.

 

 

Skype for Business Server 2015, LYNC1

It was now time to move over to the Lync-part of the TLG. The first change was actually the software itself – I’m installing Skype for Business Server 2015 instead of Lync Server 2013. As with other software in this lab, the prerequisites are way different for SfB Server compared to Lync Server. I used a combination of

https://blogs.perficient.com/microsoft/2017/08/skype-for-business-how-to-install-on-windows-server-2016/ and
http://www.garethjones294.com/install-skype-for-business-server-2015-on-windows-server-2016-step-by-step/

as a base for my deployment. Some additional notes:

  • Note to self: Use at least 3GB ram for the VM.
  • (Newest) Cumulative Update has to be installed, otherwise SfB Server won’t work at all on Windows Server 2016.
  • As I installed SfB Server in an isolated network (no internet access), I also had to define the source (which is the Windows Server 2016 DVD) in the PowerShell prerequisite command:

          tlg2016_sfb_install_prereq_from_powershell

          Fig 25. Prerequisites installation for SfB Server 2015 on Windows Server 2016.

 

I then continued following the TLG guide again, and moved over to the chapter “To prepare Active Directory”. Some notes:

  • Installed newest version of offline-Silverlight manually.
  • Chose not to check for updates.
  • Added the DNS SRV records, but they didn’t work when I tested them (probably outdated info in the TLG). This was no big deal, as lyncdiscoverinternal can be used instead for example. You could also Google for updated information, but I didn’t feel it was necessary for this TLG.
  • Everything went fine until “21. From the Topology Builder Action menu, select Publish Topology.” I was greeted with:

          tlg2016_sfb_publishing_topology

          Fig 26. Publishing Topology error.

          tlg2016_sfb_publishing_topology_error

          Fig 27. Publishing Topology error, continued

Well, after some investigation (googling), it turned out this just had to do with UAC: http://terenceluk.blogspot.fi/2013/03/publishing-new-lync-server-2013.html. Surely, after running the deployment wizard again as an administrator (run as), it worked!

 

I now moved over to the “To install Lync Server 2013 core components” -part of the TLG. Notes:

  • I was only running step 1 and 2 at this stage.
  • The IIS URL Rewrite Module problem was well known, I’ve even blogged about it.
  • After step 2 was done, it was time to install the newest CU for SfB, otherwise SfB Server won’t run at all on Windows Server 2016.
    • Remember to run the SfB Management Shell as an Administrator.
    • Everything went smoothly with the CU installation!
  • I moved over to Step 3 – Deployment Wizard/SfB Server 2015 Core Components.
    • Everything went fine!
  • Step 4 was different for SfB compared to Lync. You can’t start services from the Deployment Wizard in SfB Server 2015.
    • Instead, you start them from the SfB Management Shell with the command “Start-CsWindowsService
    • The command didn’t run as planned though:

                tlg2016_sfb_start-cswindowsservices_error

                 Fig 28. SfB Server 2015 Deployment Log.

    • I tried to manually start the “Skype for Business Front-end” service from “Services” in Windows.
      • Did not work either, got stuck in “starting…”
      • Tried old school method and rebooted the server.
        • Worked, all services were now up ‘n running after reboot 🙂
  • I moved over to the “To enable users in the Lync Server Control Panel”-part of the TLG and enabled the users.
  • Yep, all done, working! 🙂

 

 

SharePoint 2016, SP1

SharePoint was the last (and the “easiest”) software candidate on the list. Yet again the prerequisites were different compared to the 2013 version of the TLG. My notes:

  • SP1 is the server name, not Service Pack 1 🙂
  • I tried various offline methods for the prerequisite installation. What a headache. Spare yourself the pain and DO NOT try to install the prerequisites without an active internet connection. I repeat, DO NOT try it.
  • I then installed the prerequisites with “Install software prerequisites” from default.hta. Everything went smoothly.
  • I continued following the TLG and the “To prepare DC1 and SQL1 –part. Nothing to add or comment here.
  • I continued following the TLG and the “To install SharePoint Server 2013” –part. Nothing to add or comment here.
  • Happy days, SP1 installed!

 

 

Configure integration between EX1, LYNC1, and SP1

As a last step in this TLG, I configured server integration between the servers. I would advise you to stay away from the TLG script and use newer information instead. The script has failed me before, and surely it failed this time also when I tried it. In other words, skip the script.

As a first step though, check the SP1/APP1 certificate. The TLG tells you to add a https site binding and select the certificate with the name sp1.corp.contoso.com. This won’t work, at least not for me (never has). Instead, when creating the new https binding, choose the certificate that has been issued to the SP1/APP1 server (never mind the confusing “friendly” name):

tlg2016_sp_checking_cert

Fig 29. Checking SSL certificate in SharePoint/IIS

I got a warning about the certificate already being used for the Default Web Site, but this can be ignored (at least in this TLG).

 

Now we’re ready to move over to some “fresh” information about integration. For starters, have a look at:

Exchange <-> SfB: http://lyncdude.com/2015/10/06/the-complete-skype-for-business-exchange-2016-integration-guide-part-i/index.html
Exchange –> SharePoint and SfB: https://technet.microsoft.com/en-us/library/jj649094(v=exchg.160).aspx
SfB –> SharePoint: https://technet.microsoft.com/en-us/library/jj204975.aspx
SharePoint –> Exchange: https://technet.microsoft.com/en-us/library/jj655399.aspx
SharePoint –> SfB: https://technet.microsoft.com/en-us/library/jj670179.aspx

All links are compatible with the 2016 versions also. Here are the results from my own environment:

 

Skype for Business:

tlg2016_oauth_sfb_check_current_cert

Fig 30. Checking current OAuth certificate and OAuth configuration.

tlg2016_oauth_sfb_setting_cs_auth_configuration

Fig 31. Setting OAuth configuration and checking the configuration.

 

tlg2016_oauth_sfb_to_ex

Fig 32. SfB –> Exchange integration

tlg2016_oauth_sfb_to_sp

Fig 33. SfB –> SharePoint integration

 

tlg2016_oauth_sfb_to_ex_and_sp_check

Fig 34. Checking partner applications. Both Exchange and SharePoint are integration partners.

 

Exchange:

tlg2016_oauth_ex_to_sfb

Fig 35. Exchange –> SfB integration

tlg2016_oauth_ex_to_sp

Fig 36. Exchange –> SharePoint integration

 

tlg2016_oauth_ex_to_sfb_and_sp_checking

Fig 37. Checking partner applications. Both SfB (Lync) and SharePoint are integration partners.

 

SharePoint:

tlg2016_oauth_sp_to_ex

Fig 38. SharePoint –> Exchange integration

tlg2016_oauth_sp_to_sfb

Fig 39. SharePoint –> SfB integration

 

tlg2016_oauth_sp_to_ex_and_sfb_checking

Fig 40. Checking partner applications. Both Exchange and SfB are integration partners.

 

The integration chapter above now finalizes this whole TLG. It was a fun project and I hope someone will find this information useful.

Installing SharePoint 2013 in a two-tier topology

Update! This blog post is dated, check out https://sysadminblogger.wordpress.com/2018/05/24/installing-and-configuring-sharepoint-2016-on-prem-with-a-combination-of-powershell-and-configuration-wizards/ instead.

I got the task of installing SharePoint 2013 for a small business. The SharePoint site won’t be used by that many people simultaneously, so the server load will remain quite small. With that in mind I had to figure out a suitable topology. There are many, many sources on the web describing this, so getting information wasn’t a problem. In the end, I decided to go with a two-tier topology. A single-tier would have been sufficient, but It’s nice to have a separate SQL-server which can be used by other applications/servers as well.

“In a two-tier deployment, SharePoint 2013 components and the database are installed on separate servers. This kind of deployment maps to what is called a small farm. The front-end Web servers are on the first tier and the database server is located on the second tier. In the computer industry, the first tier is known as the Web tier. The database server is known as the database tier or database back-end”.

Source: https://technet.microsoft.com/en-us/library/ee667264.aspx

Another useful link:

https://technet.microsoft.com/en-us/library/cc263199.aspx (you’ll find a nice document/pdf describing Streamlined Topologies for SharePoint 2013). The document states that a two-tier farm is sufficient for up to 10.000 users. More than enough in my case.

My installation is actually based on https://captainofsharepoint.wordpress.com/2013/02/27/the-art-of-installing-sharepoint-2013-in-a-3-tier-topology-part-one/, even though I would call this a two-tier topology and not three. The SQL-guide from this post is not used, as it suggest installing every component (which is unnecessary). Shortly said there are only two servers included in my setup, namely:

  • SharePoint 2013 (more about features and roles later in the document)
  • SQL Server 2014 Standard

I won’t go into the hardware details of the servers themselves because it varies so much from deployment to deployment. It’s easy to scale out with more memory or better/faster SAN-disks if you have the need for it in the near future. It’s also a good idea to read the following information before installing: http://sharepointpromag.com/sharepoint-2010/top-10-sharepoint-2010-configuration-mistakes-and-how-fix-them

 

AD Accounts for SharePoint and SQL

My first task was to create the needed service accounts in Active Directory. There’s a very good site describing the needed accounts at http://www.toddklindt.com/blog/Lists/Posts/Post.aspx?ID=391. I only used

  • sp_install (SharePoint installation)
  • sp_farm (SharePoint Farm Account)
  • ​sql_install (SQL server installation account)
  • ​sql_user (SQL user account)

from the list. Later I created an account named sp_srv for running miscellaneous services. This is more than plenty for such a small deployment. You can read more about service accounts here:

SharePoint 2013 Service Accounts Best Practices Explained:
http://absolute-sharepoint.com/2013/01/sharepoint-2013-service-accounts-best-practices-explained.html (I’m using medium security option)

Initial deployment administrative and service accounts in SharePoint 2013:
https://technet.microsoft.com/en-us/library/ee662513.aspx

SharePoint 2013: Service Accounts:
http://social.technet.microsoft.com/wiki/contents/articles/14500.sharepoint-2013-service-accounts.aspx

 

SQL Server 2014

Next on the checklist was the installation of SQL Server 2014. SQL is a requirement for SharePoint so it should be installed before you install SharePoint itself. I decided to go with http://sharepointpromag.com/sql-server-2012/sql-server-2012-sharepoint-2013-database-server-setup as a base for my installation. Before installing, I also suggest reading the following (you can never be too prepared):

A simple install of SQL Server 2012 for SharePoint Server 2013 or 2010:
http://blogs.msmvps.com/shane/2012/09/17/a-simple-install-of-sql-server-2012-for-sharepoint-server-2013-or-2010/

Instruction Guide for Installing SQL Server 2012 SP1 for SharePoint 2013:
http://www.sharepointdoug.com/2013/02/instruction-guide-for-installing-sql.html

Install SharePoint 2013 – Part 4 SQL Server:
https://www.youtube.com/watch?v=JVBmzG0p76M

Service Account Suggestions for SharePoint 2013:
http://www.toddklindt.com/blog/Lists/Posts/Post.aspx?ID=391

“The SQL Guy” Post #15: Best Practices For Using SQL Server Service Accounts:
http://blogs.technet.com/b/canitpro/archive/2012/02/08/the-sql-guy-post-15-best-practices-for-using-sql-server-service-accounts.aspx

 

Security

After doing some homework (reading articles) I came up with the idea of using SQL with a Named Instance (with SQL-aliases for SharePoint) instead of the Default Instance. I also thought of blocking the default SQL port and using a new static one (configured by SQL aliases). All of this to get better security. I buried this idea however, and instead running with the Default Instance following this guide: http://blogs.technet.com/b/rycampbe/archive/2013/10/14/securing-sharepoint-harden-sql-server-in-sharepoint-environments.aspx. (The server itself is already quite well firewalled by a hardware firewall). Some more information regarding the same matter:

Best practices for SQL Server in a SharePoint Server farm:
https://technet.microsoft.com/en-us/library/hh292622.aspx

Blocking the standard SQL Server ports:
https://technet.microsoft.com/en-us/library/cc262849.aspx#PortProtocolService

Configure SQL Server security for SharePoint 2013 environments:
https://technet.microsoft.com/en-us/library/ff607733.aspx

If one ever decide to use SQL aliases, it’s advisable to read the following document: http://blogs.msdn.com/b/sowmyancs/archive/2012/08/06/install-amp-configure-sharepoint-2013-with-sql-client-alias.aspx

I secured the SQL server using “server isolation” instead.

“Server Isolation can be done several different ways, but the end result is the same: configuring the server to only respond to authorized machines.”

Source: http://blogs.technet.com/b/rycampbe/archive/2013/10/14/securing-sharepoint-harden-sql-server-in-sharepoint-environments.aspx

In my environment, I’m only allowing traffic from the soon-to-be installed SharePoint server (using the above method).

 

Installation

With the security taken care of, it’s finally time for installation! Following the guide I mentioned earlier (http://sharepointpromag.com/sql-server-2012/sql-server-2012-sharepoint-2013-database-server-setup), I went through the steps. I got a firewall warning in the setup (Fig 1), but it was easily fixed by poking a hole in the windows firewall (Fig 2).

sql2014_install_firewall_warning

Fig 1. SQL Server 2014 Setup warning

 

sql2014_firewall_opening

Fig 2. Poking a hole in the firewall (Added the SharePoint server IP).

Next step:

  • Enabled Server Feature: .NET Framework 3.5 (needed for SQL server installation)

Continued the setup:

  • SQL Server Feature selection:
    • Database Engine Services
    • Management Tools – Complete
  • That’s it, no extra crap;

“After selecting SQL Server Feature Installation and clicking Next, a list of SQL Server features is displayed, as shown in Figure X. We really need only one SQL Server feature for SharePoint: Database Engine Services. However, I will also install the Management Tools (Complete) feature, which gives you handy tools such as SQL Server Management Studio. As you browse through the list of features, you might be tempted to check more features than you really need. But unless you’re going to use a particular feature immediately, I don’t recommend installing it. If you want to add a feature later, such as SQL Server Reporting Services, you can just run Setup again and add the feature to your existing instance.”

Source (again): http://sharepointpromag.com/sql-server-2012/sql-server-2012-sharepoint-2013-database-server-setup

Server Configuration/Service Accounts:

  • SQL Server Agent and SQL Server Database Engine: sql_user (the AD account created earlier).

Database Engine Configuration/Specify SQL Server Administrators:

  • myadminaccount and sql_install (the AD account created earlier).

I’m using the default installation paths for SQL as this is a small scale installation.

Installation complete!

 

Tweaking

All tweaks are based on the following articles:

http://sharepointpromag.com/sql-server-2012/configure-sql-server-2012-sharepoint-2013
http://sharepointpromag.com/sql-server-2012/fine-tune-your-sql-server-2012-configuration-sharepoint-2013

  • Max degree of parallelism = 1
  • Maximum server memory 3.5GB (out of 4GB)
  • Model Database’s Recovery Model: simple
  • Compressed backups
  • Also adding the sp_install user to SQL, see below:

“To give the sp_install account the permissions it needs, in SSMS navigate to Security, Logins in Object Explorer. Right-click and select New Login. Under General, type the username and make sure you include the domain. Then on the Server Roles page, shown in Figure 3, select the dbcreator and securityadmin check boxes and verify that the public check box is still selected. Then click OK.”

sql_permissions_for_sp_install

Fig 3. Assigning Permissions to the sp_install Account

“Let me offer a few words of advice about setting the sp_install permissions. SharePoint assumes that those three roles, dbcreator, public, and securityadmin, have the default set of permissions in SQL Server. Don’t alter those permissions. I’ve seen DBAs in very secure environments try to lock down these three roles. Doing so will most certainly break SharePoint in crazy and unusual ways. That might not happen right away, and it might not happen to you when you’re using the interface. It could be a monthly timer job that fails, for instance. Also, don’t change any SQL Server permissions that SharePoint sets. SharePoint is very fussy, and if it sets permissions, it really needs them. Because of SharePoint’s rigidity on its SQL Server permissions, I recommend that you put SharePoint in its own SQL Server instance. SharePoint will thank you, and so will your DBAs.”

Source: http://sharepointpromag.com/sql-server-2012/configure-sql-server-2012-sharepoint-2013

That’s it for SQL, moving on to the SharePoint installation.

 

 

SharePoint Server 2013 installation

I’m being a bit lazy now and just copy/pasting information… why rewrite something that someone has already written (well)?

SharePoint Server 2013 checklist:

Before you begin to install and configure SharePoint 2013, do the following:

Source: https://technet.microsoft.com/en-us/library/cc262243.aspx

Everything in order, let’s continue! (Again, the installation is quite much based on https://captainofsharepoint.wordpress.com/2013/02/27/the-art-of-installing-sharepoint-2013-in-a-3-tier-topology-part-one/)

Well, I didn’t get so far. The prerequisite checker failed with the message: Application Server Role, Web Server (IIS) Role: configuration error.

A suggested solution was to install a hotfix from Microsoft; https://support.microsoft.com/en-us/kb/2765260. This didn’t work however, as the fix was only for Windows Server 2012, NOT the R2 version. Next test was to follow a guide from http://blogs.msdn.com/b/fabdulwahab/archive/2013/08/29/sharepoint-2013-installation-and-configuration-issues.aspx:

Steps to fix (Installing .Net Framework 3.5):

  1. Insert the Windows Server 2012 installation image or DVD
  2. Open a command prompt window (run as Administrator) and run the following:
  3. Dism /online /enable-feature /featurename:NetFX3 /All /Source:D:\sources\SxS /LimitAccess

sharepoint_all_prereq_complete

Fig 4. Success! 🙂

 

Continuing with the setup…

sharepoint_install_server_type

Fig 5. Complete installation (production). Using default file locations (because small scale installation).

Done. The SharePoint Configuration Wizard will then run:

sharepoint_products_configuration_wizard1

Fig 6. Create a new farm

 

sharepoint_products_configuration_wizard2

Fig 7. Database settings. Database server and account settings were discussed in the SQL chapter.

 

sharepoint_products_configuration_wizard3

Fig 8. SharePoint Central Administration Web Application

Port 18811 (or whatever SharePoint chooses for you) must be blocked (outside the domain), otherwise the Central Administration URL will be open for anyone on the Internet.

 

sharepoint_products_configuration_wizard4

Fig 9. Completing the configuration wizard

 

sharepoint_products_configuration_wizard5

Fig 10. Configuration successful!

 

Services

There are A LOT of different services running on a SharePoint server. However, in a small scale environment, you’ll probably only need/use a few of these. I took a look at the old server and compared the services running there. Here’s a screenshot of SharePoint 2010 and its active services:

sharepoint_services_on_old_server2

Fig 11. SharePoint 2010 services

From the screenshot we can see that the following services are running:

  • Central Administration
  • SharePoint Foundation incoming E-Mail
  • SharePoint Foundation Web Application
  • SharePoint Foundation Workflow Timer Service

With this in mind, I tried to keep the services at a minimum on the SharePoint 2013 server as well.

I couldn’t find the exact same ones in 2013, but I decided to go with the following:

sharepoint_services

Fig 12. SharePoint 2013 services

  • Search Service Application
  • State Service
  • Usage and Health data collection

 

After SharePoint had configured itself I was greeted with a message that some services are running with the “wrong” accounts (Fig 13).

sharepoint_service_account_warnings

Fig 13. SharePoint Failing Services

The failing services are:

  • SharePoint Central Administration v4 (Application Pool)
  • SPTimerV4(Windows Service) = Farm
  • AppFabricCachingService (Windows Service)

 

My idea was to run the default SharePoint services with the “sp_farm” account. Other services can be run with the “sp_srv” account if/when needed.

Update: It’s not recommended running the Wizard, instead you should manually configure the settings.

 

You change the account settings in SharePoint –> Central Administration –> Configure service accounts. I changed the farm account to “sp_farm”. Everything more or less broke after that 😦 I had to do some googling to get it up running again.

Solution (before changing farm account to sp_farm):

  • Register the account (sp_farm) as a managed account. To change a managed account password go to Central Admin > Security > Configure Managed Accounts (/_admin/MangedAccounts.aspx). Click the Edit icon next to the account whose password you want to change.

           sharepoint_managed_accounts

           Fig 14. Register Managed Account.

  • Go to the Configure Service Accounts page and Select the Farm Account and set the new managed account
  • Reboot the server.

 

Source: https://social.technet.microsoft.com/Forums/office/en-US/8c330449-b9cd-4ed5-adeb-342466a8a59e/central-administration-no-longer-accessible-by-any-account-after-changing-farm-account-in-sharepoint?forum=sharepointadminprevious

Done. SharePoint is now installed 🙂

 

Security

You shouldn’t use http with SharePoint outside your domain. Instead you should use https (http over SSL). Request a certificate for your SharePoint site from a 3rd party certificate issuer (or similar), and then apply the certificate. You could/should also use http redirection (http –> https) and/or Alternate Access Mappings. You can follow these guides for example:

https://www.digicert.com/ssl-certificate-installation-microsoft-sharepoint-2013.htm
http://www.sharepointconfig.com/2010/03/configuring-a-sharepoint-website-to-allow-ssl-connections/
https://griffindocs.wordpress.com/2013/03/20/sharepoint-2013-how-to-add-ssl-to-a-web-application/
http://blogs.msdn.com/b/fabdulwahab/archive/2013/01/21/configure-ssl-for-sharepoint-2013.aspx

http://blogs.msdn.com/b/sharepoint_strategery/archive/2013/05/27/alternate-access-mappings-explained.aspx
http://blog.blksthl.com/2012/12/03/a-guide-to-alternate-access-mappings-basics-in-sharepoint-2013/
https://technet.microsoft.com/en-us/library/cc261814.aspx
https://technet.microsoft.com/en-us/library/cc263208.aspx

https://social.msdn.microsoft.com/Forums/en-US/eaab487a-bc94-4f06-981b-c62711764367/redirect-http-to-https-for-sharepoint-2013
http://www.jppinto.com/2010/03/automatically-redirect-http-requests-to-https-on-iis7-using-url-rewrite-2-0/
http://pcfromdc.blogspot.fi/2013/10/how-to-redirect-from-http-to-https-with.html
http://wellytonian.com/2014/01/sharepoint-http-https-url-redirect/
http://sharepoint.stackexchange.com/questions/64484/http-to-https-redirection-using-aam
http://www.sharepointbitme.com/?p=8