Data Deduplication in Windows 8.1

I got a hot tip from my friend regarding the Data Deduplication feature available (only) in Windows Server 2012 (R2) (Thanks Mats 🙂 ). With some small tricks you’re now able to use this cool feature in Windows 8/8.1. I’m currently using a lot of disk space for virtual machines and iso-files, so for me this is (was) the best thing ever. With deduplication enabled, I’m now able to free up a lot of my precious disk space. Yes A LOT. You’ll love it on your small SSD. So, what is Data Deduplication you may ask. Here’s a short answer to that:

Quote

“Data deduplication involves finding and removing duplication within data without compromising its fidelity or integrity. The goal is to store more data in less space by segmenting files into small variable-sized chunks (32–128 KB), identifying duplicate chunks, and maintaining a single copy of each chunk. Redundant copies of the chunk are replaced by a reference to the single copy. The chunks are compressed and then organized into special container files in the System Volume Information folder.

Source: http://technet.microsoft.com/en-us/library/hh831602.aspx

//Quote

And while you’re at it, also have a look at http://blogs.technet.com/b/filecab/archive/2012/05/21/introduction-to-data-deduplication-in-windows-server-2012.aspx

 

The GOOD…

I have to say that after finding out that Deduplication was doable on windows 8.1, it wasn’t that hard finding information on HOW to do it. I’ll start by linking to the original source as usual, thanks to the author for the guide!

http://weikingteh.wordpress.com/2013/01/15/how-to-enable-data-deduplication-in-windows-8/

Well, not much to say really. I followed the guide (for Win 8.1) and it worked 🙂 Here are some screenshots of the process:

deduplication1

Fig 1. Adding Deduplication packages and enabling the Deduplication feature.

 

deduplication_status

Fig 2. Enabling and starting Deduplication job on the D: –drive. Also checking the status with Get-DedupJob (currently at 0% because the process just started).

 

My D: –drive consist of 2 x Western Digital Black 2TB 7200 RPM drives configured in software raid-0 (stripe). The Deduplication job ran forever the first time (4.5h), but it was well worth the wait – check the before and after screenshots below;

before_deduplication

Fig 3. Before Deduplication (1.54TB free on D: –drive).

 

after_deduplication

  Fig 4. After Deduplication (2.77TB free on D: –drive 🙂 )

 

and the same thing checked with Get-DedupStatus:

get-dedupstatus

Fig 5. Get-DedupStatus

 

As you can see, the space savings are HUGE (1.25TB saved space). All I can say is that I’d recommend Deduplication for everyone. Now the same procedure is waiting for my E: –drive (a 512GB SSD) 🙂 And remember folks, do NOT run/use Data Deduplication on your system drive! You have been warned.

For some more Deduplication PowerShell commands, have a look at: http://technet.microsoft.com/en-us/library/hh831434.aspx

For information about Deduplication and backups, have a look at: https://social.technet.microsoft.com/Forums/windowsserver/en-US/ff22c1f5-f704-4173-9aed-e57ad4d3508d/data-deduplication-and-file-backup?forum=winserverfiles

 

…and the BAD

Well, all of this seemed too good to be true. And for me, apparently it was. I didn’t notice any slowdowns in normal usage, but when I started playing around with my virtual machines things started getting slow. It took FOREVER to load a virtual machine from a suspended state for example. My guess is that Microsoft’s version of Deduplication is best suited for Hyper-V, NOT VMware. (They even have an option for it in the Hyper-V settings). What a shame. For now, it’s not usable and I decided to disable it altogether. Luckily this was an easy (but slooooow) process. It took me about 7-8h to De-Deduplicate my 4TB Raid-0 striped volume. Steps for reversing the deduplication process:

In PowerShell:

Start DedupJob -Type Unoptimization -Volume D:

That’s it. No other command was needed (even though I found some articles saying that I should also disable deduplication and clean up the garbage collection after unoptimization). This was however unnecessary/impossible;

PS C:\> Start-DedupJob -Volume D: -Type GarbageCollection
Start-DedupJob : MSFT_DedupVolume.Volume=’D:’ – HRESULT 0x80565323, The specified volume is not enabled for deduplication.
At line:1 char:1
+ Start-DedupJob -Volume D: -Type GarbageCollection
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : NotSpecified: (MSFT_DedupJob:ROOT/Microsoft/…n/MSFT_DedupJob) [Start-DedupJob], CimException
+ FullyQualifiedErrorId : HRESULT 0x80565323,Start-DedupJob

 

PS C:\> Disable-DedupVolume D:
Disable-DedupVolume : MSFT_DedupVolume.Volume=’D:’ – HRESULT 0x80565323, The specified volume is not enabled for deduplication.
At line:1 char:1
+ Disable-DedupVolume D:
+ ~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo         : NotSpecified: (MSFT_DedupVolume:ROOT/Microsoft/…SFT_DedupVolume) [Disable-DedupVolume], CimException
+ FullyQualifiedErrorId : HRESULT 0x80565323,Disable-DedupVolume

 

Finally I also removed the Deduplication (and File Server) feature from Windows Features (in “Turn Windows features on or off”). After a test run I can confirm that my virtual machines now start and resume more rapidly again.

Sources:

http://nickwhittome.com/2014/10/01/disabling-data-deduplication-on-windows-server-2012r2/
http://jeffwouters.nl/index.php/2012/01/disk-deduplication-in-windows-8-explained-from-a-to-z/
https://translate.google.com/translate?sl=auto&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Fwww.amaxing.de%2Fwindows-server-2012-deduplizierung-der-festplatten-rueckgaengig-machen%2F2014%2F11%2F22%2F&edit-text=

 

Final words

Unfortunately, luck wasn’t completely on my side with Deduplication 😦

However, I’d still recommend deduplication for people running low on disk space (using non-disk intensive stuff). A good place for deduplication would probably be your non-system stash-disk filled with music, photos, videos and so on.

That said, happy deduplication after all 🙂

Advertisements

Epson Interactive Projector

Nowadays everything is supposed to be “Smart” or in the Cloud. That said, we were thinking about buying a SmartBoard for one of our meeting rooms. We had heard both good and bad things about these boards and we didn’t quite know if it was our cup of tea.
We were fortunate enough to get a demonstration of a “Smart-Projector” instead (actually an Interactive Projector). The model was EPSON ultra short throw projector EB-1410Wi. We were pleased with all the functions/features so we decided to get one.

Features:

  • Whiteboard Function – Perfect alternative to copyboards, conventional whiteboards and old fashioned flip charts, this easy-to-use function facilitates “instant meetings” without the need for a PC. After the meeting, you can save, print or email the meeting notes.
  • Save – Load and save image files on USB or network drive.
  • Print – Print your meeting notes wirelessly (peer to peer) or to the network printer.
  • Email – Instantly share your meeting notes via email – to an individual or group address.
  • Capture – Capture a screenshot from other sources (eg PC, DVD, iPad etc) and annotate over it on the whiteboard. From there you can save, print or email the annotated version
  • Split Screen Function – Allows images from 2 input sources to be projected simultaneously.
  • Control Pad for Easy Operation – Neatly designed and easy to install control pad for efficient and intuitive operation. All buttons on the control pad can also be found on the remote control.
  • Remote Interactive Collaboration – Collaborate from multiple remote locations (up to 4).
  • Wireless Projection and Interactivity – Projection and interactive function without the wires (using peer to peer wireless and Epson’s EasyMP Network Projection software).
  • Dual Pen Interactivity – With simultaneous dual pen functionality, you and a colleague can annotate at the same time.
  • No Driver Installation Required – Installation has never been easier (Driver install is required for Mac however).
  • Advanced Networking Capability – Enables users to monitor, control and present from a remote location.
  • Broad Connectivity Including HDMI – HDMI, DisplayPort, USB display, PC Free slideshows, network and wireless connectivity.
  • Lowest Epson Lamp Price Ever Seen – Incredibly low price of only $79.
  • 10W Built-in Speaker – Built in audio with the power to fill the meeting room.

Source: http://www.epson.com.au/products/projector/EB-1410Wi.asp?groupid=70

 

Installation

People were requesting a projector with a function that enables you to save all the notes (made on top of a slide for example) during a presentation. This is possible with the Epson in two ways; either save to a USB stick or to a network drive. The projector can also work as a “normal projector”. We had to buy some equipment and make some modifications to the existing room to get all this working. For starters, the room only had cables for vga. We wanted to use dvi/hdmi as vga is getting old and the quality isn’t that great either. We used the existing wireway and added three new LAN cables (+ extenders). Other equipment include:

  • 3 x 15m Ethernet Cat-5 cable (one for video, one for usb and one for the Buffalo)
  • Aten UCE60 USB-extender
  • Aten VE800 HDMI Extender
  • Existing LAN cable with existing vga extender
  • On the meeting room table: Aten VS481A 4 Port HDMI Switch
  • Buffalo Airstation Nfiniti WZR-HP-AG300H (hidden on top of the roof tiles)

With this done we now have 1 x VGA, 4 x HDMI (with different adapters) and 1 x USB port available on the meeting room table. It’s easier and cheaper to use an Ethernet cable instead of a hdmi/usb cable. It’s also much easier to install it in the wireways. This configuration will work for almost any laptop configuration out there. You can (and you should) use the wireless projection anyway.

That was the hardware part…almost. We also have a Buffalo Airstation Nfiniti WZR-HP-AG300H that has NAS support. This means that you can connect a usb stick or external hard drive to it and use it as a network drive. This way people can save their files to a network share. We have placed the Buffalo (with a connected usb stick) on top of the roof tiles in the meeting room. This way it’s invisible and you are also literally saving your files to the cloud 🙂 (You are requested to join the wireless “projector network” when you enter the meeting room. There are also instructions available on how to access your files from the Buffalo).

Setting up the projector was a bit harder than normal because there are  A LOT of options. Actually the setup itself wasn’t that hard, it was the challenge of memorizing all the stuff you can do with it. Well, that’s our problem. We have written a more compact mini-manual that is available in the room.

Everything went fine except for the network part which turned out to be a bit tricky. We wanted to have the projector connected to the internal Buffalo network so it could connect to the network share. Connecting it to the network itself was no problem, it was the network share-part that got us confused. You can enter a hostname for the projector itself (epson in our case), but you can not connect to a network share with an IP address, only by UNC path. We were a bit confused on how to get the UNC path from the Buffalo. I don’t actually remember how, but in the end I got the idea of looking at the samba configuration on the Buffalo. First we enabled ssh to the Buffalo (this is possible because it’s running dd-wrt by default), and then ssh’ed in to it to look at the configuration. I had a hard time finding the configuration file as it’s not in the “normal place” (/etc/samba/ or similar). A few google attempts later it turned out that the smb.conf was located in /tmp/smb.conf. How logical, don’t you think? Source: http://tips.desipro.de/2010/09/09/dd-wrt-samba-custom-configuration-kong-mod/. Anyways, I had a look at the file:

BusyBox v1.19.4 (2012-06-03 12:59:27 CEST) built-in shell (ash)
Enter ‘help’ for a list of built-in commands.

root@xxx:~# cat /tmp/smb.conf

[global]
netbios name = DD-WRT
server string =
syslog = 10
encrypt passwords = true
obey pam restrictions = yes
preferred master = yes
os level = 20
security = user
mangled names = no
max stat cache size = 64
workgroup =
bind interfaces only = Yes
guest account = nobody
map to guest = Bad User
smb passwd file = /var/samba/smbpasswd
private dir = /var/samba
passdb backend = smbpasswd
log file = /var/smbd.log
max log size = 1000
socket options = TCP_NODELAY
printing = none
load printers = No
usershare allow guests = Yes

[epson]    (This section has been auto entered by the Buffalo itself  after successful connection with the projector)
comment = epson
path = /tmp/mnt/sda_part1
read only = No
guest ok = Yes

force user = root

and there it was; netbios name = DD-WRT.

With this information in place, it was just a matter of entering it in “Settings –> Save –> UNC Address” (\\DD-WRT\epson in our case) on the projector. Now the projector can save files directly to the usb stick on the Buffalo. People can also map the network drive to retrieve their saved files/presentations.

 

Using the projector

The projector can be used in three ways (see Fig 1 below):

  • Whiteboard only, with ability to save scribbling (see Fig 2 below)
  • Projector only
  • Interactive projector (see fig 3 below)

projector

Fig 1. Areas of usage

epson1

Fig 2. Whiteboard

epson2

Fig 3. Interactive projector

As said before, you can also use the projector as you would with a “normal” one. This means connecting vga or hdmi cables to your laptop and project the picture. You don’t have to use the interactive bit at all if you don’t feel like it. However, you can also write stuff on the whiteboard or on top of slides and save your work. When saving you can choose network or usb. You can plug your usb stick into the accompanied Control Pad (fig 4) which is situated next to the whiteboard. If you are using the Easy Interactive Tools (and the usb cable) you can save directly to your computer.

You can also use Network Projection by using Epson’s EasyMP Network Projection software. The software will find the projector if the PC and the projector are on the same network. You can also project from an iPad or iPhone using a downloadable app called iProjection (by Epson). If you install Easy Interactive Tools on a PC or Mac, you can get the “drawing tools” (Fig 2) to your computer instead. This means that you can draw on your computer instead of on the whiteboard (or on both).

epson3 

Fig 4. Control Pad

This was just a short description of all the projector functions. The projector can do just about anything so it’s probably best for all of you to read the manual if you are interested in getting one: https://www.projectorpoint.co.uk/imagelibrary/projectors/epson/eb-1410wi/epson_eb-1410wi-manual.pdf

Windows Updates in Nagios

We’re currently using Nagios as our main monitoring system at the Department. There’s actually no need to change that (even though I tried SCOM 2012). Things that I’ve been missing in our current Nagios setup are notifications about Windows Updates. Well, honestly I haven’t even looked into that specific “problem” before now. That said, I decided to give it a try.

I started by doing some googling and found a nice solution which uses  NSClient++ (http://www.nsclient.org/nscp/) and a script (http://zeldor.biz/2012/02/icinganagios-check-windows-updates/) which checks for updates. I followed the steps with some minor changes:

  • Installed NSClient++ on the Windows Server(s)
  • Edited nsclient.ini (NSC.ini is for older versions):

[/modules]
NRPEServer = 1

[/settings/NRPE/server]
port=5667 (default port wouldn’t work for some reason)
command_timeout=90
allow_arguments=0
use_ssl=1
socket_timeout=90

[/settings/external scripts/scripts]
check_win_updates=cscript.exe //T:90 //NoLogo scripts\\check_windows_updates.wsf /w:1 /c:10

[/settings/default]
; ALLOWED HOSTS – A comaseparated list of allowed hosts. You can use netmasks (/ syntax) or * to create ranges.
allowed hosts = abcdef.abo.fi

     

  • On server side:

commands.cfg:

define command {
    command_name    check_win_updates
    command_line    $USER1$/check_nrpe -H $HOSTADDRESS$ -p 5667 -c check_win_updates -t 120
}

services.cfg:

define service {
    hostgroup_name               check-win-updates
    service_description            Windows Updates
    check_command               check_win_updates
    use                                     generic-service
    check_interval                  2880
}

host-groups.cfg:

define hostgroup {
    hostgroup_name          check-win-updates
    alias                           Windows Updates
    members                 server1,server2 (Lets call the servers server1 and server2 in this example)
}

 

Play around with the timers ( –t) and see what suit your needs. If you want to check other stuff as well (harddisk space, cpu usage and so on) you have to configure a bit more.

Examples:
http://nagios.sourceforge.net/docs/3_0/monitoring-windows.html
http://awaseroot.wordpress.com/2012/11/23/monitoring-windows-with-nagios/

In our case it was enough with Windows Updates checking however.

Here’s a screenshot (Fig 1) from nagstamon running on my Windows 8 client:

nagastamon

Fig 1. nagstamon

There’s currently no information about Windows Updates as all our servers were already updated before the screenshot 🙂 The other information is from different linux servers and printers. Here’s another screenshot (fig 2) from the Nagios web interface:

winupdates

Fig 2. Host information/status detail for host in Nagios

Here you can see that the Windows Update check is running and that no updates are waiting or installing.

So, there you have it – Windows Updates in Nagios.

New Touch Screen system

Our Department is located on two separate floors (4 and 5) in the ICT building. There’s a room map on each floor. This map contains only room numbers and no person names.  See Fig 1.

display1

Fig 1. Room map for the 5:th floor (and some updated touch info Smile )

The problem with this map is that you only have a room number and no idea what person (name) you are looking for.

In the past we’ve had a printed A4 paper next to this room map which included both person name and room number (fig 2 below). Whenever the personnel (list) changed on our webpage, “the system” printed a new paper. This was done automatically by a script in our CMS (Department webpage). The personnel listing includes BOTH room number and person name. The negative side of this system is that someone has to (manually) replace the piece of paper in the corridor every now and then. It tends to be forgotten…

display4

Fig 2. Old school

Anyways, we decided to enter the touch screen area and try out an electronic version of this map with both room numbers and person names. Actually this new system includes more than just room numbers and person names. It includes all the details about a person that is available on the Departments homepage (a picture, an e-mail address and a phone number). We also have a “You are here”-sign and a red dot which mark the persons room location on the map (see Fig 5). As we already have this listing electronically on our webpage, only some small tweaks were needed for the touch layout. Thanks to my colleague for the html / coding part 🙂 Now let’s look the project in more detail.

 

Hardware:

Acer 5600 U, All in One PC.

We had a look at many different brands and models, and this one had the looks. It was also one of the wall mountable models and the price was right. We decided to go with the cheapest model of the 5600 series because even those deliver good performance. We bought one for the fourth floor and one for the fifth floor.

http://us.acer.com/ac/en/US/content/model/DQ.SNPAA.001 

 

Software/tweaks:

  • Windows 8 originally
  • Replaced it with our Windows 7 image (no need for Windows 8 in our case)
  • Tweaked the OS a little bit:
    • Added a local user named “kiosk”. This user logs on automatically when the computer starts (Sysinternals Autologon).
    • Internet Explorer runs at system startup. It runs in “kiosk mode”.
      • This is our main “touch interface”. It’s basically a webpage in full screen.
      • iexplore.exe -k www.yourwebpage.com
    • Disabled some touch functions, for example right click in Internet Explorer. Also disabled pinch zoom.
    • Disabled Bluetooth
    • Disabled USB ports
    • Disabled Wireless interface
    • Changed Power Options/Choose what the power buttons do to “Nothing”. (No accidental power off when people are messing around…)
    • Installed TightVNC Server for easier remote access
    • Enabled Concurrent Sessions so you can RDP into the machine without disrupting the current (kiosk) user http://www.missingremote.com/guide/how-enable-concurrent-sessions-windows-7-service-pack-1-rtm
    • Monitor should switch off during the night and switch on automatically in the morning. (Now done, see Fig 3.)
  • Keyboard and mouse are stored in my room Smile

powersaving

Fig 3. Computer sleep/wake

Steps for sleep/wake:

  • Create two new power plans (Control panel/Power options/Create a power plan):
    • For example “Monitor Always ON” (balanced with Turn off the display and Put the computer to sleep both set at never
    • For example “Monitor OFF” (balanced with Turn off the display put to never and Put the computer to sleep set to 1 minute
  • Use Task Scheduler  to activate the power plans according to your needs. My computer goes to sleep every day at 19.00 and wakes up at 07.15 in the morning (See Fig 3).
    • Use powercfg –list to list existing power schemes
    • Use powercfg -setactive <GUID> to set your default scheme
  • Fig 3 shows my schedules. I have one called “Monitor OFF” and one called “Monitor ON”. They use powercfg –setactive <GUID> to switch between the two schemes.
  • Works as it should… almost. I noticed that the monitor/computer wasn’t switched on in the morning.
  • Had a look at the system logs in Event Viewer. Lots of stuff with source from “Power-Troubleshooter”. Further investigation shows:
    • Sleep Time: ‎2013‎-‎09‎-‎02T02:00:59.919249500Z
      Wake Time: ‎2013‎-‎09‎-‎02T03:06:58.670908100Z

      Wake Source: Device -Realtek PCIe GBE Family Controller

    • Apparently the computer woke up every now and then without me even knowing it. I did some googling and this has to do with the power options of the network card itself. Long story short: have a look at the “only allow magic packet to wake the computer” (Fig 4. below)

Allow this device to wake the computer

           Fig 4. Power Management options for the network adapter

  • In my case, the “only allow a magic packet to wake the computer” was unchecked, which caused the computer to wake up with almost every sent Ethernet packet out there. Not good. Source: http://windows7-issues.blogspot.fi/2011/03/wake-on-lan-wol-for-windows-7-made-easy.html
  • The Monitor OFF sleep schedule works fine. However, the computer didn’t wake up at 07.15. Well, powercfg can’t run if the computer is sleeping, right?
  • Solution: Put a tick in “only allow a magic packet to wake the computer” and have another computer send a “magic packet” to wake this computer.
  • Installed Wake on Lan Command Line utility on on of our servers which sends a wake-up call in the morning.
    • Made a batch file which included two lines to wake up both our touch screen computers (example in the url above)
    • Had our server run a scheduled task with the batch file at 07.14.30. This made the touch screen computers wake up 30 seconds before they changed their power profile to “Always ON”.
  • I now feel like MacGyver but at least it’s working 🙂

Source: http://superuser.com/questions/226974/forcing-windows-to-be-awake-during-certain-hours

So far so good. Seems “secure enough”, no complains either (one week has passed). I probably forgot to add something to the list above though…

 

System in action

Below are some pictures showing the the system in action. Sorry for the crappy quality.

display3 vs. display4

Fig 5. New vs. old. Which one do you prefer? Smile

As you can see below, there’s also a  bit more information available on the start page than just the personnel listing. This is mostly to fill up the page with other (useful) stuff when not displaying a room map. When you click (or perhaps I should say “touch”) a person on the list, the layout will look like the one in Fig 5 above.

 

display2

Fig 6. Main page

The main page (Fig 6) is by default displaying some of the Departments projects and todays lunch in our restaurant. None of the elements (to the right) are clickable as this would open up a new browser window and so on. We only wanted the personnel listing clickable (at least for now) so we try to keep it as tidy as possible. You can use the QR codes to retrieve the information for your mobile phone however.

This is just one way of using the touch screen pc. We’re currently open to new ideas and we are already trying to figure out what we could do with the embedded webcam Smile