Secure Remote Desktop Sharing with VNC on Windows

What is Remote Desktop Sharing? It’s being able to give access to yourself or another user from a PC anywhere in the world. The remote user can use the PC just as if they were sitting in front of it. If a remote user is controlling the PC, the local user can see everything — mouse cursor movement, menu use, text use, and so on — just as if they were using the computer themselves. VNC is the most popular remote desktop sharing application.

VNC is the abbreviated form of Virtual Network Computing.’ It is a system that allows users to remotely manage and share their desktop environment of operating systems. Basically, it is possible to view and control any system via the Internet all over the world that has VNC up and running provided the supposed user is eligible to do so — in other words, has account permission. VNC is quite popular because it is the easiest solution and has a lot of benefits.

Official Logo of the VNC

On the DevHardware Forums (as elsewhere), every now and then we face questions about remote desktop sharing and such. In most cases we’re helping out newcomers by linking VNC software. But the problem in essence is that there is more than one VNC program (client + server). A dilemma occurs when the user can’t decide which one is the "best" (if there even is a "best"). Eventually, the user might have problems through the installation process and configuration.

This is the first of a two part series and its purpose is, in a nutshell, to describe the benefits and usefulness of VNC software, giving a little background on it, analyzing a few of the most popular applications and pointing out their pros and cons. Ultimately I’m going to lead you through very strong encryption methods to secure your VNC connections. Therefore reading this article will get you familiar with VNCs regardless of whether or not you have had any prior experience in this field.

This part will discuss the VNC solutions for Microsoft Windows. The second part will do the same, but for Linux, and its style will be similar to this. Since it will be dealing with open source, the second part will appear on Dev Shed.

{mospagebreak title=Background}

Let’s drift back into the past for a bit. In 1994, the Olivetti & Oracle Research Laboratory located at Cambridge, England successfully developed VNC. It ran on an ultra-thin-client/server that was connected to an ATM-based device. The co-inventors were Andy Harter (project leader) and Tristan Richardson.

Now let me give you a little more background information on the lab. The Olivetti Research Laboratory (ORL) was a research institute founded in 1986 by Andy Hopper and Hermann Hauser. Later on, Professor Hopper left the institute (1988). In 1997 the laboratory became Olivetti & Oracle Research Laboratory.

In January 1999 AT&T bought the research laboratory, but the project and research continued. In April 2002 AT&T announced that this laboratory would be closed; it spelled the end of further research and of this project.

Read more about the past of this research institute at AT&T Laboratories Cambridge – Archive. Check out their technical reports and publications.

Different Versions of VNC

As I previously mentioned there are quite a few VNC programs. Three of the most popular ones are the following: RealVNC, TightVNC and UltraVNC. All of these are client + server VNC applications. Also, the viewer is platform independent; all of the previously mentioned VNC viewers can interact with any other VNC server that runs on a different operating system.

RealVNC comes in three editions: Free, Personal and Enterprise. The Free one is usually enough for the average user but it lacks many of the advanced features (i.e., file transfer, desktop scaling, session encryption, enhanced security, MAC OSX support, etc.) that are included with the Personal and Enterprise Editions. RealVNC is open-source. Its server runs on Windows, UNIX and GNU/LINUX.

TightVNC on the other hand is freely available under the terms of GNU GPL. This alternative software has slightly more features (i.e., file transfer on Windows). It does not offer encryption. Its server runs on Windows, UNIX, GNU/LINUX and MAC OSX.

UltraVNC has an amazing set of features and is freely available. Its server-side runs only on Windows. UltraVNC has a "Data Stream Modification" (DSM) plugin system feature included. It’s very powerful. It allows the use of an external .DLL, which means that you can use encryption, data recording, and more — virtually anything that specific DLL allows you to do. This DSM plugin system is actually a "tunnel" feature for your connection.

First I want to point out that all of the applications above are great and useful. Regardless of which one you choose I’m sure you’re going to benefit. If you choose the UltraVNC (remember, its server runs only on Windows) then you might want to check out this site, where you can download several DMS plugins for encryption. Otherwise with RealVNC and TightVNC you can manually set up an SSH ‘tunnel’ to secure your connection. We’ll get to that a bit later.

Now you can decide how you handle your VNC desktop sharing. You can use either UltraVNC with a DMS encryption plugin or TightVNC/RealVNC with an SSH tunnel. Depending on your decision you can go directly to the parts of this article that specifically address your chosen approach. 

{mospagebreak title=Installing and Enhancing UltraVNC}

You can find installation and configuration guides for UltraVNC here and here. Since the installation process for every VNC software is similar on Windows, I won’t get into the details of setting up VNC. Read those two guides; they’re very comprehensive and illustrative with tons of screen shots.

In this section I will lead you through the easy steps of how to add a DMS plugin so that you can have a secure connection. We already discussed previously that UltraVNC has the ability to run DMS plugins for encryption. That’s what we’re going to do now. At the installation process you might notice the DSM Encryption Plugin checkbox; it is an older release of the MSRC4Plugin. You might want to check it or not, it doesn’t really matter since we’re going to replace it with the latest release.

Assuming that you’ve installed UltraVNC you should visit this website and download one of the available plugins.

Currently there are three plugins: MSRC4Plugin, ARC4Plugin and AESV2Plugin. Each of them has a quick introduction explaining what and how their specific encryption works. Read that thoroughly, examine and decide which one you want. For the purpose of this article I’m going to use the first one, MSRC4Plugin. An argument for my choice could be that it is the most updated plugin, thus it is more powerful. Its latest release, version 1.2.2, came out on 8/16/2006; the other ones are from 2005 and are RCs (Release Candidates).

As I’ve mentioned earlier the UltraVNC package already contains the MSRC4Plugin (version 1.2.0). If you don’t check that box while setting up your UltraVNC then don’t make problems since we’re going to use the latest one anyway. If you checked that box then you have an older version of MSRC4Plugin. In this case I strongly advice replacing it with the latest release (that is in our case 1.2.2); see the changelog to see the list of updates.

Extract the content of your downloaded archive into your UltraVNC folder (i.e., "C:Program FilesUltraVNC"). Just a warning note: do not extract into the ‘plugin’ folder because it’s just a place to store plugins for VNC. You need to copy them to your VNC’s main folder to get them automatically recognized. Now, fire up UltraVNC Server!

You should see the above screen shot at the lower left corner of your VNC’s "Current User Properties" (or also called "Admin Properties") interface. Click on that arrow, select "MSRC4Plugin_NoReg.dsm" (that should be your only choice) and make sure you check that box near "Use." After this, just click on "Config."

The MSRC4 Plugin Configuration window shows up and automatically searches for your plugin keys. It won’t find any. Therefore you need to make your first key at your first run; that key will be used in the future. You are going to need that key at every client computer, otherwise you won’t be able to view your server running on that specific key-based encryption. Select "128 bit" (it’s automatically selected anyway, but make sure it is) and click on "Gen Key." If you leave its name at default then it’s going to be "new_rc4.key." Rename it to "rc4.key" for convenience; also, this way it will get auto-detected by your client(s).

After hitting "Gen Key" your specific key will be created and the configuration tool will close. Just for the sake of example here’s the content of the key I’ve just generated:

128 bitL h ¤ Ýžm"Äš`°c-˜(Vqk nʐ¨d®,ŠõdñhKBP¼·ç´*œCwŸ*7"Õ’S…,¾UDÇ×‑ôD


Now you’re ready. Set your Authentication VNC Password, make your other changes and click on "OK." Your secure VNC server will be up and running!

Don’t forget to copy your "rc4.key" onto your flash drive or send it to yourself via an email. You won’t be able to view and interact with your very own server from a client if you don’t have your RC4 key! When you’re preparing your client all you need to do is to copy your "rc4.key" into your UltraVNC main directory. Run your client, check the box called "Use DSMPlugin" and enter your server’s IP address. Click on connect, enter your password and you’re ready to rock. Enjoy…

That covers everything related to UltraVNC and DMS Plugins. It’s a very powerful client + server combo package for Microsoft Windows operating systems, and now you’re going to use the advantages of its capabilities to the max and benefit enormously.

{mospagebreak title=Installing TightVNC and Setting Up an SSH Tunnel}

 As I’ve already explained in the previous section the installation process of VNC clients is pretty much the same so you should read this and this if you’re facing any dilemmas or problems while setting it up.

Through this section I am going to assume that you have TightVNC server installed correctly. Here you’ll read a sort of how-to about setting up an SSH tunnel to secure your VNC connection(s). I will call SSH server SSHd (daemon) later on.

Before we move on, what is SSH? Secure Shell is a network protocol that allows the establishment of secure channels between a local and remote user. This is crucial because it enhances confidentiality and data integrity by encrypting the data that is transferred. This way it prevents interception; all in all it means security!

First of all, you need to allow interaction via port 22 to allow SSH. In your case perhaps this is the Windows Firewall (if you have Windows XP with Service Pack 2). Go to the ‘Exceptions’ and add ’22’ there as a port on TCP; name it SSH. If you have a router (D-Link, Linksys, SMC, Sitecom, etc.) then don’t forget to set up port-forwarding. Go to your router’s configuration screen (usually and forward port 22 to your server’s IP (on which SSHd will run).

 Now you should be ready on the hardware level. But you still need the software! Visit OpenSSH. I dare to call it the best SSH client that is also open source and free. You might want to run the latest Open SSH daemon via Cygwin (that is an emulation of the Linux environment -POSIX- and API) or download a slightly older server that works on Windows natively from here. If you choose the first solution then I invite you to take a look at this guide; it’s an awesome how-to that explains and illustrates how to set up and install SSHd on Cygwin. Do that only if you want to get into Cygwin and experiment.

Nonetheless, we’re lucky because there is copSSH. It’s a pre-made installation package that installs both Cygwin and an openSSH daemon, configures and runs both of them, sets up the account information and runs the service — all automatically. There’s little to nothing that you actually need to do. You can download it from the previous link (official link at Sourceforge) or from the Softpedia mirror (here).

Once you’ve downloaded it, just install it. After installation you need to activate a user: Start-> Programs-> COPSSH-> "1. Activate a user." Select your username (I assume it is Administrator), leave your command shell at "/bin/bash" then click on ‘Next.‘ Type in your pass phrase and hit ‘Activate.‘ Then a message box will state: "User Administrator is activated successfully and can establish an ssh connection to this machine now." After this stage your SSHd is up and running.

Now you’re all set. On your remote computer install TightVNC viewer and an SSH client. In our case that’s going to be PuTTY. I wholeheartedly recommend it. It’s a great SSH and TELNET client. Download the latest release for Windows here

 Extract PuTTY to a specific place and run it. The Configuration window will appear. At the "Session" menu at ‘Host Name‘ type in the EXTERNAL IP address of your server. Leave ‘22‘ for the port and ‘SSH‘ for the connection type. Save this session, name it "Server" or whatever you wish. This is crucial. If the server is on the same system as you’re running PuTTY currently then use "loopback", otherwise use the external IP.

The previously mentioned scenario about PuTTY being on the same machine as your SSH server is possible only when testing; it’s impossible when you’re trying to connect to your server from a remote computer — thus technically you should never need to set the loopback for hostname to connect on. However if you still want to try it then don’t forget to tick "Allow loopback connections" when configuring TightVNC.

Drifting back to PuTTY, select at the left menu-list "Connection" -> "SSH" and then "Tunnels." Type at the ‘Source port‘ 5900 (that’s the default for TightVNC) and at the ‘Destination‘ 192.168.XXX.XXX:5900 – note: replace those X’s with your internal IP address of your TightVNC server or if both SSHd and TightVNC are on the same computer/server then you need to forward to, obviously. Leave the rest on Auto and Local. Finally hit "Add."

In the screen shot below I’m assuming that the SSH server (SSHd) is on the same computer as your TightVNC server, therefore I’ve set up forwarding via the loopback. Since most people will have only one server, chances are that the SSHd will always be running on the same system with the VNC server — so the aforementioned scenario is quite often possible.

That’s all; click on "Open." PuTTY should make the connection to your server, and then you need to log in. Just use the password you’ve set up for SSH server. Then run TightVNC viewer and connect to to your server!

The above sketch illustrates in a nutshell the whole SSH tunneling procedure. Throughout this scenario I will call the server ‘Server’ and your remote client computer ‘Remote Laptop’ (that’s from where you want to view your server). You need to set up TightVNC and SSHd servers — both on your ‘Server,’ obviously.

On your ‘Remote Laptop’ you start up PuTTY (your SSH client) and set it up to connect to your Server (thus you set your server’s external IP address as Host Name to connect), but you also set up port forwarding to forward from your SSH server to your VNC server (thus if both are on the same machine loopback is used; if they are on different machines then you need to use an internal network IP addresses, i.e., ‘192.168.X.X.’). After you’ve connected with PuTTY to your Server, you start VNC Viewer on your laptop and let it connect to your Server via PuTTY. Since PuTTY runs on the laptop too you will use loopback again.

Imagine the flow of data going from your laptop VNC Viewer to PuTTY where the code gets secure and encrypted, and then from PuTTY to your SSH Server where the code gets understood and decrypted, and ultimately from here gets transferred to your VNC server. Of course this is valid going the other direction as well. Now I really hope that it makes sense.

Final Words

This article should definitely help you to understand and realize the methods and different ways of securing your VNC connections, but you should also have gotten a basic understanding of SSH tunneling. You might already see its benefits for other applications too, from FTP transfers up to even secure Instant Messaging. You can basically tunnel every connection via SSH. But many people still neglect it.

This part of the series was focused only on Microsoft Windows operating systems, but what if you’re running Linux? Have no fear! Stay tuned for the next part of this article that will be focused only on Linux. I’m sure you don’t want to miss it. The main ideas there are going to be similar: setting up VNC server, SSH tunneling and then viewing your VNC server from a remote Linux machine and additional information.

Good luck and see you at Dev Shed for the second part!

One thought on “Secure Remote Desktop Sharing with VNC on Windows

  1. Suggestions and other feedback is greatly appreciated.

    Be sure to stick around for the second part. Thanks for reading!


[gp-comments width="770" linklove="off" ]