Using Shadow Copy to back up folders in Windows 7

It is always a good idea to keep backups of your stuff. Unfortunately, some folders like %APPDATA% can not be copied while Windows has locked files inside. That's where the Volume Shadow Copy Service (VSS) comes into play. It basically creates a consistent snapshot of the data and allows to access it regardless of file locks. I wrote a small batch script for Windows 7 that does the following:

1. Create snapshot of folder
2. Mirror the folder to another place
3. Destroy snapshot

The script relies on three tools, only one of which ships with the vanilla Windows 7 installation. These are the dependencies:

• vshadow.exe (part of the Windows 7 SDK: Bin\x64\vsstools\vshadow.exe)
• dosdev.exe (part of the free Volume Shadow Copy Simple Client)
• robocopy.exe (installed with Windows 7)

You can download the batch file below. It creates a backup of your own user's %APPDATA% folder (C:\Users\USERNAME\AppData\Roaming) to the target folder D:\Backup\AppData. The script needs to run as Administrator in order to create snapshots.

Download: backup_appdata.cmd

Copying files between two remote hosts

There are a few ways to copy files between two remote hosts using scp:

• download the file from host1 locally and upload to host2
• ssh into host1 and scp to host2
• ssh into host2 and scp from host1

I wasn't happy with either of them. Downloading and uploading is not practical for huge files whereas the other two methods are inconvenient. They also require me to have my private key file lying around on one of the involved hosts as I use public key authentication.

Luckily scp allows to copy files between two hosts directly, or so I thought. From the man page: "Copies between two remote hosts are permitted."

However, I found that scp uses ssh to connect to the first host and scp's from there. This is exactly method 2 as described above and not an option for me. After some research I noticed that I am not the only one with this problem. Eliot from SaltyCrane already solved one of my problems: To make remote scp more convenient. His scp_r2r script can be run like this:

  # scp_r2r host1:/some/file host2:/target/path

Host1 and host2 are set up via an SSH client configuration file:

  Host host1
    User myremoteuser1
    Hostname host1.example.org
    Port 2222
    IdentityFile /path/to/host1-id_rsa

  Host host2
    User myremoteuser2
    Hostname host2.example.org
    Port 2222
    IdentityFile /path/to/host2-id_rsa

His script copies the private key file to the remote host, then ssh's into it and starts scp from there. I consider this very insecure, because if an attacker were to break into host1 he could find the private key. This would make it easier for him to force his way into host2 as well. The private key should never leave my own computer.

The solution is SSH agent forwarding: First an SSH agent program is started locally and the SSH connection to host1 is established with the -A option (agent support). The subsequent authentication request from host2 is then forwarded back to the local machine and handled by the agent. For this to work, the remote SSH server must support agent forwarding. In OpenSSH the option "AllowAgentForwarding yes" is required.

Basically, all that is needed are the following commands:

  # eval `ssh-agent`
  # ssh-add /path/to/host2-id_rsa
  # ssh -A host1 scp /some/file myremoteuser2@host2.example.org:/target/path
  # ssh-agent -k

There is a lot of information about Agent Forwarding in SSH: The Definitive Guide by Barett and Silverman.

I wrote a python script to combine the convenience of Eliot's script with SSH agent forwarding. I've also added some additional functionality. By default, the given file is pushed from host1 to host2. But it can also be pulled by host2 from host1. The latter is useful if host2 is firewalled and cannot be reached from host1.

Examples:

  # scp_r2r host1:/some/file host2:/target/path
  # scp_r2r --pull host1:/some/file host2:/target/path

The -v or --verbose option can be used to see what's going on internally.

Download: scp_r2r.py

Update 2010-05-14: Added script to GitHub.
Update 2012-05-19: Fixed download links.