Sonntag, 18. April 2010

Copying files between two remote hosts

There are a few ways to copy files between two remote hosts using scp:

  • download the file from host1 locally and upload to host2
  • ssh into host1 and scp to host2
  • ssh into host2 and scp from host1

I wasn't happy with either of them. Downloading and uploading is not practical for huge files whereas the other two methods are inconvenient. They also require me to have my private key file lying around on one of the involved hosts as I use public key authentication.

Luckily scp allows to copy files between two hosts directly, or so I thought. From the man page: "Copies between two remote hosts are permitted."

However, I found that scp uses ssh to connect to the first host and scp's from there. This is exactly method 2 as described above and not an option for me. After some research I noticed that I am not the only one with this problem. Eliot from SaltyCrane already solved one of my problems: To make remote scp more convenient. His scp_r2r script can be run like this:

  # scp_r2r host1:/some/file host2:/target/path

Host1 and host2 are set up via an SSH client configuration file:


Host host1
User myremoteuser1
Hostname host1.example.org
Port 2222
IdentityFile /path/to/host1-id_rsa

Host host2
User myremoteuser2
Hostname host2.example.org
Port 2222
IdentityFile /path/to/host2-id_rsa

His script copies the private key file to the remote host, then ssh's into it and starts scp from there. I consider this very insecure, because if an attacker were to break into host1 he could find the private key. This would make it easier for him to force his way into host2 as well. The private key should never leave my own computer.

The solution is SSH agent forwarding: First an SSH agent program is started locally and the SSH connection to host1 is established with the -A option (agent support). The subsequent authentication request from host2 is then forwarded back to the local machine and handled by the agent. For this to work, the remote SSH server must support agent forwarding. In OpenSSH the option "AllowAgentForwarding yes" is required.

Basically, all that is needed are the following commands:


# eval `ssh-agent`
# ssh-add /path/to/host2-id_rsa
# ssh -A host1 scp /some/file myremoteuser2@host2.example.org:/target/path
# ssh-agent -k

There is a lot of information about Agent Forwarding in SSH: The Definitive Guide by Barett and Silverman.

I wrote a python script to combine the convenience of Eliot's script with SSH agent forwarding. I've also added some additional functionality. By default, the given file is pushed from host1 to host2. But it can also be pulled by host2 from host1. The latter is useful if host2 is firewalled and cannot be reached from host1.

Examples:


# scp_r2r host1:/some/file host2:/target/path
# scp_r2r --pull host1:/some/file host2:/target/path

The -v or --verbose option can be used to see what's going on internally.

Download: scp_r2r.py

Update 2010-05-14: Added script to GitHub.
Update 2012-05-19: Fixed download links.

2 Kommentare:

  1. Hi, I'm getting this error:

    [user_a@remote_host ~]$ ./scp_r2r.py -v hostA:/tmp/jboss hostB:/tmp/
    Traceback (most recent call last):
    File "./scp_r2r.py", line 301, in ?
    main()
    File "./scp_r2r.py", line 298, in main
    main_transfer(host1, host2, options)
    File "./scp_r2r.py", line 244, in main_transfer
    sshconfig = read_ssh_config()
    File "./scp_r2r.py", line 124, in read_ssh_config
    sshconfig.parse(open(SSH_CONFIG_FILE, 'r').readlines())
    File "./scp_r2r.py", line 92, in parse
    raise Exception('Unparsable line: %r' % line)
    Exception: Unparsable line: 'hostA'
    [user_a@remote_host ~]$

    Could you please tell me what am I doing wrong ??

    AntwortenLöschen
    Antworten
    1. It looks like an error in your SSH config file. According to the error message you simply wrote "hostA" on one line, where it should read "Host hostA". Check out the examples in my post. You can copy&paste them into your SSH config file and only replace the host data.

      More information about the SSH config file format can be found in the ssh_config man page.

      Löschen