The other day I had an interesting situation to solve:

I have some data on machine B and I want it copied on machine C. However, machines B and C have no direct connection (known_hosts), but machine A can connect to B and C without password.

It is obvious that one needs to use machine A as a proxy between B and C.

    [B] ---> [A] ---> [C]

After a while thinking of tunnelling and using some sort of actual network proxy, I realised that I/O would also do the trick: We can have a command runing on B to output data to stdout and another command running on C reading from stdin. The end command would look something like:

ssh user1@B 'cd src_dir && tar -Ozc .' | ssh user2@C 'cd dest_dir && tar zxf -'

where:

  • user1 and user2 are the users under which machine A has password-less access to B and C respectively
  • src_dir is the location where data is
  • dest_dir is the location where data needs to be copied to

Few notes:

  1. We change to src_dir first in order to have the archive with a relative path to the actual data that needs to be copied. Otherwise, the path in dest_dir will also contain src_dir
  2. -z is used to compress/decompress data via gzip. One can use bz2 too if available on the source/dest machines.
  3. While using -z results in less data transferred between machines, the CPU impact of the compression may result in longer copy times. My particular case worked best without the -z flag
  4. Depending on the number of files to be copied, you might want to also drop the verbosity flag at destination (-f)

Taking into account the above, my fastest variant of the commant is:

ssh user1@B 'cd src_dir && tar -Oc .' | ssh user2@C 'cd dest_dir && tar f -'

HTH,