scp and ssh
Written by Sam Moffatt   
Thursday, 11 October 2007 10:54

scp, or Secure Copy, is a file transfer system based on the SSH (Secure Shell) system available to modern Linux distributions. SSH is the method of choice for logging into machines via the network or internet as communications are encrypted, but a part of SSH is a "SFTP" server (Secure FTP) that uses SSH as a transport layer to encrypt data transfer. scp can be an easy and secure way of transferring files over the network and even as a general purpose replacement to FTP. There are various GUI's available for all operating systems (WinSCP on Windows, Cyberduck on Mac OS X, and Nautilus/Konqueror on Linux). Since this is a Linux group I'll cover the Linux options.

Using SCP with Nautilus


There are two ways to get to SFTP/SCP interfaces with Nautilus. The first is to go "File -> Connect to Server" and select "SSH" and enter the details in the box. This will also create a location on the left hand side of your Nautilus view if you have the Tree Sidebar panel open. Additionally it'll add an entry to the Places Sidebar panel and drop down and finally the Computer view of Nautilus. However if you're only temporarily visiting a server you can use the Location switcher to go directly there (it will appear in your history). If you're still using the button style interface for Nautilus you'll have to hit Ctrl-L to expose the Location bar. Now type in "ssh://hostname" or "sftp://hostname" (both go to the same place, I prefer ssh) and it will attempt to login using your username and if you have a SSH key, that key, or it will prompt you for a password. You can also use "ssh://username@hostname" to use a different username to log into the server. Konqueror uses a similar system, however I believe their handler is fish:// instead of sftp:// or scp://


Using the command line

The 'scp' command is used to control secure copy on the command line. scp has a simple syntax:

scp user@host:path/to/file user@host:path/to/file

scp has issues going between two different hosts, but you can use the following to send a file to a different server:

scp localfile.txt remoteserver:

This will put it in the default directory on the server (usually the home directory), you can also be more specific:

scp localfile.txt user@remoteserver:/home/user/localfile.txt

Also renaming the file is supported as well:

scp localfile.txt user@remoteserver:/home/user/remotefile.txt

Copying files back is just the reverse:

scp user@remoteserver:/home/user/remotefile.txt .

We use '.' here to copy it to the present directory. Think of it similar to that of the cp except we have "user@server:" in front of the paths. Remember "servername:" (there is a colon there on purpose) is important before the paths otherwise it won't recognize that you're copying to a server and will do a normal copy operation.

Some other useful flags for SCP is the recursive operator:

scp -r remoteserver:/some/directory .

This will copy all of /some/directory on the remoteserver to the local directory. This also works both ways so you can send entire directories to machines as well as recieve them. The -l option limits bandwidth (measured in kilobits) and the -p option preserves file information (modified time, etc) .


Disk backups using SCP

Note: Since I wrote this article it appears that SCP starts to complain that device files are 'regular files'. If this is happening to you, don't fear! Read down to the next section. 

The last interesting feature of SCP is a combination of SCP and Linux. In Linux, everything is a file, including devices. We can use this to our advantage and write something like this:

scp /dev/sda1 backupuser@backupserver:/backups/servername/diskimage.img

What this does it basically send the contents of the first partition of the first SCSI (or SATA) disk and send it over the network to "backupserver" and into that file and it encrypts it as it goes. Network restores are just as easy, just reverse the command:

scp backupuser@backupserver:/backups/servername/diskimage.img /dev/sda1

Of course it will have to be run as root to complete the operation (hence why we specify a different user, backupuser) but this when tied with a Live CD it captures a partition completely. By specifying /dev/sda captures the entire drive. This isn't a full alternative to ghost programs that compress the image but it is an easy option for creating and moving images over the network in one hit. Last but not least, it allows this interesting ability:

scp /dev/sda1 root@remoteserver:/dev/hdb1

This is more an example of what can be done, it is advisable not to log in as root remotely, however in this case we directly move one partition onto another partition on another machine. 


Disk backups using SSH

Since the above method has stopped working for what ever reason (maybe the SSH dev's didn't like the feature, I know I did), I've had to come up with another way of handling things. The solution is deviously simple:

sudo cat /dev/sda1 | ssh backupuser@backupserver "cat > /backups/servername/diskimage.img"

So if you've read my previous comment this is much the same. You'll need to be root (or use sudo/su -c to execute the cat) for this to work as above. Restoring is of course the reverse operation:

ssh backupuser@backupserver "cat /backups/servername/diskimage.img" > /dev/sda1

And of course transferring partitions directlry will also still work:

cat /dev/sda1 | ssh root@remoteserver "cat > /dev/hdb1"

As stated above I'd avoid using root to do tasks, but the possibility is still there. Lastly we can also compress our data on the fly as well:

cat /dev/sda1 | gzip | ssh backupuser@backupserver "cat > /backups/servername/diskimage.img.gz"

Through the magic of pipe's you can usually cobble together almost anything you need, the reverse you need to run it through gunzip to get your data back but it should do the trick nicely.

Last Updated on Thursday, 13 November 2008 15:14