Last night my Sheevaplug’s power supply unit blew up. Slept clean through it but I think the fumes were responsible for the unorthodox dreaming I did that night. Incidentally, the dream was about electronics. I dreamt I was part of some development project, bringing electronics to the Indian countryside and I had just invented this incredibly tiny airplane which could be made from readily available components such as tins and shrubs. It could fly a route autonomously (!) and had a cargo net which could be used for transporting lumber, cattle, children – anything a farmer family would need transported. The size of a large model airplane, one was supposed to hold on to its landing gear (with one’s legs taking over landing gear functions – better bring some bandages) and enjoy the flight. So I was flying around the hot Indian plains, soldering iron tucked under my belt, spreading the tale of this quantum leap in mobility in the land of Vishnu and Sheeva.

Back to my Sheevaplug. It was no more. A capacitator in the power supply unit (that converts 230VAC to 5VDC) had made a complete mess of itself. And the rest of the PSU. It looks like burnt Marshmallows but smells worse.
Also, the button cell that keeps the system clock running when power is lost, had swollen to unhealthy dimensions and had barfed its contents over part of the mainboard:

Well, at least I’m not alone in getting the marshmallow treatment. It appears to be a heat issue on the first version of the development kit.

Amazingly, the PSU still supplied about 2V, enough to make the blue LED blink to signal me that my marshmallows were ready.

I keep this box with old AC/DC adapters around. I’ve adapted my Sheevaplug to contain a socket to plug in a jack from an external 5VDC 2A supply, replaced the CMOS battery, and after some initial problems with the unit losing track of time after being powered down, all is working fine again. But better not take this Sheevaplug to the scorching heat of the land of Sheeva.

So I had assembled this computing cluster thing based on execnet for a CS project. The slave nodes were dozens of student lab computers. Those are running firewalls blocking all ingress traffic, not just from the big bad internet but also from the lab networks — which contain only trusted machines and are already protected by a firewall restricting flow to and from larger networks!
Thing is, I need access to the SSH daemons running on those nodes. And I do not (and should not) have root on those machines…
Solving this problem on OSI Layer-8 proved to be an inefficient approach. The best way to fight bureaucracy is to not take it seriously. A bureaucracy (almost by definition) is not creative and cannot keep up with workarounds. To admit existence of creative, necessary workarounds to achieve legitimate ends is to admit defeat and that is why workarounds usually get ignored. It’s ignore and counter-ignore which is a bit sad since there is (or should be) a common goal: education, in this case. But I digress. The solution proved to be to ignore layer 8 and solve the problem on OSI layer 4. That certainly saved a lot of keystrokes!
To punch through the firewall, I have each slave node initiate a reverse tunnel (ssh -R) to a central node, using pubkey authentication. This way the otherwise unreachable SSH daemons on the slave nodes are proxied on the master node and are reachable via a socket on the master nodes’ loopback interface. The slave nodes specify the reverse tunnel endpoint port based on their IP. They connect automatically, and with some clever but too-kludgy-to-show scripting I can dynamically update my running clusterfied program with newly joining slave nodes to export compute jobs to.
Obviously, the slave nodes need to login to the central node via SSH and do so using a single account on the central node. But I don’t fully trust the slave nodes. I want them to be able to initiate reverse tunnels, and not do much else. Here’s how I approached this problem, with the server running OpenSSH 5.3p1.

Chroot

Chrooting comes to mind. With a chroot, you can limit the filesystem locations the user can access to some remote uninteresting branch of the tree.
From man 5 sshd_config:

ChrootDirectory
Specifies a path to chroot(2) to after authentication.  This path, and
all its components, must be root-owned directories that are not writable
by any other user or group.  After  the  chroot,  sshd(8) changes the
working directory to the user's home directory.

The path may contain the following tokens that are expanded at runtime
once the connecting user has been authenticated: %% is replaced by a
literal '%', %h is replaced by the home directory of the user being
authenticated, and %u is replaced by the username of that user.

I set up a group for that and use the following ‘Match’ stanza in the central node’s sshd_config:

Match Group chrootage
        ChrootDirectory %h

%h is the matched user’s homedir.
I set up a user named ‘chroot’ with a not-so-regular home directory. More on that home directory later on. This user needs a valid shell, even if it will never run that shell (I do not want them to run one). Without a valid shell the user will not be able to log in.

ForceCommand

From man 5 sshd_config:

ForceCommand
Forces the execution of the command specified by ForceCommand, ignoring
any command supplied by the client and ~/.ssh/rc if present. The command
is invoked by using the user's login shell with the -c option. This
applies to shell, command, or subsystem execution.  It is most useful
inside a Match block.
The command originally supplied by the client is available in the
SSH_ORIGINAL_COMMAND environment variable. Specifying a command of
``internal-sftp'' will force the use of an in-process sftp server that
requires no support files when used with ChrootDirectory.

Well that ought to suit my purposes. Ideally, this command would be a rather boring and useless one that just sits there waiting for input. I could copy cat and the libs it links to over to the chroot directory. I might as well make it the login shell for the chroot user. But the user would then still be able to use sftp. And if I would want to disable sftp, I would have to do so server-wide! Me and the other users use sftp extensively, so I’d then need to run a second SSH daemon with no sftp subsystems especially for this purpose. Hmmm.
Another approach is to limit the user to the internal sftp subsystem. Added benefit is that I do not need to maintain any binaries and libs in the chroot. With this, the user logging in with SSH is presented by a boring interpreter in which he cannot launch any programs — it’s the sftp subsystem. Portforwarding, which is needed for the reverse tunnel, will work. SFTP will work, obviously. Let’s do it.
sshd_config:

Subsystem       sftp    internal-sftp

Match Group chrootage
        ChrootDirectory %h
        ForceCommand internal-sftp

Make SFTP boring ­— chrooted home directory contents

So what do we need inside the chroot directory? No binaries or device nodes, that is one of the advantages of using the internal-sftp subsystem.
We do need an authorized_keys file readable by the chrooted user since this user won’t be able to log in using pubkey auth otherwise. We can’t use AuthorizedKeysFile inside a Match stanza, so the authorized_keys will have to reside in the default location — the .ssh subdirectory of the user’s homedir. I will use authorized_keys to limit the set of hosts this user can login from to the domain of the slave nodes, so obviously the user should not be able to add keys or lift limitations — authorized_keys will need to be read-only. This is what I ended up with.

#ls -Rla /path/to/chrooted_homedir
/path/to/chrooted_homedir/:
total 16
drwxr-xr-x 3 root root       178 May  8 16:36 .
drwxrwxrwt 9 root sys       4887 May  8 16:36 ..
dr-xr-x--- 2 root chrootage  189 May  8 16:22 .ssh

/path/to/chrooted_homedir/.ssh:
total 12
dr-xr-x--- 2 root chrootage  189 May  8 16:22 .
drwxr-xr-x 3 root root       178 May  8 16:36 ..
-rw-r----- 1 root chrootage 1221 May  8 14:25 authorized_keys

This is all the user can view over sftp. It is minimal and extremely boring. Note that there is no place the user can write to. Even if he would have some way of launching programs (he hasn’t, due to ForceCommand) there is no place to upload them to and launch them from in the first place.
On to authorized_keys:

no-pty,from="*.science.uu.nl" ssh-rsa AAAAB3and-the-rest-of-the-pubkey

no-pty serves to prevent the allocation of a terminal. I don’t think it offers any security beyond the measures already in place, but I like to think this saves resources.

Tunneling risks

I need TCP forwarding to get the reverse tunnels to work. That means that using the account, users can set up arbitrary TCP tunnels, using the master node as a proxy… Less nice. But wait – can’t I use PermitOpen inside Match to limit the creation of arbitrary tunnels? Yes. But the reverse tunnel is subjected to the same restrictions. Therefore, I would need a PermitOpen for every slave node and I’d need to know the source port of the incoming client connection in advance. There’s no wildcarding in PermitOpen, so this is infeasable.

‘Match’ directive precedence

The order appears to be significant. That’s also what the manpage seems to hint at where it says

Match:
Introduces a conditional block.  If all of the criteria on the Match
line are satisfied, the keywords on the following lines override those
set in the global section of the config file, until either another Match
line or the end of the file.

It’s not as clear as could be what happens to keywords in the next Match directive. What if the next Match directive matches? Nothing! The next Match directives don’t matter! In my tests, incoming connections appeared to be only matched once. The match blocks have no relation to eachother, only to the main config.
That implies that you need to put the most specific match directives — the ones with the most patterns ­— on top, and that each Match block needs to hold any and all keywords you want to specify beyond the main config. It’s not CSS ;-)

You may not be aware of the fact that any picture on facebook is publicly accessible. Yes, it is. There’s no authentication & authorisation whatsoever. Handling those in a scalable way would ramp up costs. Your privacy is not worth those costs. Contrary to the impression you are trying to deliver through your profile, you are not important. Happy shareholders are important!

Due to this fact I just need to know the URLs of your pictures. From the URL I can determine whether it’s a profile picture, profile picture thumbnail, photo, photo thumbnail, etc.

Wouldn’t it be fun to mix the pictures of the facebook page you are currently viewing with those from facebook pages others are viewing? So when you’re browsing your friend’s albums, you not only see his pictures but pictures from other peoples’ albums too, and vice versa?
The pictures may be requested by the guy across the bar, or by the girl one floor down in the library, or by anyone on the same network as you are — all of you are browsing together with the people in your physical vicinity, sharing whatever pictures you encounter! It’s beyond Facebook. It’s crowdbrowsing. It’s Megafacebook.
While you may not know these newly inserted friends, you might get to. Maybe you bump into one another at the toilets, or at the counter.
“Why is everyone staring at me like that?” you naively wonder. (They’ve seen those pictures).
“Does she know that I know about those pictures of her and her friends? But wait… what might she know about me?”, your paranoid mind ponders.
It’s all about what you think of others and what others think of you. Total absorption. Now that’s what I call social networking. All hail Facebook Social!

Give the wifi crowd at your local coffeeshop the pleasure of learning a little bit more about eachothers lives and friends.

Get to work

You need:

• one network vulnerable to ARP poison routing (that’s most of them) or one network which you already control anyway. Make everyone route their traffic through your machine.
• one installment of the Nginx web server, configured with --with-http_random_index_module. I use the 0.8.3x series.
• one installment of the Squid http proxy server. I use the 3.1 series.
• Perl and LWP::Simple.

Set up Nginx

Create some directories to hold the images:

mkdir /var/www/facemix/{albums,photos,photosthumb,smoelen,smoelenthumb}

Tell Nginx to respond to requests for those directories by randomly serving one of the files in them:

location ~ ^/facemix/([^/]+)(/?.*)$ {
alias /var/www/facemix/$1/$2;
random_index on;
expires -1;
}

You need the ‘expires -1′ to avoid caching. If proxies or user agents were to cache the results, they wouldn’t be very random anymore now would they.

Stick some files in there and test your installation.

Set up Squid

Set up squid in interception mode. If you’re not NATting the routed traffic, set it to run on port 80. If Nginx is already listening on that socket, make Nginx listen on some other port, or localhost only, while running squid on port 80 but only on the external interface.

Set up networking

This is for iptables.

• You’re NATting the pwned hosts. Run something along the lines of
  iptables -t nat -A PREROUTING -i $INTERFACE -p tcp --dport 80 -j REDIRECT --to-port 8080
  to redirect all traffic incoming on $INTERFACE and destined for port 80 to port 8080, which is where you need squid to listen on.
• You’re doing 2-way ARP poisoning (cheers!). Run something along the lines of
  iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j DNAT --to-destination $YOURIP
  Squid needs to run on port 80 on interface with IP $YOURIP.

Check Squid’s logs to verify that requests are intercepted successfully.

Run the redirection script

I don’t touch Perl very often, and cobbling together this script made me remember why that is. It’s very usable as a means of frightening little kids.
In a nutshell, what my redirector script does is

1. determine whether the URL fed to it by Squid is a facebook picture url;
2. if so, and if we don’t have that picture yet, fork off to download it;
3. point Squid to a random picture of the same type (served by Nginx).

I like the forking. I dislike the iffed regexes which could probably be condensed into one but then it wouldn’t be ‘cobbling together’ anymore.

Adjust the variables for your setup and tell Squid about the script (eg url_rewrite_program /usr/local/lib/facemix-squidredir.pl).

The Facebook logo will change to reflect the fact that the users are now browsing facebook in Social! mode.

One further note: This is privacy-invasive. I brush away my moral doubts by stating that anyone who signed away their privacy rights when joining facebook AND AT THE SAME TIME entertains any expectations with respect to privacy,
« inhale »
… is utterly mental and has completely lost any and all sense of proportionality. If you care about privacy, why use a service which lets you view any picture of any user regardless of who you are? Who are you kidding?

If you’re still reading, here’s the script:

#!/usr/bin/perl -w
 
use LWP::Simple;
 
$WEBROOT = 'http://localhost/facemix/';
$WEBDIR = '/var/www/facemix/';
$CHANCE = 5; #One in X requests gets mixed
$SIG{CHLD} = 'IGNORE';
 
$|=1;
while (<>) {
    local @reqfrags = split(/ /, $_);
    local $url = @reqfrags[0];
 
    if    ($url =~ /(^http:\/\/.*.fbcdn.net\/rsrc.php\/z7VU4\/hash\/66ad7upf.png$)/) {
        print "http://smormedia.gavagai.nl/2010/04/FacebookSocial2.png\n";
    }
    elsif    (($url =~ /(^http:\/\/photos-.*.fbcdn.net\/.*\/.*_n.jpg$)/) || ($url =~ /(^http:\/\/photos-.*.fbcdn.net\/.*\/n.*.jpg$)/)) {
        &mixurl('photos/',$url);
    }
    elsif (($url =~ /(^http:\/\/photos-.*.fbcdn.net\/.*\/.*_s.jpg$)/) || ($url =~ /(^http:\/\/photos-.*.fbcdn.net\/.*\/s.*.jpg$)/)) {
        &mixurl('photosthumb/',$url);
    }
    elsif (($url =~ /(^http:\/\/profile.*.fbcdn.net\/.*\/.*_n.jpg$)/) || ($url =~ /(^http:\/\/profile.*.fbcdn.net\/.*\/n.*.jpg$)/)) {
        &mixurl('smoelen/',$url);
    }
    elsif (($url =~ /(^http:\/\/profile.*.fbcdn.net\/.*\/.*_q.jpg$)/) || ($url =~ /(^http:\/\/profile.*.fbcdn.net\/.*\/q.*.jpg$)/)) {
        &mixurl('smoelenthumb/',$url);
    }
    elsif (($url =~ /(^http:\/\/photos-.*.fbcdn.net\/.*\/.*_a.jpg$)/) || ($url =~ /(^http:\/\/photos-.*.fbcdn.net\/.*\/a.*.jpg$)/)) {
        &mixurl('albums/',$url);
    }
 
    else
    {
        print $url."\n";
    }
}
 
sub mixurl {
    #args: subdir, url
    local $vork = fork();
    if ($vork == 0) {&getit($_[0], $_[1]);}
    if (int(rand($CHANCE)) == 0) {
      print $WEBROOT.$_[0]."\n";
    }
    else {
      print $_[1]."\n";
    }
}
 
sub getit {
    #args: subdir, url
    local $storedir = $WEBDIR.$_[0];
    local @urlfrags = split(/\//, $_[1]);
    local $fname = pop(@urlfrags);
    if (!stat($storedir.$fname)) {
      getstore($_[1],$storedir.'._tmp-'.$fname);
      rename($storedir.'._tmp-'.$fname, $storedir.$fname);
    }
    exit;
}

1. Make all reads come from one device, A
2. Make all writes go to another block device, B
3. Merge the writes stored in B into A
4. Without unmounting the filesystem on A!

Why? Well, as pointed out in the comments on this LWN article you could use this as a way to roll back an unfortunate upgrade without taking your system offline. But if the upgrade _is_ fortunate, you might want to merge the changes back in without having to take your filesystem offline.

Or you could use it to speed up writes — perform those against a RAM-based block device, and merge those writes back in when sysload is low. Or use it like I’m planning on doing: to avoid the small but frequent writes (growing logfiles, mainly) to the CF card in my Alix2 which I’ve built my router with.

#1+#2 are old hat — but if you’ve never made a writable snapshot with DM (not LVM) I strongly suggest you read the “Right To Your Own Devices” LinuxGazette article. It’s what got me started.

#3 is fairly recent hat, for ‘kernel 2.6.33′ values of “recent”.

#4 is shiny new hat and is what this post will be about.

Prerequisites

You must be running the Linux kernel, 2.6.33+, and have device mapper support (CONFIG_DM_SNAPSHOT) loaded and ready to go. You also need a recent LVM2 userland, I used version 2.02.60.
Usually one goes about snapshotting with the use of LVM, the Logical Volume Manager. LVM is just an easy high-level way of employing DM functionality for common use cases. Ours is not one of those, hence we’ll be using raw dmsetup-foo. That’s why I strongly recommend you read the LinuxGazette article mentioned before. We’ll not be revisiting DM basics.

Get Dirty

• Create your base block device of about 500MB.

  dd if=/dev/zero of=hard bs=1M count=500
  losetup /dev/loop0 hard

  Neat, now you have a block device which is actually a file on your file system (which is on a block device). You could mkfs and mount it, but we’re not going to, not yet. We’re going to add yet another layer of indirection: the device mapper.

  echo 0 $(blockdev --getsize /dev/loop0) linear /dev/loop0 0 | dmsetup create hard_a

  You now have a /dev/mapper/hard_a which is a linear (not mirror, not stripe, not crypt, not anything else but plain boring linear) mapping of blocks to its underlying block device, which was /dev/loop0, which is backed by the file with name hard.

• Create a device which the writes will end up on. How large should it be? You decide. If you write more data than this volume can hold, it will be dropped and all your writes will be lost. That’s what you get for bad planning! Of course, you could err on the safe side and make it huge. But I don’t think it’s of any use to make it larger than the device whose writes you want it to receive (please correct me if I’m wrong). And by making it huge, you’re wasting space. Why not have it take up exactly the amount of space it needs? Dynamically growing, so to say? With a loop device backed by a sparse file, you can! A sparse file is a file whose empty bytes are represented by metadata. That means they don’t actually have to be there until needed, which is on write.
  Let’s make a 200MB sparsely backed device to hold the writes diverted from our other device:

  dd if=/dev/zero of=soft_a bs=1 count=0 seek=200M
  losetup /dev/loop1 soft_a

• Now we make yet another device: top. This is the device we’ll be making a filesystem on. But all writes will go into the file called soft. That’s called a writable snapshot and you can assemble it like this:

  echo 0 $(blockdev --getsize /dev/mapper/hard_a) snapshot /dev/mapper/hard_a /dev/loop1 p 8 | dmsetup create top

  You now have a /dev/mapper/top block device. Go ahead, mkfs it[*], mount it and stick some files on it. Interestingly, you can see how many blocks (in units of bytes) you have dirtied by looking at the physical size of the ’soft_a’ file:

  du -B1 soft_a

• Now for the magic part. Freeze the block device!

  dmsetup suspend top

  All reads and writes to /dev/mapper/top will now block. Processes that were accessing the device will be in suspended animation. Really? Depends. You will observe that you can still do an ls on the filesystem if you’ve run one before and haven’t changed any data in the meantime. That’s because of the kernel’s caches, in this case, the dentry (directory-entry) cache. You can drop those caches (but you don’t have to) by running echo 3 > /proc/sys/vm/drop_caches, and after that, you will observe that an ls incantantion will appear to hang. Leave it like that for the moment.

• Now we want to merge the dirty blocks back into hard_a. But if we fiddle with the constituents of /dev/mapper/top through DM, it will be dropped! We can’t merge dirty blocks right back into /dev/mapper/hard_a as DM doesn’t trust us prodding the fundaments. So we fool it by creating a second loop device backed by the same file as the loop device backing /dev/mapper/hard_a:

  losetup /dev/loop2 hard
  echo 0 $(blockdev --getsize /dev/loop2) linear /dev/loop2 0 | dmsetup create hard_b

  If you look at the output of dmsetup table you will notice that DM thinks hard_a and hard_b are different devices. And rightly so, because they are, but they happen to point to the same data: the file called hard.

• Now we can merge the device that holds the dirty blocks for hard_a (that would be /dev/loop1) into hard_b:

  echo 0 $(blockdev --getsize /dev/mapper/hard_b) snapshot-merge /dev/mapper/hard_b /dev/loop1 p 8 | dmsetup create mergeomatic && dmsetup status mergeomatic

  You don’t have to do anything with the mergeomatic device. Just wait for it to finish merging. How do you know when it’s finished? I couldn’t figure it out so I asked the dm-devel list. Turns out you can use dmsetup status mergeomatic, its output format is <sectors_allocated>/<total_sectors> <metadata_sectors> and when the amount of sectors_allocated equals the metadata_sectors it’s finished. I looked at dmsetup status in my experiments but I had never seen anything but the same numbers, possibly because I/O to small loopback devices on a machine with loads of RAM will be blazing fast — writes to the backing file are cached in RAM, I guess.
  Done? Remove the snapshot-merge target:

  dmsetup remove mergeomatic

• All that’s left is to create a new device, based on /dev/mapper/hard_b or its backing file. If you’re tired of COWing around you could do:

  dmsetup remove hard_b
  echo 0 $(blockdev --getsize /dev/loop2) linear /dev/loop2 0 | dmsetup load top

  And if you want to continue this trick you’d just make a fresh sparsely backed loop device and make a snapshot target out of it:

  dd if=/dev/zero of=soft_b bs=1 count=0 seek=200M
  losetup /dev/loop3 soft_b
  echo 0 $(blockdev --getsize /dev/mapper/hard_b) snapshot /dev/mapper/hard_b /dev/loop3 p 8 | dmsetup load top

• And now for the grand finale:

  dmsetup resume top

  All blocked processes will spring to life. ls will return a directory listing as if nothing has happened. Mission accomplished.
  With dmsetup load we swapped block devices in-flight. The mounted filesystem on /dev/mapper/top didn’t notice — all blocks still look the same. But both DM and we know that the blocks are somewhere else physically now. Little did DM know that /dev/mapper/hard_b is backed by the same physical blocks as /dev/mapper/hard_a.
  We’d better do some cleaning up then:

  dmsetup remove hard_a
  losetup -d /dev/loop{0,1}
  rm soft_a

This hasn’t left the toying-around phase yet. But I’m thinking about writing some wrapper scripts and doing some more testing. Maybe I should call it the “WhatwasIthinking Volume Manager” as it’s rather tricky stuff. Especially with all the write buffering on multiple levels. I messed it up more than once during testing ;-)

[*] Not just any FS will do. Don’t use journaled filesystems. They assume their journal will be written to disk in-order. That assumption does not hold with file-backed loop devices, because on those there is yet another filesystem (and buffer layer) below it, deciding what data gets committed first.

CRYPTO_DEV_MV_CESA
This driver allows you to utilize the Cryptographic Engines and Security Accelerator (CESA) which can be found on the Marvell Orion and Kirkwood SoCs, such as QNAP’s TS-209.
Currently the driver supports AES in ECB and CBC mode without DMA.

Whether the accelerator will be used depends on whether an application uses the in-kernel crypto algorithms. OpenSSL does not unless it is explicitly enabled to use a certain accelerator, such as is the case with the Via Padlock engine. So this particular engine won’t let your SSH run faster. But it will speed up device mapper crypto if you use an AES cipher.

Right, let’s get on with it.

#uname -a
Linux sheeva 2.6.32-gentoo-r3 #3 Thu Feb 4 23:02:42 CET 2010 armv5tel Feroceon 88FR131 rev 1 (v5l) Marvell SheevaPlug Reference Board GNU/Linux


Quick & oh-so-dirty way of getting a RAM-backed block device (that is, if you don’t have swap enabled):

#mount -t tmpfs tmpfs /mnt/tmp/
#dd if=/dev/zero of=/mnt/tmp/blob bs=1M count=224
#losetup /dev/loop0 /mnt/tmp/blob
#cryptsetup -c aes -h sha1 -d /dev/urandom create test /dev/loop0

First we test without CESA.

#dd if=/dev/zero of=/dev/mapper/test bs=1M count=224
234881024 bytes (235 MB) copied, 41.858 s, 5.6 MB/s

Only 5.6 MB/s and the [kcryptd] kernel process is having your CPU for lunch.
Enter CESA:

#dmsetup remove test
#modprobe mv_cesa
#cryptsetup -c aes -h sha1 -d /dev/urandom create test /dev/loop0
#dd if=/dev/zero of=/dev/mapper/test bs=1M count=224
234881024 bytes (235 MB) copied, 18.0525 s, 13.0 MB/s

13.0 MB/s and there’s a new kernel process, [mv_crypto]. It’s eating about three times as much CPU as [kcryptd]. That means it’s offloading, which is good. The results are consistent over time so let’s say there’s a 2.5-fold performance gain.
The loop device setup causes some overhead. Out in the wild you’ll get about 19 MB/s writing to USB HDD. Cheers!

One particularly nasty thing that can happen to you is when your machine is subverted into using the attacker’s machine as the router. That is known as ARP poison routing. The attacker can proceed to not only read the traffic coming from your machine (which, on a shared medium, could be done anyway), or read the traffic going into your machine (again: on a shared medium, that could be done anyway), but the attacker can now also modify the traffic between you and the rest of the non-local network, e.g., the internet, in both directions. And that’s when he can really go to town with your traffic. Injecting a javascript keylogger into all the webpages you visit. ‘Sidejacking‘ your sessions, so he does not even need to know your passwords, just your session cookies — which you happen to transmit with every page request.

All possible unless you use transport layer security, which is tamper-proof once properly set up. Once properly set up. But setting up can have problems of itself — there are ways of preventing you ever going from HTTP to HTTPS. If you know a thing or two about HTTP and SSL you’ll be delighted to learn about Moxie’s very evil but very clever ways of doing so.

Anyway, some level of security can be achieved if you tell your machine to ignore any messages sent to you from the other machines on the local network. That includes messages that will make your machine believe that the router has suddenly changed its physical address — which is quite unlikely to happen, but those messages are exactly the type of message an impersonator would send you. Of course we’d need to whitelist the routers of the network, otherwise we can’t get traffic out of it and onto other networks. DNS resolvers will need whitelisting too, unless you’re running one on your own machine (probably not).
Not openly announcing your presence may also be something you wish for. If you have ever been on a network with a Mac user you have probably seen them popping up in your Zeroconf service browser as “Firstname Lastname’s iSomething”. Let’s cut down on that kind of promiscuity, too. But you should understand now that you can not actually hide unless you turn off your WiFi. Shared medium, remember?

I prepared a simple script to accomplish the above. I’ve used ip from the iproute2 package instead of sticking to old-school route, ifconfig, arp & co. And I must say ip neigh flush nud stale has a poetic ring to it, wouldn’t you agree?

Take note: this will only protect you from some kind of attacks, and only partially. An attacker has a window of opportunity between your machine getting assigned a DHCP lease and you running this script, for instance. Or maybe the access point is rigged. Actually all protection other than end-to-end encryption combined with mutual authentication is pretty useless on shared networks ;-)

Here’s the script. Linux-only. If you want to use it, get the latest version from my public repository.

#!/bin/bash
 
# arpshield 0.2
# Protects against ARP poisoning and cloaks your machine for all 
# local link devices but the router(s) and the DNS server(s).
# Whitelisting DHCP servers also works if you use the dhcpcd program
# to obtain DHCP leases.
# This program is of no help if your setup is already poisoned.
# Have a look at ArpON (http://arpon.sourceforge.net/manpage.html) if
# you need more extensive protection.
#
# Needs 'ip', 'awk', 'sed', 'arptables', and 'arping' and expects
# them on $PATH. Needs appropriate privileges (so use sudo).
# Takes a network interface as an argument. The network interface
# should be up and configured. If no argument is given, clear all
# rules. Obviously you should do that before connecting to a new
# network.
#
# Copyright 2010 Wicher Minnaard (wicher@gavagai.eu)
# License: Creative Commons Attribution-Share Alike 3.0
 
# Do you use dhcpcd for aquiring DHCP leases? And is it running?
dhcpcdLEASEFILE="/var/lib/dhcpcd-${1}.info"
dhcpcdPIDFILE="/var/run/dhcpcd-${1}.pid"
test -f ${dhcpcdLEASEFILE} && test -f ${dhcpcdPIDFILE} && source ${dhcpcdLEASEFILE}
 
# In case you lack the luxury of dhcpcd, where is your resolv.conf?
RESOLV="/etc/resolv.conf"
 
# No user-servicable parts below this line.
DEV="${1}"
 
# I know, I know. But if your routing table contains 0.333.456.789 you have bigger problems ;-)
IPREGEX="\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}"
 
# Register
MACreg=""
 
# If not run as root, bail
[ "$(id -u)" != "0" ] && echo "You need root privileges to modify networking parameters. Exiting." 1>&2 && exit 2
 
getmac(){
# sets MAC register by IP. Sets to nil, if the MAC is not on the local link. 
  getMAC=$(ip neigh show ${1} | awk '{print $5}')
  if [ -z "${getMAC}" ]; then
    arping -c1 -I ${DEV} ${1} > /dev/null 2>&1
    getMAC=$(ip neigh show ${1} | awk '{print $5}')
  fi
  MACreg=${getMAC}
}
 
allow(){
  # Whitelists traffic to and from particular IP+MAC pairings and
  # adds them to static ARP.
  IP=${1}
  MAC=${2}
  if [[ -n "${IP}" && -n "${MAC}" ]]; then
    arptables -A INPUT  -s ${IP} --source-mac      ${MAC} -j ACCEPT
    arptables -A OUTPUT -d ${IP} --destination-mac ${MAC} -j ACCEPT
    ip neigh replace ${IP} lladdr ${MAC} nud permanent dev ${DEV}
  fi
}
 
if [ -n "${DEV}" ]; then
  # whitelist the routers
  test -z ${GATEWAYS} && GATEWAYS=$(ip route show dev ${DEV}| sed -n "s:.* via \(${IPREGEX}\).*:\1:p")
  for GWIP in ${GATEWAYS}; do
    MACreg=""
    getmac ${GWIP}
    allow ${GWIP} ${MACreg}
  done
  # whitelist the DNS servers
  test -z ${DNSSERVERS} && DNSSERVERS=$(sed -n "s:^nameserver \(${IPREGEX}\):\1:p" ${RESOLV})
  for DNS in ${DNSSERVERS}; do
    MACreg=""
    getmac ${DNS}
    allow ${DNS} ${MACreg}
  done
  # if using dhcpcd, we can whitelist the DHCP server too
  test -n ${DHCPSID} && getmac ${DHCPSID} && allow ${DHCPSID} ${MACreg}
  # set default policy to DROP    
  arptables -P INPUT DROP
  arptables -P OUTPUT DROP
  # clear out non-hardcoded ARP cache entries
  ip neigh flush nud reachable
  ip neigh flush nud stale
else
  # No argument given, so clean up.
  arptables -F
  arptables -P INPUT ACCEPT
  arptables -P OUTPUT ACCEPT
  ip neigh flush nud permanent
fi

This happens to me regularly. Someone brings a machine along and I want to display some app, running on my machine, on their display. Networked X11 to the rescue, you say? No, their X11 server is started with ‘-nolisten TCP’ wich is the default on most modern Linux distros. Sadly, the TCP socket can’t be enabled ‘in-flight’ — if you decide you do fancy a TCP socket after all, you’ll have to restart your X server which may be a pain if you’re in the middle of something (besides, restarting is just plain uncool).
But there is a way to expose the Unix domain socket as a TCP socket, with the help of socat. The following examples all use bash, so if you run a different shell (if you don’t know, you probably aren’t) you may need to define environment variables differently.

Braindead Proof of Concept (BPOC)

Situation: You want to display an application running on a machine called w00t on another machine, called bling. There’s an X11 server running on bling, but it’s not configured to listen on any TCP socket. DNS is properly setup, so if you ping w00t from bling, you get replies from bling’s IP, and vice versa.

1. On bling, find the domain socket of bling’s X11 server. Have a look in /tmp/.X11-unix/. The socket’s name usually reflects its X server display number (which you can determine by running echo $DISPLAY in an xterm).
2. On bling, run something along the lines of
   socat TCP-LISTEN:6066 UNIX-CONNECT:/tmp/.X11-unix/X0
   This will open up TCP port 6066 on all of bling’s network interfaces, connecting it to the Unix domain socket of the X server.
3. In an xterm on bling, run xhost +. You have now opened up your X11 server to the whole wide world, a silly thing to do. Anyone with access to the TCP socket can now read your keystrokes, read your window contents, click your mouse buttons…
4. In an xterm on w00t, run DISPLAY="bling:66" xclock. You may have noticed that 66 = 6066 – 6000 and indeed, by convention the TCP port number for a certain display is its display number + 6000. Anyhow…. yay, a clock! It’s displayed on bling, but running on w00t.

Improvements

• You may have noticed that in the BPOC, you can use the display on bling only once. socat will allow only one client, and will exit once that client exits. In some situations, you may consider that a feature (it’s a one-time access grant), but in others you may not. If you want a reusable TCP socket, run something along the lines of
  socat TCP-listen:6066,fork,reuseaddr UNIX-CONNECT:/tmp/.X11-unix/X0 which forks off a socat process for every TCP connection.
• You may not want to expose a TCP socket on all interfaces. Maybe you only want to expose a socket on the LAN interface, or on the localhost interface (and wrap the packets in an SSH tunnel). Well, you can, using the ‘bind’ option:
  socat TCP-LISTEN:6066,bind=localhost UNIX-CONNECT:/tmp/.X11-unix/X0
  Now tunnel it over SSH. On w00t, run ssh -L 6011:localhost:6023 bling. Now localhost:6011 on woot is actually localhost:6023 on bling which is actually /tmp/.X11-unix/X0 on bling. So on w00t you can start an xclock with its display on bling by running DISPLAY="localhost:11" xclock.
• xhost + from the BPOC is braindead indeed. There are a couple things you could have done instead, there are good ways of tightening up your authorization scheme.
  • First off, you don’t really need to run xhost + if you properly set up X11 cookies, which you should. Here are some examples on using the xauth scheme, but take note: xauth generate will probably not work on recent X11 releases since the XSECURITY extension is disabled by default. Just use the same cookies on the client and the server.
  • Run xhost +w00t. That’s host-based authentication, which is stupid, but not as stupid as no authorization at all. Any user on w00t can now connect.
  • Suppose that on bling (of course!) you’d run xhost +SI:localuser:theuser with ‘theuser’ being the userID of the unix-user running the socat instance. Now from the point of view of the X server, any client connecting through socat will be coming from ‘theuser’ and will therefore be allowed access. Entertaining, but not much different from just running xhost +. It is something to keep in mind though! Many distros by default add the unix-user that started the X server to the authorization list. That user does not need a cookie. If you run socat as that user you will have the effect of running xhost + even if you run xhost -.
  • Just run a nested X11 server, such as Xnest or Xephyr. This way you put untrusted users in a sandbox, preventing them from snooping your keyboard and windows. It’s the X11 equivalent of a chroot.

Select – Copy – Positionyourcursor – Paste

Say I’d like to copy some text from some window (which may or may not have the input focus) into some other window (which also may or may not have input focus). This action is commonly called ‘copy-paste’. Copy-paste. That sounds like two steps, doesn’t it?
Let’s say both windows are visible on the current desktop. Outside the of the X11 world – say, on a Windows machine, you’d have to undertake the following steps:

1. Drag-select the text in the first window.
2. Press ctrl-c or click edit/copy.
3. In the second window, click the position you where you want the text to be inserted.
4. Paste the text with ctrl-v or edit/paste.

This is not “copy-paste”. This is “select-copy-positionyourcursor-paste”. Here’s how you do it on X11:

1. Drag-select the text in the first window.
2. Hover over (you don’t need to click[*]) the position you want the text to be inserted at in the second window and paste by clicking with the middle mouse button.

There. Copy-paste. There are two atomic actions involved with that. You can’t get this down to less than two. You need to specify what you want to copy, and you need to specify where you want to paste it. The X11 engineers understood this. I entertain the thought that they must have valued my time as well, put two and two together, and that this why we have the X selection buffer on X11[**] that makes copy-paste really copy-paste.
(Read the footnote. I will show that not all is smooth in X11-land with respect to clipboards.)

Window focus

Whenever I venture out of X11-land there’s something else I dearly miss. It’s the ability of an inactive window to receive cursor input events when the cursor is on top of it. It’s immensely useful. Consider the following scenario.
Let’s say you’re browsing the web and you stumble upon a page which you want to discuss with a friend. It’s a long page, so there’s going to be some scrolling. You open up an instant messaging window to chat with your friend. You’re short on screen real estate so the IM window partly occludes the browser window. Meaning the IM window is on top of the browser window.
What you want to do now is to scroll occluded pieces of webpage text into view. But you also want to continue to see what you’re friend is saying. On Windows, you simply cannot. If you want to scroll the browser window, you’ll have to activate it, which means it will be raised, which means it will be above your IM window, which means you cannot see what your friend is typing. On X11, you can[*]. If your mouse cursor is over the browser window you can use the scroll wheel to scroll text into view, without the browser window being raised. Meaning your IM window is still on top, meaning you can still see what your friend is telling you. In fact, the IM window still has input focus so you can scroll the browser window and continue typing messages to your friend, limited only by your manual dexterity.
[Update: Added video to illustrate Windows behaviour][***]

Extravagance

If you think these situations are exotic, here’s and extravagant example for you. Open a file browser (Windows Explorer) window. Expand some folder trees in the left pane until you get a scroll bar in this pane. Navigate to a folder which has a lot of files (C:\Windows will do nicely) so the file pane also receives a scroll bar. You now have two panes, both with a scroll bar, within the same application. Thing is, you can only scroll in one of them at the time! If you want to look around in a “non-active” folder pane, you first have to click it. But don’t just click anywhere! You have to take special care to click it somewhere that doesn’t change your view in the file pane — better not click (near) one of the folders! Now suppose you’d like to scroll some files in the file pane into view. Better “activate” the file pane first then. Again, take extra special care: don’t just click anywhere in the file pane because then you may lose any previously made selection. It’s perverse, it really is. This is all happening inside one single application window. And the folder pane actually does notice when I hover over it (it underlines folders) so why can’t I scroll the view, then? Perverse. The OS is wasting my time.

Sloppy focus on Windows

I remember activating this on Windows 95 and it appears it still works. You can have crude ’sloppy focus’ on Windows. Just hex-edit your UserPreferencesMask in the Registry (Half of the times that I’m doing anything remotely interesting on Windows, I find myself entering hex values. Why is that.).
Refreshing as this may be, this will not help you with any of the above scenarios:

• In the copy-paste scenario, you still can’t specify insertion position and insert into that position in one single action.
• In the IM-while-browsing scenario, your IM window loses input focus when you scroll the web page in the browser window.
• Windows Explorer cursor behaviour stays just as perverse as it was with the standard click-to-focus model.

But hey, at least Windows users have some choice (provided they understand regedit.exe and hex). On OS-X, you don’t have a choice at all because Apple understands usability better than you do. For instance, Apple knows you head will explode if they’d give you the option of resizing your application windows by pulling any border or corner. You know very well that you will only want to resize your windows by grabbing the bottom right corner and will thank Apple for protecting you against yourself. (No, you’re not allowed to rebut until you’ve read and understand this mountain of insight and humour.)

Footnotes

[*] Actually, this is dependent on the focus policy. X11 itself does not specify the focus policy. X11 strives to provide mechanism, not policy. Policy is implemented by the window manager, and there are many window managers available for X11. A couple of them provide a ‘click to focus’ policy. Many of them provide ’sloppy focus’ and/or ‘focus follows mouse’ policies. This page has some concise definitions and elaborates on focus problems encountered the different models.

[**] Most window managers also have a ‘clipboard’, running in parallel with the X selection buffer. The clipboard only holds stuff you put there explicitly, whereas the X selection buffer holds whatever text you last selected anywhere in the X11 session. Furthermore, both clipboards usually only accept character data. So if you select and copy some pixels in drawing program A, you can’t paste them in drawing program B. Program A will have a private clipboard to hold pixel data, and B has one of its own, too. If you stay within the realm of one particular desktop environment with applications specific to this desktop environment, you may actually be able to use the clipboard inter-applicationwise for both text and binary data. But in general, you can’t. This plurality confuses the hell out of newcomers.

[***]A video demonstrating the Windows scroll focus issue: