Smörgåsbord

For a little while I’ve been running a DIY dynamic DNS service. My peers think it’s convenient and pretty sweet, so in this post I’ll set things straight — a full disclosure on the mess that it really is ;-)

What is a dynamic DNS service?

Since the dial-up days of yore I have occasionaly been using “dynamic DNS” services. What is a “dynamic DNS” service, you ask? Well, it’s a marketing term, not a technical term, but most providers of such services allow for having some CNAME or A record point to an IP, and this pairing is updatable over HTTP. Usually they use a freemium business model where you can get a couple of subdomains on their second level domains.

Why use one?

An example of a use case: You run a development server on your laptop. You want someone to connect to it. Instead of saying to this person “now just go to… errr… hold on” (and now you pop your shell and run ip a s dev wlan0)¹ “yes I’m back, you need to go to 110.34.56.250″ you can just say “go to kleinebeer.ath.cx”. Much easier. And that’s with IPv4. Typing an IPv6 address that someone just yelled at you from across the room is an even less joyful experience.
To enable this use case you configured your machine in such a way that every time your interface aquires a new IP, a program contacts the “dynamic DNS service” to update the ‘kleinebeer’ DNS record to reflect this IP. There are many clients for such services out there. I have even seen domestic ADSL modem/routers on which the factory firmware contains such a client.

Doing it differently

But some time ago I got fed up with all the “account expiration warning” emails I had been getting from the service that I used. And since I’m quite familiar with the components you need for building such a service yourself, I set out to do so. The thing I ended up with is for personal use, specific to my needs, KISS, and sinful.

Sins!

It allows for instant creation/updating of DNS records through a very simple HTTP GET. That is sinful for two reasons:

1. No authentication and authorization.

   Anyone can create any record. And anyone can update any record he or she knows the name of. You understand how this could be a bad thing. Don’t have such names as MX records unless you don’t mind that mail intended for you ends up on someone else’s server.
   At the same time it is a good thing, as I will show in the use case example below.

2. The HTTP GET alters state.

   And that’s just plain Bad Taste. If you’re designing a web service you’ll usually try to make it RESTful. GET should be a safe method. But for this application, at the frontend, it is actually desirable — because grandma can GET, as you will see in the use case below.

Use cases

• Case A:

  Imagine you’re on the phone with your grandma, and after discussing apple pie recipes something else comes up and you need to know her IP (let’s say you need to SSH in and help her with her crontab). She hasn’t registered with any dynamic DNS service, she has a hard time determining her IP, she has a hard time reading it out loud without making mistakes, and you are fed up with typing IPs anyway. But she has a web browser. Wouldn’t it be nice if you could just tell her to open her browser and head over to http://grandma.reg-a-record.tld/reg.php ? Nothing to install, nothing to look up, and something reasonably simple for grandma to do. The script on http://grandma.reg-a-record.tld/reg.php creates a DNS A record named ‘grandma’ on dyndomain.tld, and this record points to your grandma’s IP. Now you can just do ’ssh grandma.dyndomain.tld’.

• Case B:

  Imagine you and your friends are on a coding binge, all running their Django/Rails/Node.js/Thing-du-jour instances on laptops, in the same room, and you all want to connect to each other’s development servers.
  You could all call out your IPs across the room and let everyone type in everyone else’s IPs. Now you all have N tabs open in your browsers. Who was 123.231.101.45 again?
  Or, everyone could just point their browser (or curl, wget…) to http://theirfirstname.reg-a-record.tld/reg.php² and be done with it. If you want to view Fred’s progress you just go to fred.dyndomain.tld.

So, sin #1 & #2 allow for use cases that were not possible with the provider I was using.

You don’t have to register the records up front with some service provider. You can make up records as you go, and you want to be able to let anyone create any record. Hence sin #1. But it’s a good thing. You don’t have to reuse records, which is better for your privacy. With the commercial service I could only use a couple of names, so anyone whom I once had told to go to kleinebeer.ath.cx could kinda virtually follow me around and determine whether I was at work, home, uni or girlfriend by resolving this record. When creating new records is incredibly easy, there’s no reason to keep using the same name every time.

On to the code

It’s very, very simple! You need very little code. Things can be made much simpler than I have done. For byzantine reasons I use two webservers, one has a python CGI which creates the actual records, and the other one runs the public-facing PHP script (through mod_php) which you actually connect to.

DNS server

You need a domain for which you run the authoritative name server. I’m using the SheerDNS DNS server package on mine, for no good reason. Here’s a patch I made so that sheerdnshash creates directories with some group permission bits set, something I need in my setup.
You can use other DNS servers, of course. I discovered that the Unbound DNS server has a socket protocol which you can possibly use to add records on the fly. But I use sheerdns now, and here’s a Python module I made to write A-records in sheerdns’s directory structure. Public domain, use it as you see fit. You need to modify the constants to reflect your config, and you can test it by running the module from a shell.

#!/bin/env python
 
import socket,os,string,subprocess,sys
 
SHEERDNS_DIR="/var/sheerdns"
SHEERDNS_HASH="/usr/sbin/sheerdnshash"
DYNDOMAIN="dyndomain.tld"
ALLOWCHARS=string.ascii_letters + ''.join([ "%d" % i for i in range(10)]) + '-'
 
def mk_A(name,ip):
	#are host and ip allright?
	host = ''.join([ c for c in name if c in ALLOWCHARS ]).lower()
	if not host:
	  return None
	try:
	  socket.inet_aton(ip)
	except socket.error:
	  return None
 
	fqdn = "%s.%s" % (host, DYNDOMAIN)
 
	#set the umask
	oldumask = os.umask(0002)
 
	#get the hash
	hash = subprocess.Popen([SHEERDNS_HASH, fqdn], stdout=subprocess.PIPE).communicate()[0].strip()
 
	#construct the dirpath
	path = os.path.join(SHEERDNS_DIR,hash,fqdn)
 
	#chmod it for everyone to read
	os.chmod(path,0755)
 
	#construct the A-record-filepath
	apath = os.path.join(path,'A')
 
	#open the record file
	with open(apath,'w') as record:
		record.write(ip)
 
	os.umask(oldumask)
	return name
 
if __name__ == '__main__':
        if len(sys.argv) == 3:
          print(mk_A(sys.argv[1],sys.argv[2]))

Web ’service’ to create the records: a CGI

This is a Python CGI which uses the above module. I stick it behind HTTP Basic authentication which is handled by the web server. You call it like this: http://server_that_has/the_CGI?name=therecordname&ip=theip .
There’s no reason that this CGI should be guilty of Sin #2, except laziness on my behalf. If I’d be following every convention even for silly little solo projects I’d never get anything done, all right?
It doesn’t propagate any errors from the sheerdns library, but it prints the name as it is registered — stripped of disallowed characters, or nothing if something failed along the way (invalid IP, for instance).

#!/usr/bin/python
import sheerdns
import cgi
 
print "Content-type: text/plain\n\n";
params = cgi.parse()
name, ip = params.get('name'), params.get('ip')
 
if ip and name:
  dnsname = sheerdns.mk_A(name[0],ip[0])
  print dnsname

The accessible part: reg.php

This is in PHP. It lives in the default virtual host of my Apache config, and for good reasons. I have a CNAME record that points *.reg-a-record.tld to this web server. I then use whatever is in * to construct the name for the dyndomain.tld-record. The wildcard record is essential.
You need to modify this script to reflect your config.

<?php
header('Content-Type: text/plain');
$hostparts = explode('.', $_SERVER['SERVER_NAME'], 2);
$ip = $_SERVER['REMOTE_ADDR'];
print $ip;
 
$cha = curl_init('https://server_that_has/the_CGI?name='.$hostparts[0].'&ip='.$ip);
curl_setopt($cha, CURLOPT_SSL_VERIFYPEER, False);
curl_setopt($cha, CURLOPT_TIMEOUT, 5);
curl_setopt($cha, CURLOPT_RETURNTRANSFER, True);
curl_setopt($cha, CURLOPT_USERPWD, 'bite:me');
$resp = curl_exec($cha);
curl_close($cha);
print($resp);
?>

Presto

If you visit http://kitten.reg-a-record.tld/reg.php, the IP address that the web server sees you coming from gets registered as kitten.dyndomain.tld. If everything went well, the script responds with this mapping.
You could set DirectoryIndex reg.php here, or rename reg.pgp to index.php, if you don’t want users to type ‘reg.php’. I did make it explicit, because web sites on this server come and go and I don’t want people to stumble onto reg.php every time they hit the default vhost because their web site doesn’t exist any more.

1)Let’s assume you’re not NATed.
2)Try to only have friends with distinct names that can be expressed in ASCII.

When SQLite is not enough

SQLite is great for ad-hoc SQL DB-ing, but it’s not so great if you need to serve multiple processes (on writing, it locks the complete database).

It just so happens that, for a certain project, I need the ad-hocishness of SQLite *and* multiprocess write concurrency.

It appears that PostgreSQL can easily be made to run in “hassle-free” mode. Meaning: no need to ask your mum or your sysadmin anything, no need to write any config files.

Minimalism

Well, almost. I put in some extra options to disable the TCP socket, and I set some permissions on the unix socket.
For these examples, I assume bash or zsh and a unixish system. And PostgreSQL 9.0.

create a database:

initdb ~/mypostgres
You now have a directory ‘mypostgres’ in your homedir. It contains some config files with defaults. We don’t care.

start a server:

postgres -D ~/mypostgres/ -k ~/mypostgres/ --listen_addresses='' \
--unix_socket_permissions=0660 --unix_socket_group=$(id -g)
Let’s go over these options:

• -D ~/mypostgres/ — Use ~/mypostgres as datadir (it’s where you created the database, so.)
• -k ~/mypostgres/ — Use ~/mypostgres as the directory to store the unix socket for clients to connect to. Look closely and you’ll discover it (it starts with a dot for no good reason). Put it somewhere else (/tmp springs to mind) if you want other users to connect to your db, more on that later.

These are not strictly necessary:

• --listen_addresses='' — Don’t listen on any TCP sockets. We don’t really authenticate users. So if you expose a TCP socket (default, on the localhost interface) any user can do anything to your databases just by running something like psql -h 127.0.0.1 -d postgres -U yourusername
• --unix_socket_permissions=0660 — By default, permissions are o=rwx. If you put the socket in a public place such as /tmp, then with our authentication-free mode of operation, you probably do not want any and all users to be able to access your DB.
• --unix_socket_group=$(id -g) — This is completely superfluous but it’s here so you don’t shoot yourself in the foot when blindly copy-pasting teh codez. $(id -g), of course, is your primary group ID and is what the group ownership of the socket will have been set to without this specification anyway (hence the superfluousness). So when would you use this? You’d use it when you want to let other users access your DB. You’d set the gid to some appropriate group that you and the other users belong to (and you’d put the socket in a public place, that much should be clear now).

some goodies:

• --silent-mode=true — Daemon mode. To quit the server, get the PID from ~/mydb/postmaster.pid and send it a SIGTERM.
• -F — Wicked fast writes mode. Also, wickedly trashed DB mode if the server is interrupted abnormally.

Create and connect

Now the server is up and running, create a database:
createdb -h ~/mypostgres mydb
and connect to it:
psql -h ~/mypostgres -d mydb
By default psql will try to log you in to the DB using the same username as your current unix username. Convenient, because that user has full privileges on the DB.
Should you need to specify the user as whom to connect, you can do so with -U.

Wrapper?

Just like with sqlite, you determine access to the DB by setting unix permissions. We do it on the socket, with SQLite you do it on the DB file itself.
So, how about a wrapper Python module to make all this just as easy as

import sqlite3
conn = connect("create_me_if_I_don't_exist.db")

Yes, one day. Fork off a db server process, terminate it using the atexit decorator. It just might be that simple.

A while ago I received a question from a reader. He was having some trouble with a pretty interesting setup. As I was rather intrigued I spent some time to figure it out — it’s solved, but I’m not 100% sure on the theory. So if you are, please comment.

the question

from Yves <theYinYeti@yeti.selfip.net>
to wicher@gavagai.eu
date Wed, Sep 15, 2010 at 16:06
subject Socat

Dear mister Minnaard,

I hope you don’t mind me asking you for help. By chance, I saw your blog post:
http://smorgasbord.gavagai.nl/2010/01/socat-access-your-x-servers-domain-socket-over-tcp/

It happens I have a problem with socat on my Sheevaplug server, and it seems you have good knowledge of this software.

Following http://zarb.org/~gc/html/udp-in-ssh-tunneling.html, I use socat to tunnel DNS through SSH, so as to have transparent DNS resolution, to be used with a transparent proxy (NAT rules + tinyproxy, backed by cntlmd, backed by corporate proxy).

My problem is that the number of socat PIDs on both the client and the server keeps growing. I’ve had floods of “accept: too many open files” messages, and I suspect socat is the culprit (at this point, a ps -ef shows almost a thousand of socat PIDs!); especially since the transparent DNS stops working after this flood of messages.

More specifically, on the client, I run:

• dnsmasq with “server=127.0.0.1#1053” as a back-end

• socat -ly udp4-listen:1053,reuseaddr,fork tcp:127.0.0.1:1054

• ssh -L 1053:208.67.222.222:53 -L 1054:127.0.0.1:1054 \
  -Tfx remoteuser@remoteserver \
  'socat -lf /dev/null tcp4-listen:1054,reuseaddr,fork \
  UDP:208.67.222.222:53'

  (208.67.222.222 is OpenDNS)

Besides, on localhost (local client), ssh is configured this way:

Host remoteserver
Port 443
ServerAliveInterval 60
ProxyCommand /usr/bin/proxytunnel -v -p corporateproxy:8080 -P login:passwd -d %h:%p

Do you have an idea on how I could make socat behave better?
I thank you beforehand for any advice you could provide.
Sincerely,

Yves.

the reply

from Wicher Minnaard <wicher@gavagai.eu>
to Yves <theYinYeti@yeti.selfip.net>
date Thu, Sep 16, 2010 at 02:19
subject Re: Socat

Dear mister Yves,

On Wed, Sep 15, 2010 at 16:06, Yves <yves@yeti.selfip.net> wrote:

Dear mister Minnaard,

I hope you don’t mind me asking you for help. By chance, I saw your blog post:
http://smorgasbord.gavagai.nl/2010/01/socat-access-your-x-servers-domain-socket-over-tcp/

It happens I have a problem with socat on my Sheevaplug server, and it seems
you have good knowledge of this software.

Thank you for your compliment. Actually, I do not know socat all too well, but I know a thing or two about networking.

Following http://zarb.org/~gc/html/udp-in-ssh-tunneling.html, I use socat to tunnel DNS through SSH, so as to have transparent DNS resolution, to be used with a transparent proxy (NAT rules + tinyproxy, backed by cntlmd, backed by
corporate proxy).

Sounds like just another day at the office ;-)
Interesting setup.

My problem is that the number of socat PIDs on both the client and the server keeps growing. I’ve had floods of “accept: too many open files” messages, and I suspect socat is the culprit (at this point, a ps -ef shows almost a thousand of socat PIDs!); especially since the transparent DNS stops working after this flood of messages.

• dnsmasq with “server=127.0.0.1#1053” as a back-end

• socat -ly udp4-listen:1053,reuseaddr,fork tcp:127.0.0.1:1054

• ssh -L 1053:208.67.222.222:53 -L 1054:127.0.0.1:1054 \
  -Tfx remoteuser@remoteserver \
  'socat -lf /dev/null tcp4-listen:1054,reuseaddr,fork \
  UDP:208.67.222.222:53'

  (208.67.222.222 is OpenDNS)

Sidenote: Google provides a solid DNS service nowadays. I don’t know what they do with the data, though the same could be said about OpenDNS.
See http://code.google.com/speed/public-dns/


Host remoteserver
Port 443
ServerAliveInterval 60
ProxyCommand /usr/bin/proxytunnel -v -p corporateproxy:8080 -P login:passwd -d %h:%p


So it’s DNS over UDP over TCP over SSH over proxied HTTPS, or something. I’m losing track of all the layers here.

Do you have an idea on how I could make socat behave better?
I thank you beforehand for any advice you could provide.

Well you got me curious now.
I managed to reproduce your problem in a simplified setup:

- on host w00tage, setup the nameserver façade:
socat udp4-listen:1053,reuseaddr tcp:localhost:1054

- from host w00tage, tunnel the TCP connection over SSH to the host hosting the to-be-proxied nameserver:
ssh -L 1054:localhost:1054 priv

- now, on this host priv, connect the TCP socket to the UDP socket where the nameserver is listening
socat tcp4-listen:1054,fork,reuseaddr UDP:localhost:53

- on w00tage, do queries on localhost:1054:
dig @localhost -p1053 google.fr

Dig is part of the bind-tools package and greatly helps in diagnosing DNS problems.
The first dig will probably return. Subsequent digs may take seconds to return. And they leave stray socat processes behind on both w00tage and priv. Kill the strays on w00tage, and the ones on priv dissolve. So the local ones (on w00tage) tie up the remote ones. The latter are not the problem, just a symptom. You can keep on querying, doing more digs, until your file descriptors run out.

A stab at what is going on here: UDP is stateless, and socat is ignorant of protocol data (as it should be). A DNS query takes one UDP packet. But socat does not know we’re doing DNS. And since UDP is stateless, there’s no ‘bye’ in the conversation. So socat doesn’t know when to stop waiting for more. Dig does, though – it recognizes a DNS answer when it sees one. So dig returns (but why the delays? socat buffers and timeouts maybe?) but socat doesn’t. It’s still waiting for more.
Now since we’re forking socat for every new independent socketpair opened (which is what happens when you do a query), we get a lot of those hanging socats.

I had a look at the URL you sent me, http://zarb.org/~gc/html/udp-in-ssh-tunneling.html, to see how they are getting it to work and it appears there is a ‘UDP-RECVFROM’ address type in socat. From its manpage:
————
UDP-RECVFROM: Creates a UDP socket on [UDP service] using UDP/IP version 4
or 6 depending on option pf. It receives one packet from an
unspecified peer and may send one or more answer packets to that peer.
This mode is particularly useful with fork option where each arriving
packet – from arbitrary peers – is handled by its own sub process.
This allows a behaviour similar to typical UDP based servers like
ntpd or named. This address works well with socat UDP-SENDTO address
peers.
————

Hey, explicit mentioning of nameservers there. Receive one packet, don’t wait for more, send it off, collect answer, send that back.
On host w00tage, I run the façade thusly:

socat udp4-recvfrom:1053,fork,reuseaddr tcp:localhost:1054

And that appears to work indeed. But wait, how does socat (on priv) know when to stop trying to collect answer packets?
I think it’s socat timeouts, but I’m not 100% sure. What leads me to think that it’s timeouts is that when I stress-test this setup from w00tage:

while true; do dig @localhost -p1053 sheeva; done
(for ’sheeva’ the nameserver never has to do recursion)
I have about 30 forked-off query-listeners when I do a pgrep socat | wc -l on w00tage. But I have about 400 on priv. That number goes up and down and the pids change, so they do not *accumulate* but rather seem to hang about until they decide their job is done. But to be sure I’d have to study the source. Or ask the socat mailing list.

Anyway, problem solved!

I was happy to help you and I got some exercise out of it. Any reason why you didn’t post this question as a comment on my blog? Maybe it would be a bit off-topic, I can see that.
Would you mind if I published this?

Regards and good luck, Wicher Minnaard.

it works

from Yves <theYinYeti@yeti.selfip.net>
to wicher@gavagai.eu
date Thu, Sep 16, 2010 at 10:01
subject Re: Socat

Dear mister Minnaard,

All in all, I was just careless in adapting the “alternative solution with socat” to my setup.

I thank you for your analysis and tests; I learnt a few things following them. And I certainly wouldn’t mind at all if you were to publish this.
However, if you do so, please change the e-mail to “theYinYeti@yeti.selfip.net” (the Sheevaplug :-) ), and my name to simply “Yves”; thank you. As for the web site (I see there’s a field for it), it is “http://yeti.selfip.net/gablin.php” (a bit lame, but I may find time someday to update it…)

As you said, I had the feeling this was a bit off-topic. But now, I see how this can shed some light on advanced socat usage.
–
Regards,

Yves.

Nice hardware, but no channel 13

Last week I got myself a TP-Link TL-WR1043ND and pressed it into service as a smart 5-port gigabit switch (with VLAN capabilities) and wireless-N bridge. It is a fine piece of hardware and unbelievably cheap. Best of all, you can install the OpenWRT linux distribution on it. OpenWRT doesn’t target the inexperienced, but it is very customizable, offers a fair amount of documentation, and the build system seems very worthy of giving it a spin.

However. I noticed I couldn’t select a wireless channel >11. That’s odd, because over here in Europe we’re allowed to use the frequencies of channel 12 and 13, too.

Here’s what dmesg told me:

cfg80211: Calling CRDA to update world regulatory domain
cfg80211: World regulatory domain updated:
    (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
    (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
    (2457000 KHz - 2482000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
    (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
    (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
    (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
ath: EEPROM regdomain: 0x0
ath: EEPROM indicates default country code should be used
ath: doing EEPROM country->regdmn map search
ath: country maps to regdmn code: 0x3a
ath: Country alpha2 being used: US
ath: Regpair used: 0x3a
phy0: Selected rate control algorithm 'ath9k_rate_control'
phy0: Atheros AR9100 MAC/BB Rev:0 AR2133 RF Rev:a2 mem=0xb80c0000, irq=2
cfg80211: Calling CRDA for country: US
cfg80211: Regulatory domain changed to country: US
    (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
    (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2700 mBm)
    (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 1700 mBm)
    (5250000 KHz - 5330000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
    (5490000 KHz - 5600000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
    (5650000 KHz - 5710000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
    (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 3000 mBm)

So why does the system think I am in the US? And how can I tell it that I’m actually in the Netherlands? At first glance, it should be as easy as iw reg set NL. Result:

cfg80211: Calling CRDA for country: NL
cfg80211: Regulatory domain changed to country: NL
    (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
    (2402000 KHz - 2482000 KHz @ 40000 KHz), (N/A, 2000 mBm)
    (5170000 KHz - 5250000 KHz @ 40000 KHz), (N/A, 2000 mBm)
    (5250000 KHz - 5330000 KHz @ 40000 KHz), (N/A, 2000 mBm)
    (5490000 KHz - 5710000 KHz @ 40000 KHz), (N/A, 2700 mBm)

Looks about right, doesn’t it? Because 2482000 > 2472000. 2472 KHz is the upper bound of the 22 KHz wide channel 11. Likewise, 2482 KHz is the upper bound of channel 13. So am I all good for channel 13 now? Turns out I am not. iwlist wlan0 channel says:

wlan0     11 channels in total; available frequencies :
          Channel 01 : 2.412 GHz
          Channel 02 : 2.417 GHz
          Channel 03 : 2.422 GHz
          Channel 04 : 2.427 GHz
          Channel 05 : 2.432 GHz
          Channel 06 : 2.437 GHz
          Channel 07 : 2.442 GHz
          Channel 08 : 2.447 GHz
          Channel 09 : 2.452 GHz
          Channel 10 : 2.457 GHz
          Channel 11 : 2.462 GHz
          Current Frequency=2.452 GHz (Channel 9)

The wherefore and the why. And the how.

No channel 13. Wut? I found this informative mailing list post detailing what’s happening here. Basically, the kernel first loads the rules for the ‘world’ domain. On planet Earth, no countries exist that allow you to go beyond these frequencies (for domestic wifi). But many countries have additional restrictions. In Japan, you can use channel 14, in Europe you can’t. You can use 12 and 13 though, which in the US you can’t.
The kernel (well, the “ath” driver) then proceeds to look for any hints from the hardware. In my case the value burned into the hardware EEPROM is 0×0. That’s a value, which, per the Atheros specs, should map to the US. The kernel calls a userspace program crda to get the list of allowed frequencies and signal strenghts for the US.
After this has been done, you can do iw reg set FOO until you drop but you wont be able to go beyond the limits set previously.

The weird thing with radio hardware is that not only the users are subject to regulation, the sellers are too. You can’t sell non-compliant radio hardware. Now, the wifi chip manufacturers do not physically limit a chip intended for sale in the US to channels 1-11. Rather, they mass produce chips and then the equipment manufacturers are supposed to configure the chip in compliance with its target market. (As if you’d never take your equipment to Japan).
As I understand it, TP-Link messed up here.

Compare the above setup with what happens on my laptop’s Atheros chip:

ath: EEPROM regdomain: 0x65
ath: EEPROM indicates we should expect a direct regpair map
ath: Country alpha2 being used: 00
ath: Regpair used: 0x65

Looking at the Linux 2.6.35 sourcecode in drivers/net/wireless/ath/regd_common.h you’ll see that 0×65 maps to WOR5_ETSIC, which holds rules for compliance with EU regulation. Which I suppose is right since I’ve bought that laptop in the EU. But my TP-link comes out as FCC3_FCCA ­— that’s FCC-land, the US.

Fixing it

So either patch the kernel’s mappings, or patch the userspace ‘crda‘ program to supply a modified list of allowed frequencies for the ‘US’. I chose the latter. Reading a bit more about this program and its database, it didn’t appear I could fix it easily — the database is digitally signed and the public key needs to be known to the crda program at compile-time. No problem on a Gentoo system, but the target system was OpenWRT. On a MIPS machine. That meant cross-compiling, so I proceeded by downloading the OpenWRT dev environment for the Backfire release, and had a look around. And whaddayaknow, package/crda/patches/101-make_crypto_use_optional.patch made me hope that signature checking wasn’t even compiled in! On my OpenWRT box:

root@t33t:~# ldd /sbin/crda 
	libnl-tiny.so => /usr/lib/libnl-tiny.so (0x2aabe000)
	libm.so.0 => /lib/libm.so.0 (0x2aad6000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x2aaff000)
	libc.so.0 => /lib/libc.so.0 (0x2ab1f000)
	ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x2aaa8000)

(Don’t you just love it). Now I don’t know a lot about embedded environments and uclibc, but I don’t see any libcrypto.so or libgcrypt.so in there. It looks like I can just supply any old unsigned database. More reading of documentation, some compiling of tools.
regdbdump /usr/lib64/crda/regulatory.bin gives me an editable version of the database. I didn’t feel like limiting myself, so I replaced this:

country 00:
	(2402.000 - 2472.000 @ 40.000), (3.00, 20.00)
	(2457.000 - 2482.000 @ 20.000), (3.00, 20.00), PASSIVE-SCAN, NO-IBSS
	(2474.000 - 2494.000 @ 20.000), (3.00, 20.00), NO-OFDM, PASSIVE-SCAN, NO-IBSS
	(5170.000 - 5250.000 @ 40.000), (3.00, 20.00), PASSIVE-SCAN, NO-IBSS
	(5735.000 - 5835.000 @ 40.000), (3.00, 20.00), PASSIVE-SCAN, NO-IBSS

country US:
	(2402.000 - 2472.000 @ 40.000), (3.00, 27.00)
	(5170.000 - 5250.000 @ 40.000), (3.00, 17.00)
	(5250.000 - 5330.000 @ 40.000), (3.00, 20.00), DFS
	(5490.000 - 5600.000 @ 40.000), (3.00, 20.00), DFS
	(5650.000 - 5710.000 @ 40.000), (3.00, 20.00), DFS
	(5735.000 - 5835.000 @ 40.000), (3.00, 30.00)

with this:

country 00:
  (2402.000 - 2494.000 @ 40.000), (N/A, 30.00)
  (4910.000 - 5835.000 @ 40.000), (N/A, 30.00)

country US:
  (2402.000 - 2494.000 @ 40.000), (N/A, 30.00)
  (4910.000 - 5835.000 @ 40.000), (N/A, 30.00)

and then I converted the database to its binary form with db2bin.py which one can find in the wireless-regdb source code releases. Copied the new database over to my OpenWRT machine, restarted it, dmesg, presto:

cfg80211: Calling CRDA to update world regulatory domain
cfg80211: World regulatory domain updated:
    (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
    (2402000 KHz - 2494000 KHz @ 40000 KHz), (N/A, 3000 mBm)
    (4910000 KHz - 5835000 KHz @ 40000 KHz), (N/A, 3000 mBm)
ath: EEPROM regdomain: 0x0
ath: EEPROM indicates default country code should be used
ath: doing EEPROM country->regdmn map search
ath: country maps to regdmn code: 0x3a
ath: Country alpha2 being used: US
ath: Regpair used: 0x3a
phy0: Selected rate control algorithm 'ath9k_rate_control'
phy0: Atheros AR9100 MAC/BB Rev:0 AR2133 RF Rev:a2 mem=0xb80c0000, irq=2
cfg80211: Calling CRDA for country: US
cfg80211: Regulatory domain changed to country: US
    (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
    (2402000 KHz - 2494000 KHz @ 40000 KHz), (N/A, 3000 mBm)
    (4910000 KHz - 5835000 KHz @ 40000 KHz), (N/A, 3000 mBm)

root@t33t:~# iwlist wlan0 channel
wlan0     14 channels in total; available frequencies :
          Channel 01 : 2.412 GHz
          Channel 02 : 2.417 GHz
          Channel 03 : 2.422 GHz
          Channel 04 : 2.427 GHz
          Channel 05 : 2.432 GHz
          Channel 06 : 2.437 GHz
          Channel 07 : 2.442 GHz
          Channel 08 : 2.447 GHz
          Channel 09 : 2.452 GHz
          Channel 10 : 2.457 GHz
          Channel 11 : 2.462 GHz
          Channel 12 : 2.467 GHz
          Channel 13 : 2.472 GHz
          Channel 14 : 2.484 GHz
          Current Frequency=2.452 GHz (Channel 9)

Legality

This is a rather libertarian approach. Not legal on Earth, it doesn’t even comply with the ‘World’ regulatory domain. Yes, among other things, this gives you access to the ‘Japanese’ channel 14. The rules don’t prevent you from going totally EMP on your neighbourhood. (Your hardware does, though).
It’s now YOUR responsibility to comply with local regulations. If you look out the window and see an airliner perform a barrel roll, you know it’s time to iw reg set $YOURCOUNTRYCODE.
And remember, interference works both ways. The fact that your neighbours cannot use some frequencies doesn’t necessarily mean you’ll get good performance using those (frequencies I mean). It may well be that you’ll have lousy reception because other (possibly more powerful) equipment is legally being used on those channels. Just stick with your country’s regulations OK?

I kept the previous ‘US’ entry as ‘UX’ so I can still comply with US regulations if necessary. I hear they’re big on regulations in the US. Especially if it involves foreigners.

Intro – running a cluster

So I had assembled this computing cluster thing based on execnet for a CS project. The slave nodes were dozens of student lab computers. Those are running firewalls blocking all ingress traffic, not just from the big bad internet but also from the lab networks — which contain only trusted machines and are already protected by a firewall restricting flow to and from larger networks!
Thing is, I need access to the SSH daemons running on those nodes. And I do not (and should not) have root on those machines…
Solving this problem on OSI Layer-8 proved to be an inefficient approach. The best way to fight bureaucracy is to not take it seriously. A bureaucracy (almost by definition) is not creative and cannot keep up with workarounds. To admit existence of creative, necessary workarounds to achieve legitimate ends is to admit defeat and that is why workarounds usually get ignored. It’s ignore and counter-ignore which is a bit sad since there is (or should be) a common goal: education, in this case. But I digress. The solution proved to be to ignore layer 8 and solve the problem on OSI layer 4. That certainly saved a lot of keystrokes!
To punch through the firewall, I have each slave node initiate a reverse tunnel (ssh -R) to a central node, using pubkey authentication. This way the otherwise unreachable SSH daemons on the slave nodes are proxied on the master node and are reachable via a socket on the master nodes’ loopback interface. The slave nodes specify the reverse tunnel endpoint port based on their IP. They connect automatically, and with some clever but too-kludgy-to-show scripting I can dynamically update my running clusterfied program with newly joining slave nodes to export compute jobs to.
Obviously, the slave nodes need to login to the central node via SSH and do so using a single account on the central node. But I don’t fully trust the slave nodes. I want them to be able to initiate reverse tunnels, and not do much else. Here’s how I approached this problem, with the server running OpenSSH 5.3p1.

Chroot

Chrooting comes to mind. With a chroot, you can limit the filesystem locations the user can access to some remote uninteresting branch of the tree.
From man 5 sshd_config:

ChrootDirectory
Specifies a path to chroot(2) to after authentication.  This path, and
all its components, must be root-owned directories that are not writable
by any other user or group.  After  the  chroot,  sshd(8) changes the
working directory to the user's home directory.

The path may contain the following tokens that are expanded at runtime
once the connecting user has been authenticated: %% is replaced by a
literal '%', %h is replaced by the home directory of the user being
authenticated, and %u is replaced by the username of that user.

I set up a group for that and use the following ‘Match’ stanza in the central node’s sshd_config:

Match Group chrootage
        ChrootDirectory %h

%h is the matched user’s homedir.
I set up a user named ‘chroot’ with a not-so-regular home directory. More on that home directory later on. This user needs a valid shell, even if it will never run that shell (I do not want them to run one). Without a valid shell the user will not be able to log in.

ForceCommand

From man 5 sshd_config:

ForceCommand
Forces the execution of the command specified by ForceCommand, ignoring
any command supplied by the client and ~/.ssh/rc if present. The command
is invoked by using the user's login shell with the -c option. This
applies to shell, command, or subsystem execution.  It is most useful
inside a Match block.
The command originally supplied by the client is available in the
SSH_ORIGINAL_COMMAND environment variable. Specifying a command of
``internal-sftp'' will force the use of an in-process sftp server that
requires no support files when used with ChrootDirectory.

Well that ought to suit my purposes. Ideally, this command would be a rather boring and useless one that just sits there waiting for input. I could copy cat and the libs it links to over to the chroot directory. I might as well make it the login shell for the chroot user. But the user would then still be able to use sftp. And if I would want to disable sftp, I would have to do so server-wide! Me and the other users use sftp extensively, so I’d then need to run a second SSH daemon with no sftp subsystems especially for this purpose. Hmmm.
Another approach is to limit the user to the internal sftp subsystem. Added benefit is that I do not need to maintain any binaries and libs in the chroot. With this, the user logging in with SSH is presented by a boring interpreter in which he cannot launch any programs — it’s the sftp subsystem. Portforwarding, which is needed for the reverse tunnel, will work. SFTP will work, obviously. Let’s do it.
sshd_config:

Subsystem       sftp    internal-sftp

Match Group chrootage
        ChrootDirectory %h
        ForceCommand internal-sftp

Make SFTP boring ­— chrooted home directory contents

So what do we need inside the chroot directory? No binaries or device nodes, that is one of the advantages of using the internal-sftp subsystem.
We do need an authorized_keys file readable by the chrooted user since this user won’t be able to log in using pubkey auth otherwise. We can’t use AuthorizedKeysFile inside a Match stanza, so the authorized_keys will have to reside in the default location — the .ssh subdirectory of the user’s homedir. I will use authorized_keys to limit the set of hosts this user can login from to the domain of the slave nodes, so obviously the user should not be able to add keys or lift limitations — authorized_keys will need to be read-only. This is what I ended up with.

#ls -Rla /path/to/chrooted_homedir
/path/to/chrooted_homedir/:
total 16
drwxr-xr-x 3 root root       178 May  8 16:36 .
drwxrwxrwt 9 root sys       4887 May  8 16:36 ..
dr-xr-x--- 2 root chrootage  189 May  8 16:22 .ssh

/path/to/chrooted_homedir/.ssh:
total 12
dr-xr-x--- 2 root chrootage  189 May  8 16:22 .
drwxr-xr-x 3 root root       178 May  8 16:36 ..
-rw-r----- 1 root chrootage 1221 May  8 14:25 authorized_keys

This is all the user can view over sftp. It is minimal and extremely boring. Note that there is no place the user can write to. Even if he would have some way of launching programs (he hasn’t, due to ForceCommand) there is no place to upload them to and launch them from in the first place.
On to authorized_keys:

no-pty,from="*.science.uu.nl" ssh-rsa AAAAB3and-the-rest-of-the-pubkey

no-pty serves to prevent the allocation of a terminal. I don’t think it offers any security beyond the measures already in place, but I like to think this saves resources.

Tunneling risks

I need TCP forwarding to get the reverse tunnels to work. That means that using the account, users can set up arbitrary TCP tunnels, using the master node as a proxy… Less nice. But wait – can’t I use PermitOpen inside Match to limit the creation of arbitrary tunnels? Yes. But the reverse tunnel is subjected to the same restrictions. Therefore, I would need a PermitOpen for every slave node and I’d need to know the source port of the incoming client connection in advance. There’s no wildcarding in PermitOpen, so this is infeasable.

‘Match’ directive precedence

The order appears to be significant. That’s also what the manpage seems to hint at where it says

Match:
Introduces a conditional block.  If all of the criteria on the Match
line are satisfied, the keywords on the following lines override those
set in the global section of the config file, until either another Match
line or the end of the file.

It’s not as clear as could be what happens to keywords in the next Match directive. What if the next Match directive matches? Nothing! The next Match directives don’t matter! In my tests, incoming connections appeared to be only matched once. The match blocks have no relation to eachother, only to the main config.
That implies that you need to put the most specific match directives — the ones with the most patterns ­— on top, and that each Match block needs to hold any and all keywords you want to specify beyond the main config. It’s not CSS ;-)

Your mum told you not to touch block devices containing mounted filesystems. But today we’ll be doing exactly that. Specifically, we’re going to use the Linux Device Mapper (DM) snapshot and snapshot-merge target. To do what? To do this:

1. Make all reads come from one device, A
2. Make all writes go to another block device, B
3. Merge the writes stored in B into A
4. Without unmounting the filesystem on A!

Why? Well, as pointed out in the comments on this LWN article you could use this as a way to roll back an unfortunate upgrade without taking your system offline. But if the upgrade _is_ fortunate, you might want to merge the changes back in without having to take your filesystem offline.

Or you could use it to speed up writes — perform those against a RAM-based block device, and merge those writes back in when sysload is low. Or use it like I’m planning on doing: to avoid the small but frequent writes (growing logfiles, mainly) to the CF card in my Alix2 which I’ve built my router with.

#1+#2 are old hat — but if you’ve never made a writable snapshot with DM (not LVM) I strongly suggest you read the “Right To Your Own Devices” LinuxGazette article. It’s what got me started.

#3 is fairly recent hat, for ‘kernel 2.6.33′ values of “recent”.

#4 is shiny new hat and is what this post will be about.

Prerequisites

You must be running the Linux kernel, 2.6.33+, and have device mapper support (CONFIG_DM_SNAPSHOT) loaded and ready to go. You also need a recent LVM2 userland, I used version 2.02.60.
Usually one goes about snapshotting with the use of LVM, the Logical Volume Manager. LVM is just an easy high-level way of employing DM functionality for common use cases. Ours is not one of those, hence we’ll be using raw dmsetup-foo. That’s why I strongly recommend you read the LinuxGazette article mentioned before. We’ll not be revisiting DM basics.

Get Dirty

• Create your base block device of about 500MB.

  dd if=/dev/zero of=hard bs=1M count=500
  losetup /dev/loop0 hard

  Neat, now you have a block device which is actually a file on your file system (which is on a block device). You could mkfs and mount it, but we’re not going to, not yet. We’re going to add yet another layer of indirection: the device mapper.

  echo 0 $(blockdev --getsize /dev/loop0) linear /dev/loop0 0 | dmsetup create hard_a

  You now have a /dev/mapper/hard_a which is a linear (not mirror, not stripe, not crypt, not anything else but plain boring linear) mapping of blocks to its underlying block device, which was /dev/loop0, which is backed by the file with name hard.

• Create a device which the writes will end up on. How large should it be? You decide. If you write more data than this volume can hold, it will be dropped and all your writes will be lost. That’s what you get for bad planning! Of course, you could err on the safe side and make it huge. But I don’t think it’s of any use to make it larger than the device whose writes you want it to receive (please correct me if I’m wrong). And by making it huge, you’re wasting space. Why not have it take up exactly the amount of space it needs? Dynamically growing, so to say? With a loop device backed by a sparse file, you can! A sparse file is a file whose empty bytes are represented by metadata. That means they don’t actually have to be there until needed, which is on write.
  Let’s make a 200MB sparsely backed device to hold the writes diverted from our other device:

  dd if=/dev/zero of=soft_a bs=1 count=0 seek=200M
  losetup /dev/loop1 soft_a

• Now we make yet another device: top. This is the device we’ll be making a filesystem on. But all writes will go into the file called soft. That’s called a writable snapshot and you can assemble it like this:

  echo 0 $(blockdev --getsize /dev/mapper/hard_a) snapshot /dev/mapper/hard_a /dev/loop1 p 8 | dmsetup create top

  You now have a /dev/mapper/top block device. Go ahead, mkfs it[*], mount it and stick some files on it. Interestingly, you can see how many blocks (in units of bytes) you have dirtied by looking at the physical size of the ’soft_a’ file:

  du -B1 soft_a

• Now for the magic part. Freeze the block device!

  dmsetup suspend top

  All reads and writes to /dev/mapper/top will now block. Processes that were accessing the device will be in suspended animation. Really? Depends. You will observe that you can still do an ls on the filesystem if you’ve run one before and haven’t changed any data in the meantime. That’s because of the kernel’s caches, in this case, the dentry (directory-entry) cache. You can drop those caches (but you don’t have to) by running echo 3 > /proc/sys/vm/drop_caches, and after that, you will observe that an ls incantantion will appear to hang. Leave it like that for the moment.

• Now we want to merge the dirty blocks back into hard_a. But if we fiddle with the constituents of /dev/mapper/top through DM, it will be dropped! We can’t merge dirty blocks right back into /dev/mapper/hard_a as DM doesn’t trust us prodding the fundaments. So we fool it by creating a second loop device backed by the same file as the loop device backing /dev/mapper/hard_a:

  losetup /dev/loop2 hard
  echo 0 $(blockdev --getsize /dev/loop2) linear /dev/loop2 0 | dmsetup create hard_b

  If you look at the output of dmsetup table you will notice that DM thinks hard_a and hard_b are different devices. And rightly so, because they are, but they happen to point to the same data: the file called hard.

• Now we can merge the device that holds the dirty blocks for hard_a (that would be /dev/loop1) into hard_b:

  echo 0 $(blockdev --getsize /dev/mapper/hard_b) snapshot-merge /dev/mapper/hard_b /dev/loop1 p 8 | dmsetup create mergeomatic && dmsetup status mergeomatic

  You don’t have to do anything with the mergeomatic device. Just wait for it to finish merging. How do you know when it’s finished? I couldn’t figure it out so I asked the dm-devel list. Turns out you can use dmsetup status mergeomatic, its output format is <sectors_allocated>/<total_sectors> <metadata_sectors> and when the amount of sectors_allocated equals the metadata_sectors it’s finished. I looked at dmsetup status in my experiments but I had never seen anything but the same numbers, possibly because I/O to small loopback devices on a machine with loads of RAM will be blazing fast — writes to the backing file are cached in RAM, I guess.
  Done? Remove the snapshot-merge target:

  dmsetup remove mergeomatic

• All that’s left is to create a new device, based on /dev/mapper/hard_b or its backing file. If you’re tired of COWing around you could do:

  dmsetup remove hard_b
  echo 0 $(blockdev --getsize /dev/loop2) linear /dev/loop2 0 | dmsetup load top

  And if you want to continue this trick you’d just make a fresh sparsely backed loop device and make a snapshot target out of it:

  dd if=/dev/zero of=soft_b bs=1 count=0 seek=200M
  losetup /dev/loop3 soft_b
  echo 0 $(blockdev --getsize /dev/mapper/hard_b) snapshot /dev/mapper/hard_b /dev/loop3 p 8 | dmsetup load top

• And now for the grand finale:

  dmsetup resume top

  All blocked processes will spring to life. ls will return a directory listing as if nothing has happened. Mission accomplished.
  With dmsetup load we swapped block devices in-flight. The mounted filesystem on /dev/mapper/top didn’t notice — all blocks still look the same. But both DM and we know that the blocks are somewhere else physically now. Little did DM know that /dev/mapper/hard_b is backed by the same physical blocks as /dev/mapper/hard_a.
  We’d better do some cleaning up then:

  dmsetup remove hard_a
  losetup -d /dev/loop{0,1}
  rm soft_a

This hasn’t left the toying-around phase yet. But I’m thinking about writing some wrapper scripts and doing some more testing. Maybe I should call it the “WhatwasIthinking Volume Manager” as it’s rather tricky stuff. Especially with all the write buffering on multiple levels. I messed it up more than once during testing ;-)

[*] Not just any FS will do. Don’t use journaled filesystems. They assume their journal will be written to disk in-order. That assumption does not hold with file-backed loop devices, because on those there is yet another filesystem (and buffer layer) below it, deciding what data gets committed first.

There are times you need to connect to ‘dirty’ networks such as public WiFi hotspots. Hopefully you’re ensuring that sensitive information is encapsulated in transport layer security enabled protocols such as SSL, because anyone on the same link (in the case of WiFi, that’s the air surrounding you. A vacuum will do, too, but that’s less common) can listen in on the traffic you’re sending. With SSL encapsulation such as HTTP over SSL (https://), your traffic can still be read — but for those who do it’s an extremely boring read because they don’t know the session key, only you and the other endpoint do. Hopefully.

One particularly nasty thing that can happen to you is when your machine is subverted into using the attacker’s machine as the router. That is known as ARP poison routing. The attacker can proceed to not only read the traffic coming from your machine (which, on a shared medium, could be done anyway), or read the traffic going into your machine (again: on a shared medium, that could be done anyway), but the attacker can now also modify the traffic between you and the rest of the non-local network, e.g., the internet, in both directions. And that’s when he can really go to town with your traffic. Injecting a javascript keylogger into all the webpages you visit. ‘Sidejacking‘ your sessions, so he does not even need to know your passwords, just your session cookies — which you happen to transmit with every page request.

All possible unless you use transport layer security, which is tamper-proof once properly set up. Once properly set up. But setting up can have problems of itself — there are ways of preventing you ever going from HTTP to HTTPS. If you know a thing or two about HTTP and SSL you’ll be delighted to learn about Moxie’s very evil but very clever ways of doing so.

Anyway, some level of security can be achieved if you tell your machine to ignore any messages sent to you from the other machines on the local network. That includes messages that will make your machine believe that the router has suddenly changed its physical address — which is quite unlikely to happen, but those messages are exactly the type of message an impersonator would send you. Of course we’d need to whitelist the routers of the network, otherwise we can’t get traffic out of it and onto other networks. DNS resolvers will need whitelisting too, unless you’re running one on your own machine (probably not).
Not openly announcing your presence may also be something you wish for. If you have ever been on a network with a Mac user you have probably seen them popping up in your Zeroconf service browser as “Firstname Lastname’s iSomething”. Let’s cut down on that kind of promiscuity, too. But you should understand now that you can not actually hide unless you turn off your WiFi. Shared medium, remember?

I prepared a simple script to accomplish the above. I’ve used ip from the iproute2 package instead of sticking to old-school route, ifconfig, arp & co. And I must say ip neigh flush nud stale has a poetic ring to it, wouldn’t you agree?

Take note: this will only protect you from some kind of attacks, and only partially. An attacker has a window of opportunity between your machine getting assigned a DHCP lease and you running this script, for instance. Or maybe the access point is rigged. Actually all protection other than end-to-end encryption combined with mutual authentication is pretty useless on shared networks ;-)

Here’s the script. Linux-only. If you want to use it, get the latest version from my public repository.

#!/bin/bash
 
# arpshield 0.2
# Protects against ARP poisoning and cloaks your machine for all 
# local link devices but the router(s) and the DNS server(s).
# Whitelisting DHCP servers also works if you use the dhcpcd program
# to obtain DHCP leases.
# This program is of no help if your setup is already poisoned.
# Have a look at ArpON (http://arpon.sourceforge.net/manpage.html) if
# you need more extensive protection.
#
# Needs 'ip', 'awk', 'sed', 'arptables', and 'arping' and expects
# them on $PATH. Needs appropriate privileges (so use sudo).
# Takes a network interface as an argument. The network interface
# should be up and configured. If no argument is given, clear all
# rules. Obviously you should do that before connecting to a new
# network.
#
# Copyright 2010 Wicher Minnaard (wicher@gavagai.eu)
# License: Creative Commons Attribution-Share Alike 3.0
 
# Do you use dhcpcd for aquiring DHCP leases? And is it running?
dhcpcdLEASEFILE="/var/lib/dhcpcd-${1}.info"
dhcpcdPIDFILE="/var/run/dhcpcd-${1}.pid"
test -f ${dhcpcdLEASEFILE} && test -f ${dhcpcdPIDFILE} && source ${dhcpcdLEASEFILE}
 
# In case you lack the luxury of dhcpcd, where is your resolv.conf?
RESOLV="/etc/resolv.conf"
 
# No user-servicable parts below this line.
DEV="${1}"
 
# I know, I know. But if your routing table contains 0.333.456.789 you have bigger problems ;-)
IPREGEX="\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}"
 
# Register
MACreg=""
 
# If not run as root, bail
[ "$(id -u)" != "0" ] && echo "You need root privileges to modify networking parameters. Exiting." 1>&2 && exit 2
 
getmac(){
# sets MAC register by IP. Sets to nil, if the MAC is not on the local link. 
  getMAC=$(ip neigh show ${1} | awk '{print $5}')
  if [ -z "${getMAC}" ]; then
    arping -c1 -I ${DEV} ${1} > /dev/null 2>&1
    getMAC=$(ip neigh show ${1} | awk '{print $5}')
  fi
  MACreg=${getMAC}
}
 
allow(){
  # Whitelists traffic to and from particular IP+MAC pairings and
  # adds them to static ARP.
  IP=${1}
  MAC=${2}
  if [[ -n "${IP}" && -n "${MAC}" ]]; then
    arptables -A INPUT  -s ${IP} --source-mac      ${MAC} -j ACCEPT
    arptables -A OUTPUT -d ${IP} --destination-mac ${MAC} -j ACCEPT
    ip neigh replace ${IP} lladdr ${MAC} nud permanent dev ${DEV}
  fi
}
 
if [ -n "${DEV}" ]; then
  # whitelist the routers
  test -z ${GATEWAYS} && GATEWAYS=$(ip route show dev ${DEV}| sed -n "s:.* via \(${IPREGEX}\).*:\1:p")
  for GWIP in ${GATEWAYS}; do
    MACreg=""
    getmac ${GWIP}
    allow ${GWIP} ${MACreg}
  done
  # whitelist the DNS servers
  test -z ${DNSSERVERS} && DNSSERVERS=$(sed -n "s:^nameserver \(${IPREGEX}\):\1:p" ${RESOLV})
  for DNS in ${DNSSERVERS}; do
    MACreg=""
    getmac ${DNS}
    allow ${DNS} ${MACreg}
  done
  # if using dhcpcd, we can whitelist the DHCP server too
  test -n ${DHCPSID} && getmac ${DHCPSID} && allow ${DHCPSID} ${MACreg}
  # set default policy to DROP    
  arptables -P INPUT DROP
  arptables -P OUTPUT DROP
  # clear out non-hardcoded ARP cache entries
  ip neigh flush nud reachable
  ip neigh flush nud stale
else
  # No argument given, so clean up.
  arptables -F
  arptables -P INPUT ACCEPT
  arptables -P OUTPUT ACCEPT
  ip neigh flush nud permanent
fi

Access remote X11 servers that have their TCP socket disabled

This happens to me regularly. Someone brings a machine along and I want to display some app, running on my machine, on their display. Networked X11 to the rescue, you say? No, their X11 server is started with ‘-nolisten TCP’ wich is the default on most modern Linux distros. Sadly, the TCP socket can’t be enabled ‘in-flight’ — if you decide you do fancy a TCP socket after all, you’ll have to restart your X server which may be a pain if you’re in the middle of something (besides, restarting is just plain uncool).
But there is a way to expose the Unix domain socket as a TCP socket, with the help of socat. The following examples all use bash, so if you run a different shell (if you don’t know, you probably aren’t) you may need to define environment variables differently.

Braindead Proof of Concept (BPOC)

Situation: You want to display an application running on a machine called w00t on another machine, called bling. There’s an X11 server running on bling, but it’s not configured to listen on any TCP socket. DNS is properly setup, so if you ping w00t from bling, you get replies from bling’s IP, and vice versa.

1. On bling, find the domain socket of bling’s X11 server. Have a look in /tmp/.X11-unix/. The socket’s name usually reflects its X server display number (which you can determine by running echo $DISPLAY in an xterm).
2. On bling, run something along the lines of
   socat TCP-LISTEN:6066 UNIX-CONNECT:/tmp/.X11-unix/X0
   This will open up TCP port 6066 on all of bling’s network interfaces, connecting it to the Unix domain socket of the X server.
3. In an xterm on bling, run xhost +. You have now opened up your X11 server to the whole wide world, a silly thing to do. Anyone with access to the TCP socket can now read your keystrokes, read your window contents, click your mouse buttons…
4. In an xterm on w00t, run DISPLAY="bling:66" xclock. You may have noticed that 66 = 6066 – 6000 and indeed, by convention the TCP port number for a certain display is its display number + 6000. Anyhow…. yay, a clock! It’s displayed on bling, but running on w00t.

Improvements

• You may have noticed that in the BPOC, you can use the display on bling only once. socat will allow only one client, and will exit once that client exits. In some situations, you may consider that a feature (it’s a one-time access grant), but in others you may not. If you want a reusable TCP socket, run something along the lines of
  socat TCP-listen:6066,fork,reuseaddr UNIX-CONNECT:/tmp/.X11-unix/X0 which forks off a socat process for every TCP connection.
• You may not want to expose a TCP socket on all interfaces. Maybe you only want to expose a socket on the LAN interface, or on the localhost interface (and wrap the packets in an SSH tunnel). Well, you can, using the ‘bind’ option:
  socat TCP-LISTEN:6066,bind=localhost UNIX-CONNECT:/tmp/.X11-unix/X0
  Now tunnel it over SSH. On w00t, run ssh -L 6011:localhost:6023 bling. Now localhost:6011 on woot is actually localhost:6023 on bling which is actually /tmp/.X11-unix/X0 on bling. So on w00t you can start an xclock with its display on bling by running DISPLAY="localhost:11" xclock.
• xhost + from the BPOC is braindead indeed. There are a couple things you could have done instead, there are good ways of tightening up your authorization scheme.
  • First off, you don’t really need to run xhost + if you properly set up X11 cookies, which you should. Here are some examples on using the xauth scheme, but take note: xauth generate will probably not work on recent X11 releases since the XSECURITY extension is disabled by default. Just use the same cookies on the client and the server.
  • Run xhost +w00t. That’s host-based authentication, which is stupid, but not as stupid as no authorization at all. Any user on w00t can now connect.
  • Suppose that on bling (of course!) you’d run xhost +SI:localuser:theuser with ‘theuser’ being the userID of the unix-user running the socat instance. Now from the point of view of the X server, any client connecting through socat will be coming from ‘theuser’ and will therefore be allowed access. Entertaining, but not much different from just running xhost +. It is something to keep in mind though! Many distros by default add the unix-user that started the X server to the authorization list. That user does not need a cookie. If you run socat as that user you will have the effect of running xhost + even if you run xhost -.
  • Just run a nested X11 server, such as Xnest or Xephyr. This way you put untrusted users in a sandbox, preventing them from snooping your keyboard and windows. It’s the X11 equivalent of a chroot.

Here’s a trick. Many laptop trackpads lack a middle mouse button. On a regular mouse input device, the middle mouse button is the scroll wheel, and when you press it down it emits a button event. In X11 this button event is used to paste the X selection buffer into the position right beneath the cursor (there lies sublime usability in this simple fact).
You can emulate a middle-mouse-button event by pressing the left and right mouse buttons at the same time. Since I lack the manual dexterity to do this on my tiny netbook trackpad I wanted to be able to do middle-mouse-button-paste with my keyboard. Well, that appeared to be easy to accomplish with the X11 Xtest extension for which the Xautomation collection includes a utility in the form of xte. If you’d enter xte 'mouseclick 2' in a terminal (within an X11 session, of course), you’d get the same effect as if you’d just pressed the middle mouse button. Only thing left is to add a keyboard shortcut to run this command; in my favourite window manager, XFCE, this can be done clickwise via the Settings Manager or simply by running something like xfconf-query -c xfce4-keyboard-shortcuts -p '/commands/custom/<Super>v' -s "xte 'mouseclick 2'". I can now paste my X selection buffer by pressing the funny ‘four-wobbly-squares key’ and ‘v’ simultaneously.

My X11 setup contains some truly ugly fonts. In particular, it contains bitmap variants (unaliased, of course) of some common fonts, donated by Adobe. That must have been swell in the nineties but it isn’t anymore. But they’re still included in X.org X11, spreading uglyness when referenced explicitly:

This very much isn’t how I like my webpages. In this case, drilling down to the font declaration (use the excellent Firebug if you have to) yields:

BODY { font-family: "new century schoolbook", times, serif}
H1   { font-family: "new century schoolbook", times, serif}

Is this ‘new century schoolbook’ font one of those Adobe bitmap fonts? ‘xlsfonts’ yields:

$xlsfonts  | grep -i century
-adobe-new century schoolbook-bold-i-normal--0-0-100-100-p-0-iso10646-1
-adobe-new century schoolbook-bold-i-normal--0-0-100-100-p-0-iso10646-1
-adobe-new century schoolbook-bold-i-normal--0-0-100-100-p-0-iso8859-1
-adobe-new century schoolbook-bold-i-normal--0-0-100-100-p-0-iso8859-1

[... dozens more]

So, let’s get our font subsystem to remap this font declaration to something more appealing. You’ll need to edit your fontconfig configuration which is described rather exhaustively elsewhere and add a stanza like this one:

    <match target="pattern" >    
        <test name="family" qual="any" >   
            <string>new century schoolbook</string> 
        </test>target="font">
        <edit mode="assign" name="family" >
            <string>Nimbus Roman No9 L</string>
        </edit>
    </match>

This is, of course, particular to my wishes and the fonts available on my system. Run fc-cache, reload the webpage with the offending font, and tadaaaa:

Much better.

Update @20091208: With Gentoo, you can use the eselect system to accomplish this. Run eselect fontconfig list and pick the number corresponding with 70-no-bitmaps.conf. Enable it with eselect fontconfig enable $thenumber.
From the 70-no-bitmaps.conf file, it appears there’s a much better way of handling this:

<fontconfig>
<!-- Reject bitmap fonts -->
 <selectfont>
  <rejectfont>
   <pattern>
     <patelt name="scalable"><bool>false</bool></patelt>
   </pattern>
  </rejectfont>
 </selectfont>
</fontconfig>