Smörgåsbord

Ambachtelijk bereide beschouwingen.

So on one of my NFS client systems I was getting the dreaded nobody:nobody ownership problem on an NFSv4 mount. It’s a problem that occurs when NFS cannot map users between systems. Files will show up as being owned by user nobody and group nobody.

A common cause is that the domains between the client and the server are not corresponding, preventing a proper mapping. Let’s do a little experiment to see if I can exclude this cause, I thought. I’ll hardcode the NFS client’s domain setting to achieve correspondence with my NFS server’s. Easy, I can hardcode the domain in /etc/idmapd.conf, right? Right?

[General]
#Verbosity = 0
# The following should be set to the local NFSv4 domain name
# The default is the host's DNS domain name.
#Domain = local.domain.edu
Domain = sub.mydomain.tld

(where sub.mydomain.tld is a stand-in for my domain proper).

Wrong! Lies, damn lies! Well actually it’s kinda speaking the truth. The host’s DNS domain name is the default — a very special case of “default”: it’s the only thing that counts! The domain variable in idmapd.conf, depicted here, is moot, a folly, it doesn’t do anything!1 Or so I noticed when I ran the following:

root@sheef ~ # rpc.idmapd -f -vvv
 
rpc.idmapd: libnfsidmap: Unable to determine the NFSv4 domain; Using 'localdomain' as the NFSv4 domain which means UIDs will be mapped to the 'Nobody-User' user defined in /etc/idmapd.conf
rpc.idmapd: libnfsidmap: using (default) domain: localdomain
rpc.idmapd: libnfsidmap: loaded plugin /usr/lib/libnfsidmap/nsswitch.so for method nsswitch

Eh? Let’s see if this happens on the other side. On the server, I changed the domain variable in idmapd.conf to ‘localdomain’.

root@bling ~ # rpc.idmapd -f -vvv
 
rpc.idmapd: libnfsidmap: using (default) domain: sub.mydomain.tld
rpc.idmapd: libnfsidmap: loaded plugin /usr/lib/libnfsidmap/nsswitch.so for method nsswitch

Oh idmapd, how you cheated me. Completely ignoring my “hardcoded” ‘Domain’-variable.
It seems the true path to enlightenment (and solving this issue) will lead through the Forest of Domainname Config Fiddling.

root@sheef ~ # hostname --fqdn
sheef.sub.mydomain.tld

Looks perfect!

root@sheef ~ # domainname  
(none)

Looks imperfect!
Could this help?

root@sheef ~ # echo sub.mydomain.tld > /proc/sys/kernel/domainname 
root@sheef ~ # domainname                                             
sub.mydomain.tld

Huzzah! Or so I hoped. But no. rpc.idmapd still tells me it’s unable to determine the NFSv4 domain. Desperately, I grep around for sub.mydomain.tld in the /etc/ of boxes that are properly and happily NFS’ing. I found the clue in /etc/hosts. To make idmapd happy, it should contain the FQDN, like this:

127.0.0.1       sheef.sub.mydomain.tld sheef localhost

This makes all the difference!
Somehow I forgot this when setting up this machine. But at least I have now had the pleasure of debugging the lying bastard that is rpc.idmapd.

1)This is with nfs-utils-1.2.5 from linux-nfs.org.


Tags: , ,

I wanted to know whether I could use some tricks to make more efficient use of the 4GB Compact Flash storage i have in my Alix (running Gentoo Linux). I would like to keep a local portage tree on it — it currently mounts the tree over NFS due to space concerns.
Why no harddisk? Because that goes against the idea of having a small, reliable, always-on, energy-efficient home server.
Techies like pictures of hardware with the cover off so here’s my Alix:

Alix 2C3

Such a setup can lead to unfortunate situations, such as needing a package to restore connectivity to the network on which the fileserver resides that contains the package that I need to restore connectivity to the network on which… see where I’m going? Nowhere! ;-)
The portage tree fulfills the role of a package database for us Gentoo ricers. I need it locally.

Diego, one of the Gentoo devs, wrote a blog post about space inefficiency incurred through the use of many small files in the portage tree. He puts the Portage tree on different filesystems to arrive at an accurate and detailed picture of incurred overhead.

I wanted to quickly find out what filesystem block size would suit different parts of my filesystem (such as /etc/, /usr/src/linux/, /usr/portage/, /var/db/pkg/) best, and how much I could save. Testing all block sizes with all parts of my FS was not very appealing, so I decided to simply calculate the file slack and wrote this simple Python script:

slacktastic.py

#!/usr/bin/env python3
 
"""
slacktastic.py - calculate filesystem file slack for different block sizes.
Invoke me thusly:
    find /path/to/tree -xdev -type f -printf "%s\\n" | slacktastic.py 4096
for a calculation using the contents below /path/to/tree with a block size of 4096 bytes.
"""
 
import sys, functools, math
 
blksz = None
try:
  blksz = int(sys.argv[1])
except (IndexError,ValueError):
  print(__doc__, file=sys.stderr)
  print('I need a blocksize as the first argument.\n', file=sys.stderr)
  sys.exit(1)
 
sizes = [int(strsize) for strsize in sys.stdin.read().strip().split('\n')]
 
sumslack = functools.reduce(lambda sumslack, sz: sumslack + (blksz - (sz % blksz)), sizes, 0)
sumblks = functools.reduce(lambda sumblks, sz: sumblks + (math.ceil(sz / blksz)), sizes, 0)
sumsizes = sum(sizes)
 
print('{:n} total slack'.format(sumslack))
print('{:n} bytes in files'.format(sumsizes))
print('{0:n} total blocks of {1} bytes'.format(sumblks,blksz))
 
print('{:.2%} inefficiency'.format( sumslack / (sumblks*blksz) ))

This doesn’t take into account things such as tail packing (à la ReiserFS), compression (Btrfs), or directory slack. Just file slack.
On my laptop filesystem, with a 4096 block size, this leads to the following observation:

find /var/db/pkg -xdev -type f -printf "%s\n" | slacktastic.py 4096
171942364 total slack
92655140 bytes in files
64506 total blocks of 4096 bytes
65.08% inefficiency

My /var/db/pkg, Gentoo’s ‘database’ of installed packages (containing their build environment and all kinds of stuff you wouldn’t need on a binary distro) contains 65% air! That’s 170 megs of space which I don’t want to waste on my Alix’s 4GB CF card. With a 1024 byte block size — Ext4’s minimum — the situation is better, but it’s still over 40 megs of hot air.

I ended up choosing btrfs with compression. It has a fixed 4096 byte leaf/node size but it does tail packing (good for small files) and compression (for my /var/log). My script is useless for estimations on such a filesystem so I ran some actual tests and it turns out I can fit my /usr/src/linux, /var/db/pkg, /var/log and /usr/portage on a 1GB btrfs filesystem. They didn’t fit on an bs=1024b Ext4 FS.


Tags: , , , , ,

Legoland. Je kunt er van alles worden. Behalve als je vrouw bent. Dan ben je dat gewoon, “vrouw”.

“Hee, vrouw!” zegt de beroepsbevolking dan op straat tegen je. Of: “Zeg meisje daar! Word jij ook vrouw als je groot bent? Zal wel hé?”

inc4mini
inc5mini


Stel: je beheert zo’n miljardenpensioenfonds, en je belegt dat in vastgoed — woningen in de vrije verhuur, flatjes op Kijkduin. Dan is wel zo aardig als daar daadwerkelijk huurders in komen. Jij wil verhuren, huurders willen huren, vraag en aanbod zoeken elkaar op. Alhoewel…

Als verhuurder wil je er waarschijnlijk Nette Mensen in. En dan met name die Nette Mensen die de huur met relatief gemak kunnen betalen. Je prijzen zijn marktconfom en er kloppen allerlei geïnteresseerde lieden aan die je flatjes willen huren — misschien ondanks het feit dat ze het op enig later moment op financieel gebied misschien niet meer helemaal kunnen bolwerken.

Maar. Gezien de huurbescherming die ook een wanbetalende huurder in Nederland geniet, ben je als pensioenfondsmeneer of -mevrouw een beetje voorzichtig met het in huis halen van huurders. Je wil bijvoorbeeld weten wat zo iemand ongeveer verdient en of er sprake is van een stabiele bron van inkomsten. Enige prudentie is dus op zijn plaats.

Wees ook weer niet TE voorzichtig — als je alleen maar huurders met inkomen boven de Balkenendenorm wil aannemen, denk ik niet dat je het flatje op Kijkduin verhuurd krijgt. En leegstand was niet de bedoeling van al dat mooie vastgoed, dat moet namelijk rendement opbrengen in de vorm van huurinkomsten!

Vergis je je naar de ene kant toe — neem je een huurder aan waarvan je denkt dat-ie braaf huur gaat betalen, maar doet hij dat niet — dan kost dat geld. Wanbetaling = gemiste inkomsten = gemist rendement.

Vergis je je naar de andere kant toe — neem je bijna niemand meer aan als huurder, omdat je een beetje wantrouwend bent — dan heb je leegstand. Leegstand = gemiste inkomsten = gemist rendement.

Ergens hier tussenin zit een optimum (om precies te zijn; precies dan als je je nooit vergist, maar dat is erg hypothetisch in de financiële sector).

Als je nu leert van vergissingen krijg je steeds betere informatie waarmee je risico’s kunt inschatten. Bij volledige informatie krijg je een equilibrium dat precies op dat optimum zit. Dat geeft een minimale verkwisting en een maximale efficiëntie, precies zoals Adam Smith (en ik) het in het vrijemarktmodel voor zich zien.

Echter, zulke directe interacties zijn er niet altijd, en zuivere en volledige informatie is er ook niet altijd. Wat automatisch voor vertroebeling zorgt is als je de boel uitbesteedt en aan een makelaar vraagt om de huurbemiddeling te doen. Dan gebeurt er iets wonderlijks.

De makelaar kan een verkeerde inschatting maken. Dat is geen nieuws, dat kon jouzelf ook overkomen. Wat wel nieuws is, is dat er nu verantwoording afgelegd moet worden, en wel aan jou, de pensioenfondsbeheerder.
“Verantwoording afleggen”, dat is toch goed? Ja en nee. Verantwoording afleggen, dat is een verhalend ritueel — het is geen zuivere informatieoverdracht.
Soms levert het een partij tactisch voordeel op om in het verhaal juist onvolledige informatie te geven. Welk verhaal kan de makelaar nu aan jou vertellen? Dat hangt van zijn vergissing af.

Heeft hij drie wanbetalende crackdealers als huurders binnengehaald, dan heeft-ie heel wat uit te leggen. Die crackdealers zijn nogal aanwijsbaar. Jij kan ze aanwijzen en zeggen “die crackdealers kosten mij geld en jij hebt ze erin gelaten”.

Maar als de vastgoedhandelaar zich aan de andere kant heeft vergist — is er landurige leegstand omdat hij veel te kieskeurig was — dan zal hij een beetje gaan freestylen. Allereerst kan-ie zeggen “Lang? Nee hoor. Dat is normaal, vraag en aanbod moeten elkaar even opzoeken.” En bij wat doorvragen zal hij wellicht ook nog iets zeggen als: “De markt! (Maakt handgebaar) De markt is nu ongunstig en afwachtend! Iedereen moet even geduld hebben.”
“De markt” is voor jou niet aanwijsbaar. En al helemaal niet aanwijsbaar de schuld van de makelaar. Het is overigens ook niet jouw schuld. Iedereen kan gerustgesteld in de auto stappen, want terwijl het eerst nog leek alsof we iets verkeerd deden en het rendement drukten, zijn we er nu achter dat we er niets aan kunnen doen.
En als je een pensioenfonds in de staalsector beheert dan denk je bij het afscheid misschien zelfs “ah, ‘De Markt’, da’s een goeie — die ga ik noemen op de bestuursvergadering”.

Logisch dus dat de makelaar zich liever aan de veilige kant vergist. Effect daarvan is dat het equilibrium opschuift, richting “onnodig voorzichtig huurders aannemen”. De makelaar wil echt geen enkel risico nemen met de huurders, want het is zo moeilijk uit te leggen. Leegstand is veel makkelijker uit te leggen.

Verliezers? Pensioengerechtigden die een indexering mislopen, en de mensen die in het almaar leegstaande flatje op Kijkduin hadden willen wonen (dat ze overigens prima hadden kunnen betalen).

De vrije markt haalt maximale efficiëntie bij volledige informatie en volledige internalisering van kosten. Een beetje zoals de klassieke natuurwetten, die doen het vaak ook heel mooi — maar alleen in een vacuüm, of bij 293⁰ Kelvin, en dan hebben ze een ‘ideale warmtegeleider’, ‘ideale weerstand’, of een ‘ideaal gas’ nodig.
Als er wel eens marktgerelateerde vraagstukken in je opkomen, is het leuk om ze eens te bezien vanuit een informatiebeschikbaarheidsstandpunt. De één z’n desinformatie is de ander z’n brood.


Quite a while ago, I ranted about Facebook, their email “validation”, and their rather eccentric take on quality assurance. Things have changed at Facebook — you can now sign up using a sub-addressed email account.

Friend suggestions

But there may have been more to it than improper validation. One of the many ways Facebook comes up with suggestions running along the lines of “so-and-so is now on facebook, do you know her? let’s connect!” is to entice you to give your password to third-party services (they call this “add friends from your address book”). So suppose you had mary@jane.com in your Gmail address book at that time, and you let Facebook read this address book, then Facebook stores the mary@jane.com address (possibly indefinately) as one of your “friend candidates”.
Some time later, Mary Jane registers a Facebook account, using her mary@jane.com email address – the one that’s also on record as one of your friend candidates. Ping! Facebook sends you a message, enticing you to connect1.
Of course, this all falls down when Mary registers using a sub-addressed address, e.g., mary+facebook@jane.com. She might want to do this to channel the flood of facebook-originating email into a separate folder. mary+facebook@jane.com is probably not on file with you or anyone else, since if you want to send her an email, you’d send it to mary@jane.com — so that’s what was in your Gmail address book.

Sub-addresses: Bad for business, unless…

That brings me to the following conspiracy theory: Initially, Facebook disallowed sub-addressed email addresses (under the guise of a “broken” validator?) because those interfere with their goal of engaging you with as many people as possible (via friend suggestions), so as to have your eyeballs on their site and in front of their advertisers until they bleed (the eyeballs, not the advertisers).
At a certain moment in the past 28 months, they fixed the improper ‘invalid email address’ designation of sub-addressed accounts. Good for facebookers, bad for business — unless they parse the email address and drop the part between the + and the @. Thing is, sub-addressing is not a standard. On my mailserver, I can specify a character other than + to use as an extension designator. It’s up to the mailserver to do something useful or silly with the sub-addressing. There are no formal semantics. If there is a + in the user-part of an email address, that does not necessarily mean that it is sub-addressed.
My guess is that Facebook made their address matching fuzzy, to account for many possibilities. They’ll plunder your address book and will still figure out that you and Mary Jane are acquainted, no worries.

Tinfoil hat

Well, I do worry.
On the web, your email address is a key to your identity. Your identity is something which many organizations (advertisers, some governments, …) very much like to link across natural domain boundaries. I don’t think that many organizations have updated their address matching algorithms with fuzzyness… yet. So at the moment, it’s still a good idea to sign up to sites and services using unique, subaddressed email accounts. But to be futureproof, you’ll need to defeat fuzzy matchers that take the many forms of sub-addressing into account. It’s probably best to just register a domain and have all email arriving at that domain be delivered to one account. That way, you can easily use any email address at that domain when signing up. If you don’t want to run your own mail server, the one that Google provides you with if you take Google Apps on your domain allows just that. But I’m not so sure that recommendation is solid advice, privacy-wise…

1)Possibly, and hopefully, they do some cross-checking first.


Tags: , , ,

For a little while I’ve been running a DIY dynamic DNS service. My peers think it’s convenient and pretty sweet, so in this post I’ll set things straight — a full disclosure on the mess that it really is ;-)

What is a dynamic DNS service?

Since the dial-up days of yore I have occasionaly been using “dynamic DNS” services. What is a “dynamic DNS” service, you ask? Well, it’s a marketing term, not a technical term, but most providers of such services allow for having an DNS “A” record point to an IP, and this pairing is then updatable over HTTP. Usually they use a freemium business model where you can get a couple of subdomains on their second level domains.

Why use one?

An example of a use case: You run a development server on your laptop. You want someone to connect to it. Instead of saying to this person “now just go to… errr… hold on” (and now you pop your shell and run ip a s dev wlan0)1 “yes I’m back, you need to go to 110.34.56.250″ you can just say “go to kleinebeer.ath.cx”. Much easier. And that’s with IPv4. Typing an IPv6 address that someone just yelled at you from across the room is an even less joyful experience.
To enable this use case you configured your machine in such a way that every time your interface aquires a new IP, a program contacts the “dynamic DNS service” to update the ‘kleinebeer’ DNS record to reflect this IP. There are many clients for such services out there. I have even seen domestic ADSL modem/routers on which the factory firmware contains such a client.

Doing it differently

But some time ago I got fed up with all the “account expiration warning” emails I had been getting from the service that I used. And since I’m quite familiar with the components you need for building such a service yourself, I set out to do so. The thing I ended up with is for personal use, specific to my needs, KISS, and sinful.

Sins!

It allows for instant creation/updating of DNS records through a very simple HTTP GET. That is sinful for two reasons:

  1. No authentication and authorization.

    Anyone can create any record. And anyone can update any record he or she knows the name of. You understand how this could be a bad thing. Don’t have such names as MX records unless you don’t mind that mail intended for you ends up on someone else’s server.
    At the same time it is a good thing, as I will show in the use case example below.

  2. The HTTP GET alters state.

    And that’s just plain Bad Taste. If you’re designing a web service you’ll usually try to make it RESTful. GET should be a safe method. But for this application, at the frontend, it is actually desirable — because grandma can GET, as you will see in the use case below.

Use cases

  • Case A:

    Imagine you’re on the phone with your grandma, and after discussing apple pie recipes something else comes up and you need to know her IP (let’s say you need to SSH in and help her with her crontab). She hasn’t registered with any dynamic DNS service, she has a hard time determining her IP, she has a hard time reading it out loud without making mistakes, and you are fed up with typing IPs anyway. But she has a web browser. Wouldn’t it be nice if you could just tell her to open her browser and head over to http://grandma.reg-a-record.tld/reg.php ? Nothing to install, nothing to look up, and something reasonably simple for grandma to do. The script on http://grandma.reg-a-record.tld/reg.php creates a DNS A record named ‘grandma’ on dyndomain.tld, and this record points to your grandma’s IP. Now you can just do ’ssh grandma.dyndomain.tld’.

  • Case B:

    Imagine you and your friends are on a coding binge, all running their Django/Rails/Node.js/Thing-du-jour instances on laptops, in the same room, and you all want to connect to each other’s development servers.
    You could all call out your IPs across the room and let everyone type in everyone else’s IPs. Now you all have N tabs open in your browsers. Who was 123.231.101.45 again?
    Or, everyone could just point their browser (or curl, wget…) to http://theirfirstname.reg-a-record.tld/reg.php2 and be done with it. If you want to view Fred’s progress you just go to fred.dyndomain.tld.

So, sin #1 & #2 allow for use cases that were not possible with the provider I was using.

You don’t have to register the records up front with some service provider. You can make up records as you go, and you want to be able to let anyone create any record. Hence sin #1. But it’s a good thing. You don’t have to reuse records, which is better for your privacy. With the commercial service I could only use a couple of names, so anyone whom I once had told to go to kleinebeer.ath.cx could kinda virtually follow me around and determine whether I was at work, home, uni or girlfriend by resolving this record. When creating new records is incredibly easy, there’s no reason to keep using the same name every time.

On to the code

It’s very, very simple! You need very little code. Things can be made much simpler than I have done. For byzantine reasons I use two webservers, one has a python CGI which creates the actual records, and the other one runs the public-facing PHP script (through mod_php) which you actually connect to.

DNS server

You need a domain for which you run the authoritative name server. I’m using the SheerDNS DNS server package on mine, for no good reason. Here’s a patch I made so that sheerdnshash creates directories with some group permission bits set, something I need in my setup.
You can use other DNS servers, of course. I discovered that the Unbound DNS server has a socket protocol which you can possibly use to add records on the fly. But I use sheerdns now, and here’s a Python module I made to write A-records in sheerdns’s directory structure. Public domain, use it as you see fit. You need to modify the constants to reflect your config, and you can test it by running the module from a shell.

#!/bin/env python
 
import socket,os,string,subprocess,sys
 
SHEERDNS_DIR="/var/sheerdns"
SHEERDNS_HASH="/usr/sbin/sheerdnshash"
DYNDOMAIN="dyndomain.tld"
ALLOWCHARS=string.ascii_letters + ''.join([ "%d" % i for i in range(10)]) + '-'
 
def mk_A(name,ip):
	#are host and ip allright?
	host = ''.join([ c for c in name if c in ALLOWCHARS ]).lower()
	if not host:
	  return None
	try:
	  socket.inet_aton(ip)
	except socket.error:
	  return None
 
	fqdn = "%s.%s" % (host, DYNDOMAIN)
 
	#set the umask
	oldumask = os.umask(0002)
 
	#get the hash
	hash = subprocess.Popen([SHEERDNS_HASH, fqdn], stdout=subprocess.PIPE).communicate()[0].strip()
 
	#construct the dirpath
	path = os.path.join(SHEERDNS_DIR,hash,fqdn)
 
	#chmod it for everyone to read
	os.chmod(path,0755)
 
	#construct the A-record-filepath
	apath = os.path.join(path,'A')
 
	#open the record file
	with open(apath,'w') as record:
		record.write(ip)
 
	os.umask(oldumask)
	return name
 
if __name__ == '__main__':
        if len(sys.argv) == 3:
          print(mk_A(sys.argv[1],sys.argv[2]))

Web ’service’ to create the records: a CGI

This is a Python CGI which uses the above module. I stick it behind HTTP Basic authentication which is handled by the web server. You call it like this: http://server_that_has/the_CGI?name=therecordname&ip=theip .
There’s no reason that this CGI should be guilty of Sin #2, except laziness on my behalf. If I’d be following every convention even for silly little solo projects I’d never get anything done, all right?
It doesn’t propagate any errors from the sheerdns library, but it prints the name as it is registered — stripped of disallowed characters, or nothing if something failed along the way (invalid IP, for instance).

#!/usr/bin/python
import sheerdns
import cgi
 
print "Content-type: text/plain\n\n";
params = cgi.parse()
name, ip = params.get('name'), params.get('ip')
 
if ip and name:
  dnsname = sheerdns.mk_A(name[0],ip[0])
  print dnsname

The accessible part: reg.php

This is in PHP. It lives in the default virtual host of my Apache config, and for good reasons. I have a CNAME record that points *.reg-a-record.tld to this web server. I then use whatever is in * to construct the name for the dyndomain.tld-record. The wildcard record is essential.
You need to modify this script to reflect your config.

<?php
header('Content-Type: text/plain');
$hostparts = explode('.', $_SERVER['SERVER_NAME'], 2);
$ip = $_SERVER['REMOTE_ADDR'];
print $ip;
 
$cha = curl_init('https://server_that_has/the_CGI?name='.$hostparts[0].'&ip='.$ip);
curl_setopt($cha, CURLOPT_SSL_VERIFYPEER, False);
curl_setopt($cha, CURLOPT_TIMEOUT, 5);
curl_setopt($cha, CURLOPT_RETURNTRANSFER, True);
curl_setopt($cha, CURLOPT_USERPWD, 'bite:me');
$resp = curl_exec($cha);
curl_close($cha);
print($resp);
?>

Presto

If you visit http://kitten.reg-a-record.tld/reg.php, the IP address that the web server sees you coming from gets registered as kitten.dyndomain.tld. If everything went well, the script responds with this mapping.
You could set DirectoryIndex reg.php here, or rename reg.pgp to index.php, if you don’t want users to type ‘reg.php’. I did make it explicit, because web sites on this server come and go and I don’t want people to stumble onto reg.php every time they hit the default vhost because their web site doesn’t exist any more.

1)Let’s assume you’re not NATed.
2)Try to only have friends with distinct names that can be expressed in ASCII.


Tags: , , ,

While planning my upcoming “Except Hungary”1 cycling trip I did some research on land mines, since I might find myself passing through areas mined during the Yugoslav wars of the 90’s.

Croatia is one of the world’s worst affected countries when it comes to land mines, right up there with Angola and Cambodia. In the Croatian War of Independence, a grand total of about 3M anti-personnel mines have been laid in border areas — both sides were stretched on personnel, and Croatia happens to have very long borders (it has quite an unusual shape). About 100K mines have yet to be cleaned up — a rather costly business (at $300-$3K a pop (no pun intended), according to the Red Cross). Perhaps bees will come and help out.

Of course I’m not the first to wonder about the perils of traveling through such regions. Shallow googling (or duck-duck-go-ing as I tend to do nowadays) does not yield any comprehensive information on the location of the mined areas. It mostly boils down to “don’t go off the beaten track” and “tourist areas are safe”, “ask the locals”, “look for signs”, “nah. most of them are up north in cow country now why would you want to visit our cows, go visit our beaches instead”.

My land usage patterns are often quite different from those of other tourists. I camp out in a small tent in the fields. My usual method of finding a suitable spot to pitch said tent is to scoot up some dirt tracks and stroll into those fields willy-nilly. At dusk.
From a health perspective, this mode of habitation-seeking is incompatible with mine fields.


Luckily, the CROMAC (Croatian Mine Action Centre) offers a web-based GIS application showing, in detail, all areas suspected to be mined. An overview of how the GISing of military maps, aerial photography and other sources of information comes about is provided in this presentation.
It appears many mine fields (~60%) are in forests (or: some forests are on mine fields), possibly because these have had the lowest priority in de-mining operations and/or because abandoned fields tend to become forested.

As for Serbia, I won’t be passing through the locations mentioned on the “Landmine and Cluster Munitions Monitor” report. There’s talk of mine fields near the Kosovo border but the Kroatian border appears to be fully demined on the Serbian side. Unexploded cluster bomblets from the NATO bombardments are not found north of Beograd.

So now I’m coloring mined regions on the map of Croatia which I will take with me on my bicycle journey. It’s a solemn activity, and a weird thing to do — marking up mine fields is not a standard part of the holiday planning process. These mines were laid twenty years ago. I was still in primary school then. I remember watching weekly reports on “school-TV weekjournaal” and discussing them in class. There was a refugee girl from Bosnia2 in our class, I think her name was Çenka.

And it never occured to me that contaminating an area with land mines can have nonlocal effects. From the country report on Croatia by the “Landmine and Cluster Munitions Monitor”:
“Another priority is suspected mine contamination along canals and river banks, which has prevented maintenance and resulted in flooding of ploughed land, particularly along the border with Hungary. In addition to canals, parts of the banks of the Kupa river in Sisak-Moslavina county, the Sava river in Brod-Posavina and Vukovar-Srijem counties, and the Drava river in Osijek-Baranja county are inaccessible due to mine contamination. Protection from flood is also impossible.”
Mining riverbanks is a terrible idea for another reason, too: in autumn and spring torrents, mines can wash away and end up god knows where.
On the bright side, there hasn’t been a casualty caused by a land mine in Croatia since 2006 (except for a 2009 incident with demining personnel — brave people). And the Drava/Danube rivers running wild east of Osijek are bound to offer an undisturbed wetland habitat for avifauna. I see ecotourism opportunities.


1) Not that I dislike Hungary. I spent a wonderful time there back in 2001. But I do dislike cycling on endless plains, so I am trying to avoid the area known as the Great Hungarian Plain (the “Puszta”).

2) In the Bosnian war another 3M mines were laid, often in a haphazard way which makes clearing them an even harder task. The excellent pitch-dark comedy/drama/tragedy “No Man’s Land” revolves around a Bosnian trench, three soldiers, a land mine, and a UNPROFOR/media travesty.


Tags: , ,

While decrufting my homedir I found some scripts that might be of public* interest ­— either because of their amusement value, or their utility ;-)
Some are so tiny that you’d actually be better off just defining them as functions in your ~/.{ba,z}shrc .

*) For unixoid values of ‘public’.

groe

Open a file (argument 2) in joe (works with vim, too) with the cursor on the first line matching the regex in argument 1.

#!/bin/bash -eu
THEL=$(grep -m 1 -n ${1} ${2} | cut -d ':' -f 1)
[ -n "$THEL" ] && joe +$THEL $2

paserv

Set pulseaudio server (in X11, for display in $DISPLAY env var) to argument 1. Useful if you use several pulseaudio network audio servers. Without arguments, resets to default.

#!/bin/bash
if [ -n "${1}" ]; then pax11publish -e -S ${1}
else pax11publish -e -r; fi

gmrun

Wrapper launcher for the excellent “gmrun” program launcher. (queue “yo dawg” meme).
Raises and/or pulls any existing gmrun windows to your current desktop.

#!/bin/bash
/usr/bin/wmctrl -x -b add,raise -R "gmrun.Gmrun" || PATH="/home/boer/bin:$PATH" /usr/bin/gmrun

getx

Get an X11 authorization cookie for a display on some other host, over SSH.

#!/bin/bash
if [ -z ${1} ]; then
	echo "Usage: $(basename ${0}) sshstanza"
	echo ""
	exit 2
fi
 
SSH=${1}
XHO=${1##*@}
DIS=${2-":0"}
 
ssh ${SSH} "xauth list ${XHO}${DIS} | sort -u" | while read line; do xauth add ${line}; done

gaap

Ghetto power management. We don’t need no stinkin’ gnome-power-manager, we’ll just grep our keyboard/trackpad interrupt counter thankyouverymuch (I got that idea from reading the xscreensaver man page).
Comes in two parts.
is_user_alive:

#!/bin/bash -eu
#Succeeds if keystrokes or touchpad input has taken place since last time this was run, or if there is no record of the last run.
 
RECORD=${1:-/dev/shm/i8042_intcnt}
 
RETVAL="1"
 
upd_intcnt() {
  grep i8042 /proc/interrupts > ${RECORD}
}
 
is_alive() {
  grep i8042 /proc/interrupts | diff - ${RECORD} > /dev/null 2>&1 || RETVAL=0
}
 
[ ! -f ${RECORD} ] && upd_intcnt && exit 0
is_alive
upd_intcnt
exit $RETVAL

Run this one (gaap.sh) from cron:

#!/bin/bash -eu
 
misschien_slapen() {
  test -f /tmp/koffie || on_ac_power || echt_slapen
}
 
echt_slapen() {
  /usr/local/sbin/is_user_alive /dev/shm/gaap_alive || true
  echo -e "Hibernating in 30 secs, unless you prove you're alive by pressing\na key or touching the touchpad. Physically. On ${HOSTNAME}." | wall
  for sec in `seq 0 3 99`; do echo $sec; sleep 1; done | DISPLAY=":0" XAUTHORITY="/home/boer/.Xauthority" sudo -u boer zenity --progress --auto-close --text "hibernate?" || exit 0
  /usr/local/sbin/is_user_alive /dev/shm/gaap_alive || /usr/sbin/pm-hibernate
}
 
/usr/local/sbin/is_user_alive /dev/shm/gaap_alive || misschien_slapen

You might want to install zenity, and you’ll have to adjust the username. As you can see my laptop is a single-user system. The sudo is therefore a bit silly, but hey, it’s good practice to not present privilege escalation attack surfaces to yourself ;-)
I still need to find a good way of iterating over all X11 displays and finding the user who started the associated X server. Does anyone know of a not-too-hackish approach?

Aphorism slideshow


A friend of mine got married this summer and I got the newlyweds one of those digital photo frames. I preloaded it with a bunch (5.5K) of images generated from my unix fortune database. So it’s kind of a modern incarnation of those tiles that older Dutch generations used to cement onto their walls (Wikipedia entry [Dutch]). Terrible as they are, they’re making a (campy) comeback. Personally I can’t wait for the day that camp culture itself becomes camp (queue another “yo dawg” meme).


Anyway, I made myself this sed script, “str2cu.sed”:

/^%$/d
s/"\([^"]*\)"/“\1”/g

and then did this. Which, the better part of a year later, proves quite hard to reverse-engineer:

for CATG in art definitions education fightclub food hitchhiker humorists kids literature love medicine men-women news paradoxum pets platitudes politics science smac strangelove wisdom work; \
do mkdir /home/boer/tmp/fortunes/$CATG; cd /home/boer/tmp/fortunes/$CATG; \
csplit -q -z /usr/share/fortune/$CATG '/%/' '{*}' ; for f in /home/boer/tmp/fortunes/$CATG/*; \
do cat $f | sed -f /home/boer/tmp/fortunes/str2cu.sed | fmt -w 2500 | tr -s '\t' | expand -t 2 > /tmp/fortune; \
mv /tmp/fortune $f; done; find /home/boer/tmp/fortunes/$CATG/ -type f -size +200c -exec rm '{}' \; ; \
for f in /home/boer/tmp/fortunes/$CATG/*; do convert -background pink -fill white -size 480x234 \
-stroke black -gravity Center -font Essays1743-Bold caption:@$f $f.jpg; done; done

GNU textutils are fun… but holy cow. What an obtuse kludge.
A simple python script could have done all the work up until the image generation (which is done with imagemagick, it has a nice auto text scaling feature, did you know?).


Tags: ,

When SQLite is not enough

SQLite is great for ad-hoc SQL DB-ing, but it’s not so great if you need to serve multiple processes (on writing, it locks the complete database).

It just so happens that, for a certain project, I need the ad-hocishness of SQLite *and* multiprocess write concurrency.

It appears that PostgreSQL can easily be made to run in “hassle-free” mode. Meaning: no need to ask your mum or your sysadmin anything, no need to write any config files.

Minimalism

Well, almost. I put in some extra options to disable the TCP socket, and I set some permissions on the unix socket.
For these examples, I assume bash or zsh and a unixish system. And PostgreSQL 9.0.

create a database:

initdb ~/mypostgres
You now have a directory ‘mypostgres’ in your homedir. It contains some config files with defaults. We don’t care.

start a server:

postgres -D ~/mypostgres/ -k ~/mypostgres/ --listen_addresses='' \
--unix_socket_permissions=0660 --unix_socket_group=$(id -g)
Let’s go over these options:

  • -D ~/mypostgres/ — Use ~/mypostgres as datadir (it’s where you created the database, so.)
  • -k ~/mypostgres/ — Use ~/mypostgres as the directory to store the unix socket for clients to connect to. Look closely and you’ll discover it (it starts with a dot for no good reason). Put it somewhere else (/tmp springs to mind) if you want other users to connect to your db, more on that later.

These are not strictly necessary:

  • --listen_addresses='' — Don’t listen on any TCP sockets. We don’t really authenticate users. So if you expose a TCP socket (default, on the localhost interface) any user can do anything to your databases just by running something like psql -h 127.0.0.1 -d postgres -U yourusername
  • --unix_socket_permissions=0660 — By default, permissions are o=rwx. If you put the socket in a public place such as /tmp, then with our authentication-free mode of operation, you probably do not want any and all users to be able to access your DB.
  • --unix_socket_group=$(id -g) — This is completely superfluous but it’s here so you don’t shoot yourself in the foot when blindly copy-pasting teh codez. $(id -g), of course, is your primary group ID and is what the group ownership of the socket will have been set to without this specification anyway (hence the superfluousness). So when would you use this? You’d use it when you want to let other users access your DB. You’d set the gid to some appropriate group that you and the other users belong to (and you’d put the socket in a public place, that much should be clear now).

some goodies:

  • --silent-mode=true — Daemon mode. To quit the server, get the PID from ~/mydb/postmaster.pid and send it a SIGTERM.
  • -F — Wicked fast writes mode. Also, wickedly trashed DB mode if the server is interrupted abnormally.

Create and connect

Now the server is up and running, create a database:
createdb -h ~/mypostgres mydb
and connect to it:
psql -h ~/mypostgres -d mydb
By default psql will try to log you in to the DB using the same username as your current unix username. Convenient, because that user has full privileges on the DB.
Should you need to specify the user as whom to connect, you can do so with -U.

Wrapper?

Just like with sqlite, you determine access to the DB by setting unix permissions. We do it on the socket, with SQLite you do it on the DB file itself.
So, how about a wrapper Python module to make all this just as easy as

import sqlite3
conn = connect("create_me_if_I_don't_exist.db")

Yes, one day. Fork off a db server process, terminate it using the atexit decorator. It just might be that simple.


Tags: , , ,

Argh, soft switches

So what’s a soft power switch anyway? Let’s start with the converse: a regular light switch, say, of the tumbler type. You switch it in the ‘on’ position and voila, light. There’s a power cut and off it goes. But the switch maintains ’state’, so when the mains come back on, so does your light!

This doesn’t happen with soft switches. Pushing a soft switch momentarily closes a contact, thereby generating a signal. Typically this signal is interpreted by some circuit, turning on your laptop. Or if your laptop is already on it might put it into standby. The effect of pushing this switch depends on context. The pushbutton has no state, it just generate an event when it’s being pushed. No persistent state here. And context might get lost — after a power failure, for instance.

Context bites.

In itself this is not a bad thing, but the devil is in the details. The implementation of the circuit part can make a geek’s life miserable. I had an external harddisk enclosure sporting one of those soft power switches. Connect power, push the button, it’s on. Power cut? Off it goes (of course). Power restored — it’s still off. But I wanted it to be on whenever there is power. Duct-taping the button in the ‘pushed’ position (hey, ’state’!) didn’t do much good as, in my limited understanding, this particular switch appeared to work by ‘pulling down’ a signal from the high state to the low state. The circuit will never be armed! With the button pushed down when power comes back on, the high state will never be reached as any potential will just leak away through the switch, dammit.
I spent some time trying to solder around this problem, trying to delay the bridging of the contact with all kinds of contraptions made of stuff I had lying around (positive feedback circuit from LEDs and an LDR, fun) to no other avail than eventually frying the controller circuit. I’m such a n00b with electronics, I can’t even build a proper delay circuit.

Laptop relay

But that didn’t stop me from soldering up a solution to the following problem. My old netbook is performing home server duties now, which is nice as it’s energy-efficient, quiet, and has builtin UPS. This is the plan: If the mains go down it can keep running. After a while it sets an RTC wakeup alarm and suspends itself. The RTC alarm will be set ~12 hrs into the future, it will power up and switch on the UPS which powers the ADSL modem, border router, and switch. That way it can collect some mail. And then it shuts down the UPS, sets another RTC wakeup alarm…. wash, rinse, repeat, until battery power runs out.

Problems

  • So what if battery power has run out, and mains power gets restored?

Then I’d have to physically push the button to power the thing back on. I don’t want that. But there’s no ‘restore power state after power loss” BIOS option on this laptop. Lucky for me, with this particular piece of equipment I can just permanently short the same pins that the power switch shorts ­— no timing trouble as with the harddisk enclosure here.

  • So what if mains power comes back on after the server has just decided to go to sleep for another XX hours?

Well that would be a silly situation. I need the laptop to resume from standby in that case.
This posed problems as the pin shorting trick mentioned before doesn’t help in this case. Potential just keeps on flowing through the ‘pressed’ switch, regardless of the state of the mains power. So restoring mains power doesn’t generate an event.

Solution

…was to use a relay. The coil is wired to the internal 20V leads coming from the power connector (right picture). klz helped soldering this rather tricky one. The relais switch leads are soldered onto the switch wires (left picture, you can see the little switch connector). In the middle (middle) you can see the relais. It’s huge and makes the bottom plate of the laptop bulge a bit.
But it works! It generates the equivalent of a button push whenever mains power comes on.


Tags: , , ,
© 2009-2011 Wicher Minnaard | electronic mail | theme: righteously modified "dark strict"