syst.eng. Anders Larsen

Looking for a Blog platform

2013-03-30T20:23:00+01:00

Some time ago I decided to join the blogosphere, having read Steve McConnell's Call to Action (at the bottom of the page): "If you are actively developing software, I urge you to write about your insights. If you have worked on a project that taught you valuable lessons, share them."

Unfortunately (I thought at the time), all user-ids similar to my preferred id were already in use on the public blog hosts (although many of them seemed abandoned), so I dropped the idea again.

After my web-hoster went south last month, I quickly set up Apache to host my old home pages and noticed that those pages hadn't had a decent update for almost a decade, so I decided to ditch them and instead host the blog myself and started looking for a suitable platform.

I soon found that most "mainstream" blogging platforms were either using MySQL, coded in PHP, or both.

My preferred database is PostgreSQL (which scores higher on the performance benchmarks and feature comparisons I've seen so far), and PHP is certainly not my first choice of programming language.

So I started looking for blogging platforms written in Python and quickly came across a post on Daniel "PyDanny" Greenfeld's blog - Daniel had already thoroughly researched the subject and had found an interesting solution: Pelican.

Pelican

Pelican creates static pages from Markdown or reStructuredText input - this eliminates the need for a database backend and even allows a very minimalistic web-server to host the pages (thereby eliminating several security risks).

As an added benefit, I can keep my input pages locally in Git.

Setting up Pelican was fairly straight-forward, as the documentation is detailed and complete - the hard part was to pick a theme among the available lot...

Pelican even sports an integrated web-server and can generate HTML output on the fly as soon as it detects that an input file was modified - this enables you to check the result of your edits practically in real time.

Port-based routing

2013-03-23T20:14:00+01:00

My current network setup looks somewhat like this:

                         ________     \            /     ________
          __________    |        |    /            \    |        |
_________|          |___| ADSL-  |____\  Internet  /____| root   |
  LAN    | firewall |   | router |    /            \    | server |
         |__________|   |________|    \            /    |________|
             ^                        /            \        ^
             :..............................................:
                              OpenVPN tunnel

My local Internet connection (left-hand side) does not have a fixed IPv4 address, so I used to send outgoing e-mails through my mail-hosting provider, who unfortunately went south last month.

The root-server on the right-hand side, which hosts some virtual machines, has a fixed, global IPv4 address, and since I control the DNS zone file, I decided to have it handle outgoing as well as incoming e-mail traffic.

One of the virtual machines on the root-server hosts a Postfix mail-server, another hosts an Apache web-server.

The easy part was to route incoming traffic to those VMs using iptables on the root-server:

# iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 25 -j DNAT --to-destination "IP of Postfix VM"
# iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination "IP of Apache VM"
# iptables -t nat -A POSTROUTING -o eth0 -j SNAT --to-source "global IP of root-server"

Now, routing outgoing e-mail traffic from my LAN through the root-server without passing all traffic through the VPN tunnel was a bit more tricky.

First, declare a new routing table for iproute2 on the firewall; let's call it "smtp" and assign it the (arbitrary) number 25:

# echo "25 smtp" >> /etc/iproute2/rt_tables

Next, when the VPN tunnel comes up, add a default route using the new table smtp and force marked packets through that route:

# ip route add default via "root-server IP on VPN" table smtp
# ip rule add fwmark 0x01 lookup smtp

(the above two commands could be placed in the OpenVPN "up" script).

Finally, mark outgoing SMTP packets using iptables on the firewall:

# iptables -t mangle -A OUTPUT -p tcp --dport 25 -j MARK --set-mark 0x01

Et voilà - SMTP packets originating on the firewall (which queues outgoing e-mails using Postfix) are routed through the VPN tunnel and exit from the root-server (using its global IP address).

With outgoing e-mail traffic originating from a fixed IP address, configuring DKIM and SPF is now possible.

Server outage

2013-03-04T22:01:00+01:00

For the past twelve years my domain has been hosted by a regional company (separate from my ISP), but although problems have been few and far between (and were always solved swiftly), I was forced to take over and handle it myself.

Here's a record of my ordeal:

Tue 2013-02-05, day one

At 09:15 UTC the lights suddently went out at my hoster's data center (but I didn't notice until the next day).

My logs show that my local server successfully fetched a batch of e-mails at 09:13 UTC, but two minutes later fetchmail stopped in the middle of the next batch.

Since then, all connection attempts timed out.

Wed 2013-02-06, day two

I wondered why I hadn't received any e-mails (not even the occational SPAM) for the past day and took a look at my server logs.

Having noticed the timeouts I tried to contact the hosting company, but their land lines (VoIP) were dead. I couldn't reach their home-page either, so I wondered if perhaps their backbone was damaged.

Not knowing what was going on or even if the problem was transient or not, I patiently waited.

Thu 2013-02-07, day three

Still no connectivity to the hoster and no news whatsoever about the incident, so I started preparing for the worst.

As the hoster was located in my region (less than 20 km away) I decided to drive by after work and have a look at the surroundings.

I got there about half an hour before the end of their office hours, but I could not spot neither light nor any other sign of life in the entire building, so I decided to act.

At the advice of a knowledgeable collegue I opened an account with the domain registration robot of the German hoster Hetzner Online AG in order to move my domain elsewhere.

I placed the order rather late in the evening, so I just had to wait until the next day.

Fri 2013-02-08, day four

I received the access code to the domain registration robot in the morning and immediately initiated a domain transfer.

Alas, my request was rejected; it turned out I had to provide an authorisation code which the domain registrar is supposed to provide.

Now things threatened to get out of hand - the old hoster (still unreachable) was also the registrar, so I had to quickly learn how the Internet works (organisationally, that is).

My hoster was a member of CORE Internet Council of Registrars in Geneva, so I emailed them that my hoster/registrar was Missing In Action and asked for help.

The person from the CORE secretariat was friendly and helpful; after having cleared some formalities regarding proof of identity I was handed the precious authorisation code, only two hours after the first contact.

With the authorisation code at hand, the robot finally accepted the domain transfer request.

The confimation stated that the actual transfer would only take place after five days (unless the previous registrar would ACK the transaction earlier, but alas he was still unreachable).

I guess I could have expedited the transfer (by contacting CORE again), but as it was now friday afternoon and I still hadn't decided where to host my domain, I instead started searching for a new hoster.

Sat 2013-02-09, day five

I spent an hour or two looking for and comparing virtual servers and root servers, since by now I had more-or-less abandoned the possibility of selecting a new web-hoster but instead decided to regain complete control by renting a server and configuring the web- and email-services myself.

Sun 2013-02-10, day six

I decided to rent the smallest (read: cheapest) root-server from Hetzner, still a fairly powerful box (i7 quad-core, 16 GiB RAM and 2 x 3 TB disk drives), suitably for hosting a handful of virtual machines.

My plan was to compartmentalise the services (e-mail, web, whatever), each in its own VM.

Mon 2013-02-11, day seven

The server was ready, complete with a Ubuntu 12.04 minimal installation.

Having restricted access according to my wishes the next step was to install and configure OpenVPN on the server to get a comfortable routing to and from my LAN.

Next I installed qemu-kvm and prepared a 'template'-VM, also with Ubuntu 12.04.

I then cloned the template and configured my new mail-server.

Since my domain was still lingering awaiting transfer, I could only test that local emails were handled correctly, though.

Tue 2013-02-12, day eight

I configured the next VM as a web-server and uploaded the backed-up content of my home-page.

Using iptables I redirected incoming traffic on port 25 to the mail-VM and port 80 to the web-VM, then verified the connectivity.

Wed 2013-02-13, day nine

Early in the afternoon I got the longed for message that my domain was finally transferred, so I configured DNS, logged into the mail-VM and let /var/log/mail.log run by.

Less than a minute after having submitted my DNS zone file, the first e-mail arrived: SPAM...

At any rate, it meant that after nine days I was now up-and-running again.

Oh, just noticed that my new server came with a global /64 IPv6 subnet! Hmmm...

The QNX4 file-system in Linux

2004-02-03T21:01:00+01:00

The QNX4 file-system option in Linux allows you to mount QNX4/QNX6-formatted floppies and hard-disk partitions on your Linux box and access the files as if they were on your native file-system.

It was originally written by Richard A. Frowÿn and Frank Denis, who - due to lack of time - passed the maintainership on to me in december 1999.

I fixed the last severe bugs in Kernels 2.2.15 and 2.3.35; previous versions were quite buggy and did not support fragmented files at all.

Starting with Kernel 2.4.19 it is possible to mount partitions created with QNX version 6.1 and above.

A bug that got exposed by subtle changes to the Virtual File System can cause a complete system hang when using the QNX4 file-system in Kernel 2.4.8 and 2.4.9 - the solution is to comment out line 399 of fs/qnx4/inode.c (the problem was fixed in Kernel 2.4.10)

There's an alternative implementation which supports writing to the QNX partition at http://qnxfs.narod.ru/, but I don't use it myself and cannot guarantee that it will work for you.

mkQNXnbi

2003-12-29T22:41:00+01:00

I wrote mkQNXnbi, a utility to allow net-booting QNX4 using Etherboot - a versatile, open-source dhcp/bootp Ethernet boot-ROM package.

mkQNXnbi is included with the Etherboot-distribution tarball (look into the directory /contrib/mkQNXnbi/).

Booting Linux on a WindowsCE device

2001-03-13T20:23:00+01:00

I've written a utility to boot Linux 2.2.x via the WinCE loader on the credit-card size PC "DIMM-PC EC" (manufactured by the German company Gesytec GmbH

This module utilizes an AMD Élan SC410 PC-on-a-chip, which, unfortunately, has some peculiarities.

The company Pengutronix provides patches against 2.4.13 .. 2.4.17 which fixes a clock drift problem and provides a work-around for a nasty serial interface bug.

The patches were merged into 2.4.18, so you probably don't need them separately, though.