submit to programming reddit
 

(November 2013)

For the TL;DR crowd: I cannibalized an old ARM board, and made a Debian mini-server out of it. I now have an always-on gateway into my house (from wherever I am), which...
  • costs me, energy-wise, only about 3-3.5W
  • is always accessible even though I have a dynamic IP (via DynDNS)
  • has an Nginx web server so I can share things with the world
  • has an Exim mail server, so I can receive mail over (E)SMTP, store it in my house, and read it over SSH/mutt
  • can be SSH-ed into, which allows me to Wake-On-LAN my main desktop, whenever I need access to it
  • run long running web downloads (e.g. wget/rtorrent) from within screen-ed sessions, and share them over Samba to my house-bound devices (e.g. watch movies from my Android tablet)
  • SSH is exposed over sslh in my HTTPS port (bypassing several firewall limitations in various places)
  • etc.
It is one of my very best hacks - I loved building it, and don't know how I lived without it :‑)
MyBook WE image

Building a tiny ARM-based server

About a year ago, a no longer functional NAS device made by Western Digital came into my hands - a MyBook World Edition. This is a rather old, low-powered, ARM-based file server... and even though it is considered obsolete by today's standards, I knew the moment I got it that I would very much enjoy hacking it to do my bidding :‑)

I am a programmer by vocation, but I don't really believe in the separation of programming and administration - in my humble opinion, there are a lot of advantages in practising both.

Disassembling and soldering

First, I had to break the thing down, so I could build it up to what I wanted. Following in the tradition of many engineers before me, I slaved over it for half an hour and succeeded in disassembling it - only to realize afterwards that other kind souls had documented the process. Oh well - figuring things out on your own is the best part about all this :‑)

I proceeded to remove the two drives in the machine, and after attaching them to my external USB/SATA enclosure, run a quick SMART check on them: smartctl -a /dev/sdX showed that both drives had bad sectors (Reallocated_Event_Count, Current_Pending_Sector were both non-zero - that's not the kind of drive you can trust)... I therefore cannibalized an old 160GB external USB disk, and attached it to the empty enclosure.

Since this is an embedded platform, with no VGA or serial plug on it, I needed to find a way to monitor its boot process. After a bit of Googling, it turned out that the board in fact had soldering pads for traditional RS-232 serial interfaces. The "specs" provided by the almighty Internet were simple:

...and since we are way past the age of motherboards with RS-232 interfaces, I ordered a TTL/RS-232 cable from an online electronics store (translation: a cable that I can attach to the board on one end, and plug the other end into a normal PC USB port - where the serial interface will be accessible by any serial port program).

Serial interface
The board's serial interface, after soldering 3 pins and hooking 3 cables (red arrow)

Two days later, the package arrived - so I soldered the 3 pins (2,3 and 4) and hooked them up to the USB/serial cable[1].

On my main machine (an Atom 330 based ArchLinux), I attached the USB cable, fired up a serial interface program (minicom -D /dev/ttyUSB0 -b 115200), and then powered up the little board...

broken down into pieces
The game begins :‑) The board is sitting on the now naked drive enclosure.

Nothing happened.

Hmm...

U-booting from no BIOS whatsoever

I was expecting this to show some kind of BIOS, but maybe this tiny board had no such luxury... maybe it read everything it needed from the attached drives?

I mounted the old drives up to my main PC's USB/SATA enclosure, and sure enough, I saw clear signs of a Linux-based machine: the software RAID driver in my ArchLinux detected raid devices (cat /proc/mdstat showed information about the RAID structure). Apparently, MyBook had the two drives in a RAID formation - which I proceeded to successfully mount. There were 4 partitions in each of the two drives, clearly in a RAID1 mirror formation - with the 4th and final partition being the one with the "File Server" storage area.

I proceeded to copy the first three partitions (including the partition table) to my 160GB drive (via dd). I then used fdisk to fix the size of the 4th partition to be the remaining space of my drive.

Attaching the 160GB drive and booting again, minicom logged tremendous improvement:

Welcome to minicom 2.6.2

OPTIONS: I18n 
Compiled on Mar  5 2013, 16:40:55.
Port /dev/ttyUSB0, 12:22:35

Press CTRL-A Z for help on special keys

�NASOx_0800
Mon Aug  5 21:45:27 EEST 2013


U-Boot 1.1.2 (Jan 21 2008 - 08:50:09)

U-Boot code: 48D00000 -> 48D17648  BSS: -> 48D1B2B8
RAM Configuration:
Bank #0: 48000000 32 MB
In:    serial
Out:   serial
Err:   serial
Initialising disks
No FIS received from device 1
Detecting SATA busses:
Bus 0: Found first device OK
  Device 0: Model: TOSHIBA MK1652GSX  Firm: LV010J Ser#: 29GGF8WNS
            Type: Hard Disk
            Supports 48-bit addressing
            Capacity: 152627.8 MB = 149.0 GB (312581808 x 512)
  Device 1: not available

IDE read: device 0 block # 63, count 1 ... 1 blocks read: OK

...and proceeded to boot a normal Linux kernel.

Success!

Installing Debian (and some thoughts about Windows)

Having access to the Western Digital provided Linux of the board was a very good start. I then found a post by an amazing engineer who described how he hacked the board to do anything he wanted. Following his lead, I succeeded in using the GPL sources from Western Digital and built my own kernel and busybox-based mini-distro. I wanted more than just a toy, though ; and since I use Debian at work, I followed Mario's consolidated version of instructions ('Quick install steps'), and in 15min, installed the debootstrap-ed main parts of Debian on my 160GB drive.

<rant>

When it is possible, I always prefer doing things from the console - not only does it improve my knowledge of the OS I work with, it also allows me to do things over serial lines or SSH connections. In the case of this little ARM box, I applied the same knowledge that I am using for normal machines: editing /etc/network/interfaces, /etc/resolv.conf, etc.

Sadly, this is a skill that Microsoft all but destroyed, making people hopelessly dependent on "wizards". That, in itself would have been fine, if it were not for the inevitable side-effect: people clicking on buttons unaware of what is going on behind them, end up with systems that can only be rescued with "OS reinstalls". My own settings - since I work with UNIX systems - are simple files under /etc and my $HOME... and have always been living under revision control (e.g my main dot files and vim configuration), happily migrating over the last 15 years across many machines. Backing them up and restoring them - e.g. in brand new machine installations at work - is simply a matter of checking out files and folders from a repos...

That alone, from what I can see in my dealings with my poor Windows-locked brethren... is something they would kill for. And if they ever realized what apt-get does, and has been doing for decades...

But I digress - people reading this blog probably already know all this.

</rant>

I quickly setup the network interface, and got up to a working...

# apt-get update && apt-get upgrade

At that point, I knew the hard part was over - I now had a Debian/ARM distribution, which I could configure to do whatever I wanted.

Power? Not so much :‑)

Spec-wise...

# cat /proc/cpuinfo
Processor   : ARM926EJ-Sid(wb) rev 5 (v5l)
BogoMIPS    : 99.73
Features    : swp half thumb fastmult edsp java 
CPU implementer : 0x41
CPU architecture: 5TEJ
CPU variant : 0x0
CPU part    : 0x926
CPU revision    : 5
Cache type  : write-back
Cache clean : cp15 c7 ops
Cache lockdown  : format C
Cache format    : Harvard
I size      : 32768
I assoc     : 4
I line length   : 32
I sets      : 256
D size      : 32768
D assoc     : 4
D line length   : 32
D sets      : 256

Hardware    : Oxsemi NAS
Revision    : 0000
Serial      : 0000000000000000

...the machine definitely doesn't win any contest: it's very low powered, but that can also be seen as an advantage: nothing wrong with an always on "server" that consumes 0.7 watt when idling! With the 160GB drive added in the mix, I get up to 3-3.5W... not bad at all.

DynDNS

I am behind a DSL line, so my IP address is constantly changing. To make the tiny server accessible from outside, I opened up a (free) account on DynDNS. Since the board only has 32MB RAM, I chose against using the DynDNS Perl script, and instead used a native C client, inadyn:

# apt-get install gcc
...
# wget http://inatech.eu/inadyn/inadyn.v1.96.2.zip
# unzip inadyn.v1.96.2.zip
# cd inadyn
# make
...
# cp bin/linux/inadyn /usr/local/bin

I then configured it to run at startup, via cron:

# cat /var/spool/cron/crontabs/root 
...
@reboot /usr/local/bin/inadyn

...and set up my DynDNS credentials:

# cat /etc/inadyn.conf 
--username USER --password PASSWORD --alias UBER-SECRET.dyndns.org --update_period_sec 300 --background

That's it - once every 5 minutes (5 x 60 = 300), the tiny server communicates it's current IP address to DynDNS:

# host UBER-SECRET.dyndns.org
UBER-SECRET.dyndns.org has address AA.BB.CC.DD

So, now that I had a permanent Internet presence, it was time to setup "stuff".

Nginx

First of all, I installed nginx - and was now able to export everything that I wanted to friends/family - as long as they had a browser accessible.

The 32MB memory of the tiny ARM server and my 700 Kbit upstream DSL speed would never survive a Slashdoting, of course :‑) Still, there are other uses: I rsync-ed the photo folder of my (jailbroken, company-provided) iPhone, and published it in a password-protected Nginx folder...

# rsync -av mobile@iphone:/private/var/mobile/Media/DCIM/100APPLE/ \
            /var/www/nginx-default/Media/

My pictures therefore became accessible from anywhere in the world, just by browsing to my mini-server (and using the folder password). Nifty!

I added this rsync to a cronjob, so my iPhone's photos are rsync-ed automatically every night, while I sleep and the iPhone is charging.

Exim, mutt

Privacy issues not withstanding, it's nice to be able to have a mail presence that doesn't depend on others. apt-get install exim4, and my friends can now e-mail me at ttsiod@UBER-SECRET.dyndns.org. I read the mail over SSH, via 'mutt' - which runs fine even in this tiny little CPU.

<rant2>

In a world populated by self-respecting human beings, that would be the end of it ; unfortunately, even though GMail accounts accepted mail sent from me with no problems, others (e.g. Yahoo) considered me a spammer, since I was sending mail from the dark pits of hell (i.e. an IP belonging to a DSL line). How can they be sure that I am not a zombie Windows machine, infected with malware and serving The Spam Lords?

Neither SPF nor DomainKeys convinced them - so I switched my outgoing direction to use smarthost - and therefore route outgoing mail via my ISP.

(Sigh)

</rant2>

Firewalls and sslh

There are places where hitting my SSH server (e.g. to use it as a SOCKS proxy via ssh -D ...) is impossible, because there are firewalls in place that only allow HTTP traffic.

Initially, I tried exposing the server over HTTPS's port (443), but that was not enough. I ended up using sslh, which cleverly sits between a port and a number of daemons. In my case, since it speaks enough of the SSH and HTTPS protocols, it can determine when an incoming connection is hunting for SSH responses, and when for HTTPS responses - and tunnel the request to the proper local daemons (sshd, nginx). The firewall therefore sees me as a legitimate HTTPS site (which I am, thanks to Nginx) and lets ssh -p 443 ... pass.

Samba

I can download anything I want with my little server. rtorrent is a text-based torrent client that works fine, but since I am not exactly a mainstream guy, the things that actually interest me are usually found elsewhere. Mostly, I prefer gathering up interesting URLs and setting up scripts that use wget or youtube-dl to download them inside screen-ed sessions. This way, when I get back from work, I mount the tiny server's download folder from my jailbroken Android tablet (over Samba - i.e. with the usual mount -t cifs ...), and enjoy watching without network 'hiccups'.

In case you're wondering, I am currently watching Drew Neil's amazing Vimcasts series, and egghead.io's videos on AngularJS. Scrapping is relatively easy - I scrap enough HTML to find the video URLs, and then feed them to a wget with a rate limit - so that I don't overload the kind people that share these treasures.

Conclusion - UNIX glory

If you think about it, the end result is rather amazing - and done completely over ARM processors, not Intel ones:

In plain words, UNIX power put to use - in the tiny server, in my phone, in my tablet. All 3 of them, running on ARM processors. To be honest, I didn't expect that ; 15 years ago I was sure that Intel and MS had cornered the galaxy... but somehow, Linux managed to change all that.

I still have to jailbreak them and/or hack them to do my bidding, of course - the world is still not perfect.

But that's why this is fun :‑)

[1] Many thanks go to my good friend John Kydonakis, for his advice and help in everything electronics-related. You're awesome, John!


Discussion in Hacker News

Like my blog? Reward me with LTC at LMGiWMByLE8WbvvLES9biwRYsimWAKpGcu or via my Amazon wish list.
 
My CV  About me  Back to indexLast update on: Sat Mar 8 22:58:16 2014 (Valid HTML)

comments powered by Disqus