My Raspberry Pi setup

By , 30/08/2013 22:24

I have had a Raspberry Pi lying around for ages now and finally found some time to play with it. After quite some research I decided to first set up a basic system that I can easily use to start from when I think of a new usage for a Pi. Since I like keeping “full control” over everything I do it would mean setting everything up from scratch without relying too much on existing pre-built images. The only image I tried was (the excellent) Raspbmc to verify quickly that XBMC would run smooth enough to pass the WAT (Wife Acceptancy Test).

The system I had in mind when starting should provide the following:

  • Remote boot using NFS as root fs (thus enabling the usage of cheap, small SD cards)
  • Easily create a full SD card when stand-alone device is required
  • Gentoo based (yes, I do love that distro for its simplicity)
  • Distcc enabled for faster compilations (I don’t love Gentoo enough to let a Raspberry compile everything on its own)

 

Introduction

My server is currently equipped with a Xen Hypervisor allowing me to set up different virtual machines for different tasks. I created a new virtual machine (with the very original name build) to accommodate the needed extra functionality. I however already had virtual machines for NFS and other functionality so that part of the installation is not described. It is also of course not a requirement to have virtual machines or even start from a clean install to use this guide. I just put this disclaimer here to explain possible differences in host names below.

 

Basic system

In this first part I’ll describe the absolute minimum you need to get the Raspberry booting with a NFS root filesystem.

 

Creating the root fs

Create a place to put the root fs and unpack the stage3 tarball there. You’ll need to make this directory available over NFS. You need to use version 3 of NFS for root filesystems. Because I’d like to keep multiple versions of the root filesystem they are put into subdirectories together with any related files.

  1. # cd /mnt/
  2. # mkdir rasppi
  3. # mkdir rasppi/base
  4. # mkdir rasppi/base/image
  5. # mkdir rasppi/base/boot
  6. # wget http://distfiles.gentoo.org/releases/arm/autobuilds/current-stage3-armv6j_hardfp/stage3-armv6j_hardfp-20130816.tar.bz2
  7. # tar -xvjf stage3-armv6j_hardfp-20130816.tar.bz2 -C rasppi/base/image/

Edit the configuration files as you want. My configuration was as following when starting out.

etc/portage/make.conf

  1. CFLAGS="-O2 -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard"
  2. CXXFLAGS="${CFLAGS}"
  3. CHOST="armv6j-hardfloat-linux-gnueabi"
  4. USE="bindist"
  5.  
  6. GENTOO_MIRRORS="http://mirror.leaseweb.com/gentoo/ ftp://gentoo.tiscali.nl/pub/$
  7. SYNC="rsync://rsync.nl.gentoo.org/gentoo-portage"
  8.  
  9. FEATURES="buildpkg getbinpkg"
  10. PKGDIR="/usr/portage/distfiles/packages/rasppi"
  11. PORTAGE_BINHOST="file://${PKGDIR}"

etc/fstab

  1. /dev/mmcblk0p1          /boot           auto            noauto,noatime  1 2
  2. <nfsserver>:/rasppi/base/image           /               nfs             auto            0 0
  3. <nfsserver>:/portage     /usr/portage    nfs             ro,auto         0 0
  4. <nfsserver>:/distfiles   /usr/portage/distfiles  nfs     rw,auto         0 0
  5. tmpfs                   /var/tmp/portage        tmpfs   auto            0 0

The tmpfs in /var/tmp/portage is because otherwise emerge will sometimes fail with error messages on ipc. It also doesn’t hurt compilation times to do everything in memory. Of course you’ll have to remember that the memory is limited. If you want to compile openoffice in there, it won’t work 😉 You can always (temporary) mount some real storage to the directory to overcome that problem.

etc/shadow
Update the root password with an encrypted password that you know.

 

Installing boot loader

You can either download the needed files manually or install raspberrypi-firmware. Be aware that it installs to your /boot folder so it might clutter your existing installation. I copied the files to /mnt/rasppi/base/boot so I have a complete folder to copy to the boot partition. The kernel will also be installed there later on.

Edit the cmdline.txt to boot from NFS

  1. ipv6.disable=0 selinux=0 plymouth.enable=0 smsc95xx.turbo_mode=N dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 elevator=noop ip=dhcp root=/dev/nfs nfsroot=10.0.2.100:/export/rasppi/base/image,nfsvers=3,proto=tcp rw

I borrowed the following config.txt from Raspbmc

  1. arm_freq=800
  2. force_turbo=1
  3. gpu_mem=128
  4. disable_overscan=1
  5. start_file=start_x.elf
  6. fixup_file=fixup_x.dat

 

Creating build environment

Most guides set of with creating a suitable SD card, but since I wanted to roll my own kernel anyway I started with setting up what is needed for that: a crossdev based build environment. The objective is not to create a complete cross-compile environment. Instead, only the kernel will be built directly in the environment. All the rest will be built using an actual Raspberry Pi as host and distcc to offload the work.

First set up an alias to give you less typing work

  1. # echo alias rpmake="make ARCH=arm CROSS_COMPILE=/usr/bin/armv6j-hardfloat-linux-gnueabi-" >> ~/.bash_profile
  2. source ~/.bash_profile

Install crossdev

  1. # emerge -a crossdev

Compile the required toolchain

  1. # mkdir /home/rasppi
  2. # crossdev -S -v -t armv6j-hardfloat-linux-gnueabi —-ov-output /home/rasppi/

Now grab a beer and wait for a while…

 

Creating a kernel

Unmask and install the sources

  1. # emerge raspberrypi-sources raspberrypi-mkimage —-autounmask-write
  2. # dispatch-conf
  3. # emerge raspberrypi-sources raspberrypi-mkimage -a

Configure and build the kernel.

  1. # cd /usr/src/linux-3.6.9999-raspberrypi/
  2. # rpmake bcmrpi_defconfig
  3. # rpmake oldconfig
  4. # rpmake -j4

Install the kernel and modules

  1. # rpmake modules_install INSTALL_MOD_PATH=/mnt/rasppi/base/image
  2. # imagetool-uncompressed.py arch/arm/boot/Image /mnt/rasppi/base/boot/kernel.img

 

Creating SD card

As you only need to run the bootloader from the card you actually don’t need to do anything special with it, you could just format it as FAT32 and get it over with. However, since I want some devices to be standalone (but copy them from the base template we’re creating) I’ll also include creating the disk.

You should start off with a completely empty disk, remove all the partitions if there are any.

  1. # fdisk /dev/sdk
  2. Welcome to fdisk (util-linux 2.22.2).
  3.  
  4. Changes will remain in memory only, until you decide to write them.
  5. Be careful before using the write command.
  6.  
  7. Command (m for help): p
  8.  
  9. Disk /dev/sdk: 7969 MB, 7969177600 bytes, 15564800 sectors
  10. Units = sectors of 1 * 512 = 512 bytes
  11. Sector size (logical/physical): 512 bytes / 512 bytes
  12. I/O size (minimum/optimal): 512 bytes / 512 bytes
  13. Disk identifier: 0x000d4f0f
  14.  
  15.    Device Boot      Start         End      Blocks   Id  System

Enter expert mode and update the heads, sectors and cylinders. This is for better performance. You need to calculate the number of cylinders as floor(CARD SIZE IN BYTES / 255 / 63 / 512). The card size can be seen above in the disk info.

  1. Command (m for help): x
  2.  
  3. Expert command (m for help): h
  4. Number of heads (1-256, default 246): 255
  5.  
  6. Expert command (m for help): s
  7. Number of sectors (1-63, default 62): 63
  8.  
  9. Expert command (m for help): c
  10. Number of cylinders (1-1048576, default 1020): 968
  11.  
  12. Expert command (m for help): r

Now you can create partitions. You need at least a bootable FAT32 partition. The rest of the partitioning is completely free to choose, you can even include swap if you want.

  1. Command (m for help): n
  2. Partition type:
  3.    p   primary (0 primary, 0 extended, 4 free)
  4.    e   extended
  5. Select (default p): p
  6. Partition number (1-4, default 1):
  7. Using default value 1
  8. First sector (2048-15564799, default 2048):
  9. Using default value 2048
  10. Last sector, +sectors or +size{K,M,G} (2048-15564799, default 15564799): +128M
  11. Partition 1 of type Linux and of size 128 MiB is set
  12.  
  13. Command (m for help): t
  14. Selected partition 1
  15. Hex code (type L to list codes): c
  16. Changed system type of partition 1 to c (W95 FAT32 (LBA))
  17.  
  18. Command (m for help): a
  19. Partition number (1-4): 1
  20.  
  21. Command (m for help): n
  22. Partition type:
  23.    p   primary (1 primary, 0 extended, 3 free)
  24.    e   extended
  25. Select (default p): p
  26. Partition number (1-4, default 2):
  27. Using default value 2
  28. First sector (264192-15564799, default 264192):
  29. Using default value 264192
  30. Last sector, +sectors or +size{K,M,G} (264192-15564799, default 15564799):
  31. Using default value 15564799
  32. Partition 2 of type Linux and of size 7.3 GiB is set

The end result should look like this (don’t forget to safe!)

  1. Command (m for help): p
  2.  
  3. Disk /dev/sdk: 7969 MB, 7969177600 bytes, 15564800 sectors
  4. Units = sectors of 1 * 512 = 512 bytes
  5. Sector size (logical/physical): 512 bytes / 512 bytes
  6. I/O size (minimum/optimal): 512 bytes / 512 bytes
  7. Disk identifier: 0x000d4f0f
  8.  
  9.    Device Boot      Start         End      Blocks   Id  System
  10. /dev/sdk1   *        2048      264191      131072    c  W95 FAT32 (LBA)
  11. /dev/sdk2          264192    15564799     7650304   83  Linux
  12.  
  13. Command (m for help): w
  14. The partition table has been altered!
  15.  
  16. Calling ioctl() to re-read partition table.
  17.  
  18. WARNING: If you have created or modified any DOS 6.x
  19. partitions, please see the fdisk manual page for additional
  20. information.
  21. Syncing disks.

Format the partitions

  1. # mkfs.msdos -F 32 /dev/sdk1 -n RASPPI
  2. mkfs.msdos 3.0.16 (01 Mar 2013)
  3. # mkfs.ext4 /dev/sdk2
  4. mke2fs 1.42.7 (21-Jan-2013)
  5. Filesystem label=
  6. OS type: Linux
  7. Block size=4096 (log=2)
  8. Fragment size=4096 (log=2)
  9. Stride=0 blocks, Stripe width=0 blocks
  10. 478608 inodes, 1912576 blocks
  11. 95628 blocks (5.00%) reserved for the super user
  12. First data block=0
  13. Maximum filesystem blocks=1958739968
  14. 59 block groups
  15. 32768 blocks per group, 32768 fragments per group
  16. 8112 inodes per group
  17. Superblock backups stored on blocks:
  18.         32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
  19.  
  20. Allocating group tables: done
  21. Writing inode tables: done
  22. Creating journal (32768 blocks): done
  23. Writing superblocks and filesystem accounting information: done

 

Install files and boot

Copy the files from the boot folder to the boot partition and plug it in, it should boot! You’ll have a working Gentoo system with an NFS root filesystem. Adapt it any way you like (you should at least follow the Gentoo Handbook starting from after the chroot, skip kernel and bootloader) or even better: continue to read how I made my installation even more awesome.

 

To infinity & beyond!

First thing I did after booting was disabling the clearing of the console so that no boot error-messages were lost. You may want to re-enable this after you’re done for security reasons. Change the line for terminal 1 in /etc/inittab and add the –noclear parameter to the agetty command.

  1. # TERMINALS
  2. c1:12345:respawn:/sbin/agetty –noclear 38400 tty1 linux

While you’re there, also comment out the serial console lines as they trigger error messages INIT: Id “s0” respawning too fast: disabled for 5 minutes on your console.

After rebooting you’ll notice that there are indeed some issues you might want to resolve.

You should set the date manually after each reboot until you have NTP properly configured.

  1. # date 082923002013

 

Network configuration

Although you’re booting over the network your installation itself doesn’t know anything about it yet. This is especially annoying if like me you have your portage on a NFS share, you can’t do much else before fixing this.

/etc/conf.d/net

  1. dns_domain_lo="<your.domain.here>"
  2. dns_search_lo="<your.domain.here>"
  3. dns_servers_lo="<your DNS server>"
  4.  
  5. config_eth0="null"

Add the boot script for other services to work properly.

  1. # cd /etc/init.d/
  2. # ln -s net.lo net.eth0
  3. # rc-update add net.lo default
  4. # rc-update add net.eth0 default

Also set the hostname and domainname according to the Gentoo Handbook. Reboot and you should now be able to ping a hostname.

Since nfs-utils is not installed in the stage3 tarball you still can’t mount other NFS shares but you need the portage share to install nfs-utils. You could build it on the buildserver and then install the package manually but I took the easy way out: temporary put portage directly in the root NFS system.

  1. # rsync -av portage/ rasppi/base/image/usr/portage/

After installing nfs-utils I just cleared the folder again and mounted the portage NFS share. Don’t forget to add the nfsmount init script to the default boot profile or you will get a lot of Stale file handle errors.

 

Hardware clock

The Raspberry Pi doesn’t have a hardware clock, yet the config in the stage3 wants to use one. First disable it in /etc/conf.d/hwclock

  1. clock_hctosys="NO"
  2. clock_systohc="NO"

Also, you’ll want to configure your timezone as described in the Gentoo Handbook if you haven’t done so yet.

Of course you’ll want the correct time to be set otherwise you’ll run into a host of problems, so it’s best to setup NTP for that. There’s a guide available on that so I won’t cover it here. Best is though that you have your build server and Raspberry Pi pointing to the same NTP server so that you don’t get issues with building.

 

Distcc goodness

Now that the basics are in place it is safe to setup distcc so that the rest of the installation will take less time. Note that using distcc is not a guarantee for short compile times. The Raspberry Pi still has to handle things like unpacking, configuring and linking things.

If you want all that to be faster, you have to setup a real cross-compile that can roll complete packages or use a binary distribution. My aim is to have self-sufficient Gentoo images meaning that I can take one over to somebody else’s place and with minimal effort (checking out portage, disabling distcc) it can be immediately utilized.

Install distcc

Both on the build server and the Raspberry Pi you’ll have to install distcc

Configure build server

Configure /etc/conf.d/distcc (allow connections from your network), add the init script to the default run level and start the service.

Configure Raspberry Pi

If you still want to compile local configure /etc/conf.d/distcc (allow connections from localhost), add the init script to the default run level and start the service.

/etc/portage/make.conf

  1. MAKEOPTS="-j4"
  2. FEATURES="buildpkg getbinpkg distcc"

Configure distcc

  1. # distcc-config –set-hosts "10.0.2.31"
  2. # cd /usr/lib/distcc/bin/
  3. # rm c++ g++ gcc cc
  4. # nano armv6j-hardfloat-linux-gnueabi-wrapper
  5. # chmod 755 armv6j-hardfloat-linux-gnueabi-wrapper
  6. # ln -s armv6j-hardfloat-linux-gnueabi-wrapper cc
  7. # ln -s armv6j-hardfloat-linux-gnueabi-wrapper gcc
  8. # ln -s armv6j-hardfloat-linux-gnueabi-wrapper g++
  9. # ln -s armv6j-hardfloat-linux-gnueabi-wrapper c++

Contents of the wrapper script should be this script

  1. #!/bin/bash
  2. exec /usr/lib/distcc/bin/armv6j-hardfloat-linux-gnueabi-g${0:$[-2]} "$@"

Test it

You should be all ready. Start an emerge and see if it works. You can see if jobs are being submitted by using the following command on the Raspberry Pi

  1. watch -n1 DISTCC_DIR="/var/tmp/portage/.distcc/" distccmon-text

Or do as I did and the command to an alias in your profile for easy calling everytime.

Note that during configure portage will always compile locally.

Is it faster?

It should be. I did a test by re-emerging NTP.

Before:  25 minutes, 3 seconds
After: 17 minutes, 28 seconds

So you do see an improvement. Most of that time was actually spent in the configuration phase.

For those who are saying that this is bat-shit-crazy. It is, and to prove your point I’ll even help you: a native install of NTP on my build server only takes 1.5 minutes. That is not the point, you don’t get it, now get off my lawn.

 

Leftovers

If you haven’t done so according to the Gentoo Handbook yet:

  • locales
  • system logger
  • cron
  • remote access

 

It works, great! Now what?

I can’t tell you that, start building your own applications with the Pi. I’ll try to follow-up with my own adventures right here, first one probably being getting XBMC installed on it and linked into my existing (Atom-based) XBMC setup. Maybe after trying that I’ll revert to using binary distributions 😉

Starting with a new application is simple now: copy over the complete image to a new one (either on NFS or on storage card), point the kernel to it and you’re ready to go!

 

Todo

  • Find a graceful way to handle the binaries like boot loader, kernel, … Ideally they are updated on all the SD cards the next time an image is booted.
  • Roll a smaller kernel, only including the modules that I need.
  • Have a closer look to the tuning parameters

 

Resources

These are most of the helpful guides I used in getting to this point

Flattr this!

Leave a Reply

*

Panorama Theme by Themocracy