Bridging the Gap: Cross Compilation for ARM
There comes a time when the tools are available to complete a task but the way forward isn’t exactly clear. More exotic tools often lack complete or generalized tutorials, and it falls to the individual to find out how to use them for their own purposes without explicit guidance from others. Of course, it doesn’t have to be this way; so here’s another tutorial for you all on how to set up an ARM cross-compilation environment on Linux.
I’ve jumped off the Ubuntu bandwagon for the time being, so this tutorial will be written from the standpoint of an Arch user; that said, it should be fairly easy for other distributions to follow and perhaps even more so since I won’t rely on Debian packages or apt-get. My purpose in finding out how to do this was to compile binaries for a PandaBoard that I’m using as part of a robotics project, but the techniques don’t rely on the target platform either. In fact, this should serve as a quick and dirty guide to cross compiling for any platform, though I haven’t tested it against anything but ARM. With that out of the way, let’s get to business.
Key Concepts
In order to make informed decisions about our tooling, we need to understand what we’re going to do and how we’re going to do it. So, what are we trying to do?
As you probably know already, computers are not all built the same. There are several types of processors available which are suited for different tasks and use different instruction sets. This is most commonly seen in the 32 vs. 64 bit debate. These are two versions of the same instruction set, x86, which was originally developed by Intel. There are several others on the market today including ARM, PowerPC, and SPARC to name a few. Now, in order for the processor to execute your code, you need to translate the human readable source into a machine-executable form, which is a set of commands corresponding to the processor’s instruction set. This is done by the compiler. Generally you will compile code for a given processor on a machine using that processor; this is called native compilation. However, there’s no reason you couldn’t compile a given program for another instruction set on that machine; conceptually, the two are equivalent. This is called cross compilation, and is what we’re trying to do for ARM.
Why would you want to cross compile? One reason becomes apparent when you compare the resources available on different platforms. For example, my current development machine has a quad-core AMD64 processor which runs at 3.2GHz and 8GB of RAM. The PandaBoard, on the other hand, only has a dual-core 1GHz processor and 1GB of RAM. I can compile a program much faster on the larger machine than I can on the PandaBoard, even while cross compiling. Another reason is the availability of libraries and network connections on the target. I usually work in a university environment where access to internet-facing networks is restricted. It’s difficult for me to register new devices on the network so I can download libraries and tools. By cross compiling, I can build my tools on a pre-registered system and then transfer completed binaries to the target environment. As well, this saves valuable space and resources on the target; as I don’t need to maintain a full development environment, I can get away with a very minimal installation. Depending on what your target environment is and what tasks it needs to perform, you may find that the benefits of cross compilation for you are different; no matter your end goals, I think you’ll find it a useful tool to have at your disposal.
So, what do we need to cross compile a binary? We’ll need to have a compiler and linker for our chosen language that understand the target architecture. As well, we’ll require the usual standard libraries (libc and libstdc++ for C/C++ development) already compiled for our target, in addition to any 3rd party libraries we want to use. That should do for a basic set up, but it requires diligence on the part of the programmer to ensure that the correct environment is set up and that the correct tools and libraries are used for building the desired binaries. Fortunately, there are tools that we can use to simplify this; we’ll be using Scratchbox2 along with Qemu. Scratchbox2 allows you to specify a target development platform and set of tools to use and transparently converts system calls and command line arguments to match, and uses Qemu to emulate your hardware. Once we have a binary built, we’ll need to transfer it to the target system, preferably without relying on a network. Minicom and the lrzsz tools will provide file transfer services over serial connections for us.
Step 1: Get Your Tools
First things first: Let’s get the equipment we need. If you have a package manager for your distribution, I recommend using it as it simplifies this process considerably. If you don’t, you’ll need to build some of the packages manually (which shouldn’t be too onerous, but may get messy if you don’t have the dependencies installed already). I recommend you put all of the tools which you don’t get through the package manager into an ARM directory to keep them distinct from other tools.
The first tools we need are a cross compiler and a linker. For ARM I am using the CodeSourcery tools, available here. Download a copy and unzip them to your ARM tools folder. If you wish to avoid the proprietary license of those tools, you can use crosstool-ng to build a new set of tools; I leave finding documentation on that to you, although it doesn’t look too onerous to me. Once you have these tools, install them in the ARM folder.
We’ll also need to install minicom, scratchbox2, and qemu; for Arch minicom and qemu are available in [extra] and scratchbox2 is in the AUR. All three should be available in the Debian/Ubuntu repositories. If you’re unsure of how to install packages for your distribution, I highly recommend familiarizing yourself now. At this point you should also install debootstrap, which we will use to create a Debian ARM root filesystem to draw libraries from.
lrzsz, our transfer tools, will need to be cross compiled and installed on the target. If you wish, you may download the source archive now from here.
Step 2: Create Your RootFS
We’ll need to use an ARM root filesystem to draw binaries and libraries from during our builds. If you have a preferred rootfs for your target platform you might want to consider using that; we’ll be making a generic Debian one. If you decide to use your own, it should be as simple as extracting the filesystem to a directory as root (as it should contain device nodes, you’ll need superuser permissions to do it right). Otherwise, create a directory to store it (in the examples, ~/dev/arm/rootfs) and execute
sudo debootstrap --verbose --arch armel --foreign lenny ~/dev/arm/rootfs/armel http://ftp.at.debian.org/debian
to build the Debian rootfs.
Step 3: Create a Scratchbox2 Configuration
Ok, now that we’ve got the rootfs we’ll need to tell Scratchbox2 to use it. Think about what you want to call your configuration (I called mine “panda”, you can choose something else), cd into your rootfs directory and execute
sb2-init panda <path to your compiler>
To test it out, type sb2 to enter the default scratchbox. (Since we’ve only
created one, “panda” will be the default configuration; if you create another
you want to use as the default, use the -d flag to sb2-init to set it as
such. You can also specify the target configuration with -t
[SB2 simple panda] nick@disciple arm $
Now, we’ll compile a simple test program to see if everything’s working.
$ echo 'int main() { return 0; }' > test.c
$ gcc test.c
$ file a.out
a.out: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked
(uses shared libs), for GNU/Linux 2.6.26, not stripped
If all of that worked, you should have a basic C compilation environment. You can try it with g++ as well if you want to, or use a C++ version of the test program to ensure it all works; mine did on the first try. (If you run into trouble, leave a comment and I’ll do my best to help.)
Step 4: Build lrzsz
So, we now have a functioning Scratchbox2 environment. One of the cool things about SB2 is that it allows you full access to the host system while still transparently converting build commands for you. You can stay in the SB2 environment for now (when you want to quit, use exit), and go ahead and move the lrzsz source package into a convenient place for building. I would recommend against putting it in the rootfs so that you can keep it clean and make it easier to rebuild. Decompress the source package and cd into it.
Here comes the cool part: lrzsz uses the GNU Autotools to build itself, which is where Scratchbox2 excels. Execute the configure script, and make once it’s done configuring. If all goes well, you should be able to execute
$ file src/lrz
lrz: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked
(uses shared libs), for GNU/Linux 2.6.16, not stripped
Well done! We’ve built our file transfer tools. Copy lrz and lsz to your device manually; from here on out, we can use the serial port to transfer files. (Note: lrz and lsz depend on libc.so and ld-linux.so; if for some highly FUBAR reason you don’t have them on your target, you’ll have to figure out how to statically link those tools. Consider it pennance for your transgression.)
Step 5: Build Your Code
We’re done with the setup for our basic environment. You can now compile and upload ARM binaries, and most of the tools will work out of the box. One thing to keep in mind is that you need to use libraries that are compiled for ARM in addition to building your own tools for it. If you need a library, it may be best to simply compile it yourself. For my project I have built the entirety of the Boost libraries and Magick++, and both worked out of the box and were able to be linked statically into my application. If you only intend to compile for ARM, you might find it useful to modify any Makefiles or other build systems to point to your ARM library directories permanently. If not, you can add separate targets or custom configurations to build ARM-specific binaries. I leave that up to you.
Step 6: Upload
Ok, so we’ve got a binary we want to upload to our board. How do we do it?
Attach your device to the serial port on your computer, or use a USB-Serial
adapter. If your device does not automatically bind to /dev/modem, you’ll need
to either symlink it to that or use the -D
$ lsz foo.txt
will automatically download the file to the directory minicom has configured as a download directory. Read minicom(1) to find out how to configure it.
Next Steps
We’ve only covered how to set up a bare-bones cross compilation environment. There’s quite a bit more to learn out there, so here’s some suggestions on what to look at next:
Unfortunately, there’s not a coherent Scratchbox2 tutorial out there that covers the more exotic features. That said, there are man pages on all of the sb2 tools, and they should help clarify some of the details.
For getting cross compilation libraries on Debian, there’s a tool called apt-cross that can help you out.
It might be nice to have some sort of debugger on your development board… (hint hint)
Most of the issues with cross compiling can be avoided by using more portable languages such as Java and Python.
That’s all I’ve got for now, folks; if you found this helpful or you have any questions, comments, suggestions, or critiques please leave a comment and I’ll answer/converse/rebut them.