MeeGo diary notes: Packaging the Das U-boot

My Odyssey into MeeGo packaging continues today with U-boot packaging.

Decision on the source
First of all for U-boot package I needed to decide which source to use. Since I am no big fan of private repositories (even the ones hosted publicly!), the mainline u-boot was an obvious choice. Thankfully sakoman and rest of the u-boot community have done a great job in making omap3 and omap4 support available in Mainline u-boot. And yep, it does have PandaBoard and obviously beagleboard support as well.

RPM saga
Since the rule of a lazy programmer is not to invent something someone already has done, spend some time with google searching for readily available RPM packages for u-boot. One of the first things I hit was that our friends at Freescale had done a nify spec packaging for imx processors for packaging u-boot. But, looking closely I was not too impressed, searching a bit more, hit on the openmamba project and the excellent packaging that Silvan Calarco has done for u-boot. Though it was done for mainstone, it was closest to what I wanted for MeeGo - Ability to build for multiple platforms and multiple architectures with minimal tweaks. A quick email exchange with Silvan, and he was completely ok with me reusing the same MeeGo - Thanks Silvan :).

Cross building rpm on a Ubuntu PC
Almost all folks had me convinced that I needed to switch to opensuse to build RPM packaging. after a couple of hours of playing around with virtualbox and attempting to install opensuse dvd, I decided that it was a waste of my time. Instead found the following:

sudo apt-get install rpm

And viola I have rpmbuild packaging. the first version of the spec file I wrote (very little modified from Silvan's original version) build with the following command:

rpmbuild -ba u-boot.spec --define="CROSS_COMPILE arm-none-linux-gnueabi-" --define="TARGET_BOARD omap4_panda" --define "TARGET_CPU arm" --target="arm"

(note: I was still ignorant of OBS realities at this point ;).. Remember, I am a newbie ;) )..

MeeGo Packaging gimmicry
Since I had my rpm package successfully building, I was all set to hit OBS and be done in a jiffy.. Boy, was I in for a surprise (at least for a spoilt kernel programmer turned newbie rpm packager).

So in I go to the good ol' build.meego.com following the rules as written in the wiki, and reading the guidelines just to get my bearings right,
a) I created my own u-boot project home:nm:bootloader:u-boot (within my namespace ofcourse as I mentioned yesterday).
b) I created my first package inside the project called u-boot
c) Then added a new repository u-boot-omap4panda with a create new repository. -I needed a rule to base it on, so Selected devel:kernel and just added armv7le - cortex a8(omap3) and a9(omap4) were both v7 anyways..
note: I should have created the repository as u-boot.. but I guess we can fix it later on..
d) I added the source.tar.bz2 and the .spec file using the web interface and bingo found that OBS was ready to build out of the box.. and it failed miserably.

On #meego-dev Stskeeps (one of the nice, friendly and helpful MeeGo maintainers) introduced me to some ground rules in OBS
a) I cannot not define a custom build flag TARGET_BOARD omap4_panda and TARGET_CPU=arm were no no.
b) I dont need to use CROSS_COMPILE in MeeGo as it in the backend uses qemu and native-like build (basically, it executes gcc which is internally linked to a cross-arm-gcc - nifty!)
or in other words,

rpmbuild -ba u-boot.spec

should build me a meego rpm for my platform.. But, wait a minute.. how does OBS know which platforms I intended it to build? Stskeeps pointed me to the kernel package as an example - exactly the same pain - n900, netbook etc need their own separate builds. in some cases individual patches to be applied independent of each other. Lets digress a little and understand how the trick works for kernel.

MeeGo kernel package spec file trick
Put in a simple terms:

obs looks for the spec file with the same name as the obs package name

In the case of kernel, the main package kernel is linked to packages kernel-ivi, kernel-mrst, kernel-n900 kernel-netbook etc. . The core package kernel has the spec files: kernel.spec, kernel-ivi.spec, kernel-mrst.spec, kernel-n900.spec, kernel-netbook.spec along with the same source.

when obs build kernel-n900 package, it does'nt use kernel.spec, instead uses kernel-n900.spec

Nifty - so in each spec file, you put in the stuff what you need for the platform! Problem solved. But, would'nt it be a maintainer nightmare to manage few 100 spec files each forgetting to update common stuff? the MeeGo kernel maintainers had a trick up the sleeve as well.

Use a common kernel.spec.in file to put all data you need, each platform specific data is marked with a tag @@. e.g.:

@@N900 Name: kernel-n900

which basically means introduce Name:kernel-n900 only for n900 spec file. This is provided to a simple perl script called makespec.pl -> it is a very simple tag replacement logic -> It checks if a file called N900 exists, if yes, it enables all lines marked with @N900

all this is wrapped in a Makefile which basically does the following:

touch N900
makespec.pl kernel-n900.spec
rm N900

so when you do a make in the repo, it generates platform specific spec files from kernel.spec.in -> and you now have spec files for each package(aka platform) and a single spec.in file to maintain.

The U-boot package spec
For u-boot I did copied the same logic - with the fact that I did not need an additional patch for panda (atleast at the moment). Yep, I know I can improve the script improve the Makefile, remove the touch+rm nonsense.. mebbe a little later, but for the moment, it is not a tragic blocker as it allowed me my primary goal - in addition to pandaboard, the mechanism should scale to any platform and architecture we need it to scale to. so my contents now changed to:

Makefile -> This is for me to generate the platform specific spec files
makespec.pl -> tiny modifications to allow Panda replacements
series -> this was for future patches that might come in.. I need to work this out..
u-boot-2010.09.rc1.tar.bz2 -> rc1 tarball from denx.com
u-boot-omap4panda.spec -> This is the generated file
u-boot.spec -> "generic" spec file: It does a PandaBoard build at the moment, we can figure things out when we enable something else
u-boot.spec.in -> the main spec file

So, if i do a make clean all my platform specific .spec files disappear. on doing a make a new set of platform specific spec files are generated from my u-boot.spec.in file. Nifty. Tested the build and it built fine with u-boot.spec file

Created a new package u-boot-omap4panda and linked it to home:nm:bootloader:u-boot and viola it built the same with u-boot-omap4panda.spec file instead of u-boot.spec file.

I needed to see if the logic would work with omap3beagle (my fav platform)
- modified the u-boot.spec.in, Makefile and makespec.pl files for adding Beagleboard relevant entries, did a new make and used osc to commit in the changes
- created a new package u-boot-omap3beagle and linked it back to home:nm:bootloader:u-boot

Build just great.. I then noticed that I had a few rpmlint warnings as well..

RPMLINT report:
===============
u-boot-tools.armv7l: W: package-with-huge-docs  77%

More than half the size of your package is documentation. Consider splitting
it into a -doc subpackage.

u-boot.src:143: W: macro-in-%changelog %{TARGET_BOARD}

Macros are expanded in %changelog too, which can in unfortunate cases lead to
the package not building at all, or other subtle unexpected conditions that
affect the build.  Even when that doesn't happen, the expansion results in
possibly "rewriting history" on subsequent package revisions and generally odd
entries eg. in source rpms, which is rarely wanted.  Avoid use of macros in
%changelog altogether, or use two '%'s to escape them, like '%%foo'.

u-boot.src:22: W: hardcoded-packager-tag Nishanth

The Packager tag is hardcoded in your spec file. It should be removed, so as
to use rebuilder's own defaults.

3 packages and 0 specfiles checked; 0 errors, 3 warnings.

Never liked warnings and decided to clean them up
a) looking at the Packaging Guidelines, realized that the %changelog section in the .spec had to go to .changes file - so bit of mucking around later, did the same thing like the spec file:
u-boot.changes.in is copied to u-boot-.change to ensure that the changelog is replicated in all package builds.
b) removed my name off the .spec.in regenerated and checked in the changes
c) split the documentation off to u-boot-docs package (so you now get a package for u-boot, one for tools(mkimage) and one for doc) - added a few of useful docs there as well.. mostly the generic ones.

Committed and checked in my changes, noticed that with every checkin, obs would automagically build up the linked up packages. final set of files:

Makefile
makespec.pl
series
u-boot-2010.09.rc1.tar.bz2
u-boot.changes
u-boot.changes.in
u-boot-omap3beagle.changes
u-boot-omap3beagle.spec
u-boot-omap4panda.changes
u-boot-omap4panda.spec
u-boot.spec
u-boot.spec.in

Bingo - for your enjoyment - the final rpm packages are here

TODO: to figure out how to get this to mainline

Osc Commands I used
Overall, I found osc extreamely useful once I had reasonable level of confidence by using rpmbuild on my Ubuntu box, I did not have to deal with webpage based updating etc.. osc was a savior there!

osc co home:nm:bootloader:u-boot

Checks out my u-boot repository

osc status

Tells me what files have been modified or if there are any new files what they were

osc add file1 file2 file3...

Adds the files to the list of changes to be pushed to my repo

osc ci -m "my commit message"

Commits my changes and pushes to my repo with the changes noted with commit message "my commit message"

Well thats it for today.. Overall, I've been amazed how easy it's been once I figured out the basic things to create a buildable package! hmm..