View source for Dietpc3 intro

== Shortcomings ==

* Comprehensive source code, patches and compilation automata are not yet available for DIET-PC 3, and manual compilation instructions are incomplete, so you're not going to be able to build your own binaries from scratch. Basically DIET-PC 3 is a binary-only distribution at present. This is purely a manpower issue, so please curb your GPL compliance complaints!
* DIET-PC 3 is less "stable" than DIET-PC 2 - not because it won't work reliably at any given point in time, but because the developer keeps changing his mind and restructuring architectural fundamentals, such that backward compatibility and reliable online upgrade often cannot be achieved. Hence it's unstable in much the same way that Debian Sid is unstable.
* DIET-PC 3 is technically incomplete in some areas. In particular, the current assembly mechanism is a stopgap until a new system can be written from scratch. The new mechanism will build persistent-disk "firmware" by default, rather than an initrd image, and may use something more user- friendly than kbuild's mconf. It's also possible that the run-script mechanism will be overhauled again (maybe using openrc?), and certain finishing touches (such as a boot splash mechanism and postinst/prerm package scripts) aren't there yet.
* DIET-PC 3 is generally less "hackable" than DIET-PC 2, because you have to rebuild an ipkg package every time you make a minor change.
* DIET-PC 3 is technically more complex than DIET-PC 2 in many respects (e.g. the assembly mechanism and run-script concept), so there is a steeper learning curve for developers. The additional complexity might also make it boot slightly slower.
* As previously noted, minimum hardware and software requirements have increased in DIET-PC 3, which may conceivably exclude its use in some situations.

== Quickstart ==

The first thing you need to do is check ipkg.conf to make sure that "arch" directives are set appropriately for the system that you wish to build. All systems use architectures "all" and "noarch"; the one after that (typically with a weighting of 10) is the important one. Currently supported platforms are: amd64 (a.k.a. x86_64), arm, armel (a.k.a. ARM EABI), i386 (a.k.a. x86), powerpc.

The target architecture need not be the same as the build host architecture (e.g. you can assemble an arm initrd on an x86 host), although at this stage I can't guarantee that this will work properly for big-endian targets (powerpc) and little-endian build hosts (others), or vice versa.

Next, run "make fetch-ipks". This will download all available binary packages for each architecture specified in ipkg.conf from the DIET-PC website to directories with the same name as the architecture.  <font color="red">Erratum: you will first have to add -H (or --span-hosts) to the wget options in the "fetch-ipks" target of the Makefile; without this wget will refuse to follow redirections to the new DIET-PC website.</font>

Check the value of the KERNEL variable in the Makefile, and ensure that it points to a subdirectory of the diet-pc directory containing the kernel (zImage/bzImage) that you wish to use for your target and its corresponding System.map file.  A few starter kernels have been included with the assembly frameworks.  Kernels really ought to be distributed as ipks also, but handling them is awkward because they are large and usually do not need to be included in the root filesystem (this depends upon the boot loader), and thus we do not want to confuse the system builder by presenting this as an option. So for now we are just using a variation of the DIET-PC 2 assembly procedure instead.

Having done this, all you need to do is run "make", but FYI, the underlying process for building initrds is broadly as follows:


# Extract package metadata from ipk files (make metadata).
# Construct a tree of kbuild config files using the package metadata (make kbuild).
# Run mconf against the config tree to allow the user to interactively select packages to be installed and (where applicable) where to install them, outputting a ".config" file (make config).
# Use the ".config" file to compute an ordered list of ipk files to install.
# Assemble an ext2 filesystem by running the "ipkg" utility on the host, installing to an offline target, for each of the selected ipk packages, in the order computed and the the location implicitly or explicitly specified (root, usr, usrlocal).  More accurately, the ext2 filesystem is constructed twice, once as a dummy run with no side-effects (in order to obtain accurate sizing information), and again to generate the penultimate filesystem image (and perform any side-effects such as installing files on non-volatile filesystems). This behaviour is exactly the the same as in DIET-PC 2.
# Convert and compress the ext2 filesystem to the desired format (cpio.gz, squashfs, cramfs, ext2+e2compr, ext2.gz) - just like DIET-PC 2 did.

There are some other Makefile targets that you may find useful:

* "make ipkg-tree" will extract all iPKG bundles for architectures named in ipkg.conf into subdirectories (one per package) of the "ipkg-tree" directory.

* Conversely, "make ipks" will recreate iPKG bundles from the data in "ipkg-tree", and place them in the relevant architecture directories (overwriting any files with the same name).

* "make distclean" will remove metadata and kbuild directories and well as installable images and temporary files. This will cause metadata and kbuild data to be regenerated when you next run make. You should do this if you change architecture settings in ipkg.conf, or if you create new ipk files or modify existing ones such that control file data is altered.


You can use a combination of these Makefile targets to perform simple edits of iPKG packages provided by the DIET-PC site. Although a best effort is made to provide generic defaults, some packages need site-specific configuration in order to work properly/at all, and the only way to make such changes permanent at build-time is to generate a new package.

To ensure that your customized package does not get clobbered by future downloads, you should always modify the version number in the "Version:" field of the package's control file (located in the CONTROL subdirectory) when doing this. Change the build number (the final part of the version number, after the last hyphen) to something specific to your site; don't just increment it. For example, you might change etc/aumixrc in the aumix package, and then change Version: from 2.8-1 to 2.8-fred1 in aumix*/CONTROL/control. For consistency and to prevent clobbering by future invocations of "make ipkg-tree", you should also rename the package directory accoordingly. "make ipkg-tree" doesn't care what the directory is called (it only uses information in the control file to contruct the package filename).

== Regions ==

DIET-PC 3 pushes the limits of ipkg. It seems that most ipkg-based distributions only use it to manage optional components, not core firmware that is loaded into RAM (the initrd image or similar). DIET-PC 3 uses ipkg for everything, so it is superficially much more like a mainstream O/S (Debian in particular).

The problem with this approach is that ipkg packages installed at runtime may be non-persistent (that is, they will disappear when the system is rebooted), or worse yet, that parts of the package will be installed persistently and part non-persistently such that the package will be functionally broken upon next reboot, and parts of it remain installed, consuming disk resources.

DIET-PC 3's three-tier design copes with this problem to a limited extent. DIET-PC 3 consists of three distinct regions - root, /usr, and /usr/local. The root region is assumed to be small, read-write and volatile, and the other two regions are assumed to be read-only, non-volatile and progressively larger (/usr/local being the largest). Ipkg regions need not map one-to-one to filesystems with these properties, however; it is just a design feature to make it easier to perform separation at these boundaries if you so desire. The only non-negotiable constraint is that the root filesystem must be writable. You can still combine all three regions into a single initrd if you prefer, or have a combined root+usr and separate /usr/local, or whatever.

DIET-PC ipkg packages install entirely into only one region; some packages can only be installed to a specific region, and some can be installed to any region. This is determined by the value of a "Destination" header in the control file; at present this header has no special meaning to the ipkg utility (only to script DIET-PC automation), but I intend to modify it so that it does. The kbuild interface allows you to optionally select the installation region for each relocateable package. The default in all cases is /usr.

When installing or removing ipkg packages at run-time, you should always explictly specify the (un)installation region (ipkg calls this a "destination") using the "-d" option, because - as noted above - ipkg doesn't yet honour "Destination" headers, and will always default to the "root" destination even if the package requires (un)installation to/from "usr" or "usrlocal".

The DIET-PC root filesystem is always nonpersistent, so if you do a "live" (un)installation of an ipkg package to any destination that resides on it, it will have no permanent effect. The change that you made will last only until the next reboot, and will then be undone.

To permanently alter the root filesystem, you have to perform a read-write (re)mount of the source filesystem from which the running system was derived (either the initrd.img file, for an initrd-based system, or /oldroot for a unionfs-based one) and then perform an offline ipkg (un)installation to/from that using the "-o" option. After that you will either have to reboot (initrd) or signal unionfs to reload its configuration (unionfs). In practice, however, offline uninstallation doesn't seem to work (yet) because of ipkg bugginess. 

Example:

 ipkg update
 mount -o remount,rw /oldroot
 ipkg install -o /oldroot -d usr lspci
 mount -o remount,ro /oldroot
 mount -t unionfs -o remount,incgen none /

Filesystems other than root can be updated "live", although they may need to be write-enabled first. For example:

 ipkg update
 mount -o remount,rw /usr/local
 ipkg install -d usrlocal lspci
 mount -o remount,ro /usr/local

Another thing that you need to know about the DIET-PC 3 runtime environment is that the contents of /usr/etc and /usr/local/etc get copied into /etc at boot-time. This is necessary because (a) /, /usr and /usr/local are different "destinations", and package contents are not allowed to span "destination" boundaries (e.g. a package installed to /usr can't own a file in /etc); and (b) we must assume that /usr and /usr/local are read-only, yet software and/or rc.d "scripts" may want to write to corresponding configuration files. This means that you can't make permanent changes to a file or directory in /etc if there is a corresponding file or directory in /usr/etc or /usr/local/etc; you have to change the latter instead.

== Kbuild ==

The kbuild system does not work very well as a package selection tool, because it drives dependencies the wrong way; it expects leaves to depend on roots, rather than the other way around. Its decluttering feature, whereby it hides packages for which prerequisiste dependencies have not been met, causes problems when you have complex dependency trees - you have no idea that a high- level option even exists until you meet all of its (often non-obvious) prerequisites, thereby causing it to appear. Kbuild's "select" feature isn't dependency-safe, so you can't easily use this to force selection of entire subtrees. What we really need is something more like dselect or aptitude, except that it must be able to operate on an offline root, and it must only report what it would do and not actually make any changes. Until somebody writes such a thing, Kbuild is the closest thing that I can think of to a ready-made solution. Perhaps the mconf front-end can be modified to "grey out" unavailable options rather than hide them. The X11 (QT) front-end, qconf, can do this, so you may wish to edit the Makefile to use qconf instead of mconf (but this will, of course, only work in an X11 GUI context).

Because of kbuild's limitations, I've had to construct ipkg2kconfig such that it turns on almost every package by default (to ensure that all packages are initially visible), so on your first build run you'll have to turn most of them off. Once you have constructed an initial .config, things are easier because you can reuse your previous settings on subsequent builds.

Ipkg2kconfig ought to translate conflicting package sets into radio button lists, but this is technically difficult to implement, and at present it doesn't. It's usually fairly obvious from the naming scheme that conflicting options are mutually exclusive or will override each other, but there are some non-obvious exceptions (such as smbd vs cifsd, ox xdmcp-bind vs xdmcp-session).

== Documentation and Source Code ==

To save space, ipkg packages don't include any manpages, and the only places you can find any documentation for complex scripts and multi-component packages are inline comments and the terse package description. Ideally, HTML versions of all relevant manpages should be posted on the DIET-PC website somewhere, along with descriptions of how some of the more complex packages and package interactions work. But for now, you're on your own.

You probably also want to know where the development environments, source code, patches and compilation scripts are.  Well, it's all still a bit chaotic.  Reference development platforms are in [http://dietpc.org/downloads/virtual_machines/ http://dietpc.org/downloads/virtual_machines/] and source code can be found in [http://dietpc.org/downloads/source/ http://dietpc.org/downloads/source/], but there isn't any automation that you can run to compile all of this.  Right now, all we have is a manual procedure located at [http://dietpc.org/build.html http://dietpc.org/build.html].  This is intended for DIET-PC 2, but you'll find additional DIET-PC-3-specific information commented out in this HTML file - try viewing it using a text editor rather than a browser. You won't find anything about Xorg 7.4 in here, though.

Ultimately, something better will be needed, because manual compilation is much too labour-intensive when trying to keep ports to multiple architectures in synch. Perhaps I can make use of Debian tools such as dpkg-buildpackage and release source code in deb format, or use BitBake (OpenEmbedded), or invent some automation of my own. However, I don't have enough time to do this at present, so initial DIET-PC 3 package releases will be binary-only.

== Run Scripts ==

The DIET-PC 3 bootstrap procedure is more complex than DIET-PC 2, because limitations of the ipkg packaging system make it necessary to apply what DIET-PC 2 called "deltas" at run-time rather than at build-time. Deltas could be applied using post-installation (postinst) scripts, but ipkg doesn't allow you to run postinst scripts when installing to an offline target, because environmental differences between host and target might prevent the scripts from working reliably. The same is true for any other packaging system you care to mention (dpkg, rpm etc).

Hence DIET-PC 3 has to apply deltas at run-time instead. To make this more manageable, DIET-PC 3 has a simple run-script (rc.d/init.d) concept. This is a bit of a misnomer, because they aren't really scripts and you can't run them. For efficiency reasons, rc.d "scripts" are actually just collections of shell functions/aliases that get sourced by the master rc script in alphabetical order. However, you can indirectly run these "scripts" by using a Red-Hat-like front-end wrapper script (/bin/service). As in DIET-PC 2, runlevels are only used for separating GUI environments and are not relevant to the bootstrap procedure. There isn't really any way to control what the default runlevel will be, other than explicitly specifying it on the kernel command line via the boot loader.