[Grml] remastering a live cd, Can someone explain it to me?

[Michael Schierl]

[resend because I forgot to CC: the mailing list]

Doug Smith schrieb:
>  Why I wonder this is, I like to add in a few packages that I use.  I
>  would like to have my own disk with config files, new packages and
>  more so that I can just set up on any machine and rock and roll.  

If you just want to remaster it, follow one of the remastering guides.
If you want to change how the boot process works, you will have to dig
deeper. I'll try to sketch it below, but if you are really interested to
build your own live cd "from scratch", you will have to read a lot more...

>  I know how to make an iso-9660 file system, at least, well enough to
>  make a cd, but how do you make the thing bootable? 

There are, in theory, two ways.

1) create a bootable floppy image and use that one as an "emulation boot
image". The BIOS will copy it to RAM, emulate a floppy there and boot
from it. On that floppy there should be a bootloader that can boot Linux
from a CD. This works for any OS that can be booted from a floppy and
needs access to a CD, so it is the more traditional way. See mkisofs for
options how to build an iso with an emulation boot image.
2) Use a special bootloader for CDs, as a so called "no-emulation boot
image". (you don't understand the name if you do not know what option 1
is about...) The BIOS will copy the image into RAM verbatim and execute
it. Mkisofs can do that as well, but you have to check the bootloader
documentation for the offset (the position in RAM) where the image
should be loaded to.

In practice, nowadays you use only method (b). Popular boot loaders for
CDs are for example isolinux (used by grml), grub, and cdshell (which
can boot almost everything: Windows install and live cds, cd and floppy
images, and of course Linux).

>  How does the
>  technology work that is used to compress the software and
>  documentation down?

It is a compressed filesystem. It is read-only (similar to iso) but
compresses all files on disk. The filesystem itself is not stored
directly on a disk, but in a filesystem image which is on the CD and
appears a one file there. There are several compressed filesystems
available; most live cds use squashfs nowadays (which needs a kernel
patch since it is not in mainline yet), older ones used cloop.

Since the filesystem is read-only, you need some place to store
temporary data as well. The filesystem for this is tmpfs (which lives
just in RAM). Live CDs used to set up sophisticated symlinks so that all
read-only files are on compressed filesystem and all files that have to
be changed are copied to RAM at boot. But nowadays there is a better
solution called unionfs: It will take two filesystems and create a view
that unites both. All write access will go to the read-write filesystem
(i. e. RAM), unchanged files live on the compressed filesystem. This is
not limited to live cds, but can as well be used everywhere else where
you have a "base filesystem" you have to go back to regularly.

>  How does the thing know which parts of this
>  large file to use when accessing programs? 

Just like an on-disk filesystem knows what parts of your hard disk to
access to find your files.

The more tricky part is how to tell the kernel how to assemble the
filesystem out of three other filesystems. The solution is initrd (you
can also use initramfs in more recent kernels) which is loaded to RAM
(by the bootloader) and sets up the root filesystem. That means: Looks
for where the CD is (IDE, SCSI, somewhere else), mount it, mount the
compressed filesystem on it, create the unionfs, and give the new
unionfs as root device to the kernel.

Building an initrd is tricky because you will have to use tools on it
that work without having the full filesystem available. busybox is quite
useful for this. But usually you can use an existing initrd (or at least
modify an existing one instead of building one from scratch). On the
other hand, it can be very interesting to try to build one yourself -
but in that case you should have a PC emulator like VMWare or qemu
handy, because otherwise you will need lots of CDs and lots of time for
burning them.

>  If you don't have the time to explain this to me in full detail, do
>  you know of a good reference where I can just read about it and get
>  all the detæils?  

Start at the boot process howto (Umm, I can't find it at the moment). Or
if you want to learn more about Linux, try to follow a "Linux from
scratch" guide, which will tell you how to build a Linux that is on a
hard disk (including kernel, bootloader and everything) and does not use
any files from your current Linux distribution. Building chroot
environments (directories that do not allow access outside to prevent
malicious programs inside to use programs outside) helps as well because
it gives you experience what files are needed by which programs to run
(which is useful for building an initrd).

>  I am sorry to come on here with such strange questions, but I hope
>  there is a simply-understood answer for this one.  

Ask again if you do not understand the answers. Since no one here knows
what you know about Linux and what you don't, it is hard to give a
description that does not bore the intermediate used and does not ask
too much of a complete newbie (that never used a Linux console before)
either.

Michael