grml-live, fai and package retreiving
Hi,
I'm new to grml-live & fai. I don't understand why fai needs its own sources.list file. Since it will be really rarely for people to choose different mirrors from there preferred ones. Can we configure one less file (/etc/grml/fai/apt/sources.list), telling grml-live to use the official /etc/apt/sources.list by default instead?
,----- | To avoid downloading the base system again and again check out FAI's NFSROOT `-----
This is another point that I don't understand the fai's philosophy. The "normal" Debian will cache the downloaded packages. Removing and installing a package again and again won't cause the very package to be downloaded again and again. Why fai behaves this way? Are the package downloaded by fai cached at the official place?
Thanks, and for the excellent grml-live BTW.
* T o n g mlist4suntong@yahoo.com [20071110 18:49]:
I'm new to grml-live & fai. I don't understand why fai needs its own sources.list file. Since it will be really rarely for people to choose different mirrors from there preferred ones. Can we configure one less file (/etc/grml/fai/apt/sources.list), telling grml-live to use the official /etc/apt/sources.list by default instead?
No, because your chroot has absolutely nothing to do with your harddisc installation. :) You can build a Debian/stable based system on your Debian/unstable system and vice versa. And you can use a local mirror (check out approx/apt-cacher/... setup as described in the docs at http://grml.org/grml-live/) this way without modifying anything on your hd installation. It's not a bug, it's a feature! :)
,----- | To avoid downloading the base system again and again check out FAI's NFSROOT `-----
This is another point that I don't understand the fai's philosophy. The "normal" Debian will cache the downloaded packages. Removing and installing a package again and again won't cause the very package to be downloaded again and again. Why fai behaves this way? Are the package downloaded by fai cached at the official place?
What grml-live/FAI does as first step is setting up a minimal, basic chroot system using debootstrap. This chroot system is what I provide as http://daily.grml.org/base.tgz (for I386) / http://daily.grml.org/base64.tgz (for AMD64) - it's usually always the same chroot you are using as a base. So when you have an autobuild system like the one I'm running for http://daily.grml.org/ it would be stupid to download and install the same Debian packages for the base system via debootstrap again and again. That's why you have the option to use this base.tgz.
Using /etc/grml/fai/config/basefiles/AMD64.tar.gz vs. /etc/grml/fai/config/basefiles/I386.tar.gz (being the base.tgz files you will notice as "FAI's NFSROOT" in the docs) even gives us the flexibility to provide the base chroot system depending of the architecture/class we want to use/build for. If you want to you can customize the base system as well of course, thanks to the lovely design and features of FAI. :)
Thanks, and for the excellent grml-live BTW.
Great you like it. :)
regards, -mika-
On Sat, 10 Nov 2007 21:04:29 +0100, Michael Prokop wrote:
,----- | To avoid downloading the base system again and again check out FAI's NFSROOT `-----
This is another point that I don't understand the fai's philosophy. The "normal" Debian will cache the downloaded packages. Removing and installing a package again and again won't cause the very package to be downloaded again and again. Why fai behaves this way? Are the package downloaded by fai cached at the official place?
your chroot has absolutely nothing to do with your harddisc installation. :)
Ahhh, yes, silly me.
I didn't UTSL, am I right that the general procedure of grml-live would be
. setup chroot base . download/install all designated packages. . clean up, including removing all download packages then, . make iso
If so, can I suggest an option to bind mount a given apt archive into the chroot? That's what I always been doing when remastering. bind mount apt archive into the chroot, download/install packages, umount the bind mount so that we don't even need to do archive clean up.
Thanks, and for the excellent grml-live BTW.
Great you like it. :)
No I don't like it. I love it, a lot!
cheers
* T o n g mlist4suntong@yahoo.com [20071111 00:25]:
On Sat, 10 Nov 2007 21:04:29 +0100, Michael Prokop wrote:
your chroot has absolutely nothing to do with your harddisc installation. :)
Ahhh, yes, silly me.
I didn't UTSL, am I right that the general procedure of grml-live would be
. setup chroot base . download/install all designated packages. . clean up, including removing all download packages then, . make iso
Yepp.
If so, can I suggest an option to bind mount a given apt archive into the chroot? That's what I always been doing when remastering. bind mount apt archive into the chroot, download/install packages, umount the bind mount so that we don't even need to do archive clean up.
Check out "Can I use my own (local) Debian mirror?" at http://grml.org/grml-live/ (the manpage provides the same content, JFYI):
,---- [ http://grml.org/grml-live/ ] | If you want to use a local (for example NFS mount) mirror | additionally, just adjust MIRROR_DIRECTORY and MIRROR_SOURCES inside | /etc/grml/grml-live.conf as well. `----
Thanks, and for the excellent grml-live BTW.
Great you like it. :)
No I don't like it. I love it, a lot!
Hehe :)
regards, -mika-
On Sun, 11 Nov 2007 10:58:35 +0100, Michael Prokop wrote:
. . .am I right that the general procedure of grml-live would be
. setup chroot base . download/install all designated packages. . clean up, including removing all download packages then, . make iso
Yepp.
If so, can I suggest an option to bind mount a given apt archive into the chroot? That's what I always been doing when remastering. bind mount apt archive into the chroot, download/install packages, umount the bind mount so that we don't even need to do archive clean up.
Check out "Can I use my own (local) Debian mirror?" at http://grml.org/grml-live/ (the manpage provides the same content, JFYI):
,---- [ http://grml.org/grml-live/ ] | If you want to use a local (for example NFS mount) mirror | additionally, just adjust MIRROR_DIRECTORY and MIRROR_SOURCES inside | /etc/grml/grml-live.conf as well. `----
Ah, thanks. The MIRROR_DIRECTORY is outside chroot, correct? Will it get wiped when building the iso?
hmm, when trying to answer the above questions using the UTSL method, I noticed that it is not quite as what I proposed.
- One has to build a (local) Debian mirror first before he is able to use MIRROR_DIRECTORY and MIRROR_SOURCES. I haven't done that before. Will I have to pull in the full Debian mirror, not just the packages that I need? The bind mount method on the contrary, does not require one to setup anything, and does not require storing any unnecessary packages either.
- Using the MIRROR_DIRECTORY and MIRROR_SOURCES, grml-live still need to download packages into the chroot, then wipe the downloaded before building the iso. Even the mirror is local, there are still some quite heavy disk RW going on. With bind mount and up to date local apt archive, disk RW will be near-zero. Quite a performance boost, isn't it?
If I remember rightly there is a script in the debian archive to create a customised mirror. Maybe I'm wrong though.
Regards Moss
Tong wrote:
Ah, thanks. The MIRROR_DIRECTORY is outside chroot, correct? Will it get wiped when building the iso?
hmm, when trying to answer the above questions using the UTSL method, I noticed that it is not quite as what I proposed.
- One has to build a (local) Debian mirror first before he is able to use
MIRROR_DIRECTORY and MIRROR_SOURCES. I haven't done that before. Will I have to pull in the full Debian mirror, not just the packages that I need? The bind mount method on the contrary, does not require one to setup anything, and does not require storing any unnecessary packages either.
- Using the MIRROR_DIRECTORY and MIRROR_SOURCES, grml-live still need to
download packages into the chroot, then wipe the downloaded before building the iso. Even the mirror is local, there are still some quite heavy disk RW going on. With bind mount and up to date local apt archive, disk RW will be near-zero. Quite a performance boost, isn't it?
-- Tong (remove underscore(s) to reply) http://xpt.sourceforge.net/techdocs/ http://xpt.sourceforge.net/tools/
Grml mailing list - Grml@mur.at http://lists.mur.at/mailman/listinfo/grml join #grml on irc.freenode.org grml-devel-blog: http://grml.supersized.org/
* T o n g mlist4suntong@yahoo.com [20071111 18:21]:
On Sun, 11 Nov 2007 10:58:35 +0100, Michael Prokop wrote:
Check out "Can I use my own (local) Debian mirror?" at http://grml.org/grml-live/ (the manpage provides the same content, JFYI):
,---- [ http://grml.org/grml-live/ ] | If you want to use a local (for example NFS mount) mirror | additionally, just adjust MIRROR_DIRECTORY and MIRROR_SOURCES inside | /etc/grml/grml-live.conf as well. `----
Ah, thanks. The MIRROR_DIRECTORY is outside chroot, correct? Will it get wiped when building the iso?
Yes, it's just "mount --bind"-ed into the chroot und unmounted of course when building the ISO. No files will be deleted from the mirror of course, it's used just like a normal Debian mirror and accessed via apt-get/aptitude. :)
hmm, when trying to answer the above questions using the UTSL method, I noticed that it is not quite as what I proposed.
- One has to build a (local) Debian mirror first before he is able to use
MIRROR_DIRECTORY and MIRROR_SOURCES. I haven't done that before. Will I have to pull in the full Debian mirror, not just the packages that I need? The bind mount method on the contrary, does not require one to setup anything, and does not require storing any unnecessary packages either.
If all you need is a caching mirror just use approx/apt-cacher/... - it is really trivial to set up, just look in the docs for step-by-step instructions.
- Using the MIRROR_DIRECTORY and MIRROR_SOURCES, grml-live still need to
download packages into the chroot, then wipe the downloaded before building the iso. Even the mirror is local, there are still some quite heavy disk RW going on. With bind mount and up to date local apt archive, disk RW will be near-zero. Quite a performance boost, isn't it?
Hm, not sure whether I really got you, but is your suggestion to "mount --bind" the mirror directly to /var/cache/apt/archive of/for the chroot?
BTW: If you are interested in my own todolist for grml-live check out http://wiki.grml.org/doku.php?id=grml-live
regards, -mika-
On Sun, 11 Nov 2007 18:58:17 +0100, Michael Prokop wrote:
Ah, thanks. The MIRROR_DIRECTORY is outside chroot, correct? Will it get wiped when building the iso?
Yes, it's just "mount --bind"-ed into the chroot und unmounted of course when building the ISO. No files will be deleted from the mirror of course, it's used just like a normal Debian mirror and accessed via apt-get/aptitude. :)
Great, that's exactly what I thought things would work. One further question, in this case, Do I also need to set MIRROR_SOURCES as well? I think one MIRROR_DIRECTORY var would be enough for bind mount to work.
* T o n g mlist4suntong@yahoo.com [20071113 04:13]:
On Sun, 11 Nov 2007 18:58:17 +0100, Michael Prokop wrote:
Ah, thanks. The MIRROR_DIRECTORY is outside chroot, correct? Will it get wiped when building the iso?
Yes, it's just "mount --bind"-ed into the chroot und unmounted of course when building the ISO. No files will be deleted from the mirror of course, it's used just like a normal Debian mirror and accessed via apt-get/aptitude. :)
Great, that's exactly what I thought things would work. One further question, in this case, Do I also need to set MIRROR_SOURCES as well? I think one MIRROR_DIRECTORY var would be enough for bind mount to work.
Yes, currently you have to specify MIRROR_SOURCES as well. Just check out /etc/grml/grml-live.conf which provides example lines for you. (I didn't want to put too much effort into it because I'd like to merge that feature with main FAI for the future.)
regards, -mika-
Teilnehmer (3)
-
Maurice McCarthy -
Michael Prokop -
T o n g