I never thought Linux was so breakable. We're moving mount points into RAM for performance ( /var/log /var/run /var/lock etc. ). Tmpfs or ramfs can stop disk thrashing and speedup a system. (Flash USB needs it even more, to minimize writes.)
Example fstab,
http://www.debian-administration.org/articles/179
But....brittle /etc/init.d scripts fail without having their private subdirectories persist across boots, in certain places like /var/log and /var/run.
If you write or test init.d scripts, please test with tmpfs and ramfs - do not assume pre-existence of application directories.
(P.S. Ubuntu Dapper btw already defaults to tmpfs for such directories, though not /var/log. Dapper uses old style init, Edgy/Feisty have something new.)
M
* Mark 27e3kk302@sneakemail.com [20070125 03:15]:
I never thought Linux was so breakable. We're moving mount points into RAM for performance ( /var/log /var/run /var/lock etc. ). Tmpfs or ramfs can stop disk thrashing and speedup a system. (Flash USB needs it even more, to minimize writes.)
Example fstab,
But....brittle /etc/init.d scripts fail without having their private subdirectories persist across boots, in certain places like /var/log and /var/run.
If you write or test init.d scripts, please test with tmpfs and ramfs - do not assume pre-existence of application directories.
If I understand you right you are blaiming initscripts with daemons that log to something like /var/log/$FOO/bar.log and complain if /var/log/$FOO does not exist? Well, that's something you cannot work around that easy...
If you mean something different please provide boot messages/errors (screenshots would be ok for me too) so we can take a closer look at it (even though that's not a grml specific problem but of Debian in general).
"Mounting an empty directory through tmpfs" to something like /var/log has to fail if you don't take further actions. Ah, looks like you are searching for overlays via unionfs. :)
regards, -mika-
If I understand you right you are blaiming initscripts with daemons that log to something like /var/log/$FOO/bar.log and complain if /var/log/$FOO does not exist? Well, that's something you cannot work around that easy...
Oh, I wish they would complain! They just fail mysteriously! But yes, that is the idea. Our shortcut was a master /etc/init.d/tmpfs-prep script making fresh directories for broken init.d scripts.
The real answer is for script authors to write defensive code. Init.d scripts already conform to a certain style convention so it should not be hard. These days of multi-gigabyte memory means running from RAM will get more and more common. For us, there is no reason to slow down apps with unwanted disk access just because some init.d scripts are sloppy.
"Mounting an empty directory through tmpfs" to something like /var/log has to fail if you don't take further actions. Ah, looks like you are searching for overlays via unionfs. :)
Not clear what that means. Here's what we want: small system directories causing frequent disk access and opening security risks mounted in RAM.
M
* Mark 27e3kk302@sneakemail.com [20070128 22:13]:
If I understand you right you are blaiming initscripts with daemons that log to something like /var/log/$FOO/bar.log and complain if /var/log/$FOO does not exist? Well, that's something you cannot work around that easy...
Oh, I wish they would complain! They just fail mysteriously! But yes, that is the idea. Our shortcut was a master /etc/init.d/tmpfs-prep script making fresh directories for broken init.d scripts.
The real answer is for script authors to write defensive code. Init.d scripts already conform to a certain style convention so it should not be hard. These days of multi-gigabyte memory means running from RAM will get more and more common. For us, there is no reason to slow down apps with unwanted disk access just because some init.d scripts are sloppy.
"Defensive code" might mean "do not start a service if you can not log to the specified logfile" as well. So: "depends"
"Mounting an empty directory through tmpfs" to something like /var/log has to fail if you don't take further actions. Ah, looks like you are searching for overlays via unionfs. :)
Not clear what that means. Here's what we want: small system directories causing frequent disk access and opening security risks mounted in RAM.
Try unionfs. Here's a short demo (a 5 minutes hack, I hope I did not overlook something):
What we want? Everything logged to /var/log should be written into a tmpfs mounted on /tmp/tmpfs.
Step 1: Let's log string "unionfs-test" to syslog for further investigation and make sure we don't have any file handles left:
# logger unionfs-test # Stop syslog-ng Stopping system logging: syslog-ng. # tail -3 /var/log/syslog Jan 28 22:51:10 funkenzutzler logger: unionfs-test Jan 28 22:51:14 funkenzutzler syslog-ng[3139]: SIGTERM received, terminating; Jan 28 22:51:14 funkenzutzler syslog-ng[3139]: syslog-ng shutting down; version='2.0rc3'
Step 2: Now set up tmpfs and unionfs:
# mkdir /tmp/tmpfs # mount -t tmpfs -o size=20% none /tmp/tmpfs # modprobe unionfs # mount -t unionfs -o dirs=/tmp/tmpfs:/var/log=ro none /var/log
Step 3: Now let's start syslog-ng and log string "unionfs running":
# Start syslog-ng Starting system logging: syslog-ng. # logger unionfs running # tail -1 /var/log/syslog Jan 28 22:52:20 funkenzutzler logger: unionfs running
Ok, logging was successfull, but is content of original /var/log still present?
# ls -la /var/log | tail -5 -rw-rw-r-- 1 root utmp 902016 2007-01-28 13:40 wtmp drwxr-s--- 2 root adm 4096 2006-09-03 13:28 xen/ -rw-r--r-- 1 root root 0 2006-05-09 08:10 xfs.log -rw-rw-r-- 1 root root 616086 2007-01-28 20:46 Xorg.0.log -rw-r--r-- 1 root root 575769 2007-01-15 17:30 Xorg.0.log.old
Yes, sir - that's why we call it overlay. :) But what's inside /tmp/tmpfs now?
# ls -la /tmp/tmpfs total 15792 drwxrwxrwt 2 root root 100 2007-01-28 23:06 ./ drwxrwxrwt 26 root root 12288 2007-01-28 23:06 ../ -rw-r----- 1 root adm 3927493 2007-01-28 23:06 messages -rw-r----- 1 root adm 12082844 2007-01-28 23:06 syslog -rw-r----- 1 root adm 113031 2007-01-28 23:06 user.log
Ah, only the stuff changed in /var/log went to our tmpfs, nice. Now let's umount tmpfs and unionfs:
# umount -l /var/log ; umount -l /tmp/tmpfs # tail -1 /var/log/syslog Jan 28 22:51:14 funkenzutzler syslog-ng[3139]: syslog-ng shutting down; version='2.0rc3'
Huh - where is our 'unionfs running' in syslog? :) Let's double-check:
# Restart syslog-ng Stopping system logging: syslog-ng. Starting system logging: syslog-ng. # logger unions not present # tail -5 /var/log/syslog Jan 28 22:51:10 funkenzutzler logger: unionfs-test Jan 28 22:51:14 funkenzutzler syslog-ng[3139]: SIGTERM received, terminating; Jan 28 22:51:14 funkenzutzler syslog-ng[3139]: syslog-ng shutting down; version='2.0rc3' Jan 28 22:52:59 funkenzutzler syslog-ng[16218]: syslog-ng starting up; version='2.0rc3' Jan 28 22:53:27 funkenzutzler logger: unions not present
Right - all changes during use of the unionfs overlay were written to tmpfs instead of /var/log on harddisk. Cute, nor? :)
regards, -mika-
"Defensive code" might mean "do not start a service if you can not log to the specified logfile" as well. So: "depends"
Defensive code always means: check for errors, take action as needed (e.g. cleanup), and message the caller/user/system. The let-it-explode philosophy would work only if every failure were clean.
Right - all changes during use of the unionfs overlay were written to tmpfs instead of /var/log on harddisk. Cute, nor? :)
Cute - and more complex than our workaround. But thanks for the tip! Nice to know.
M
participants (2)
-
Mark -
Michael Prokop