THE
DEFINITIVE GUIDE TO CHROOTING CONT...
STARTING THE SERVICE FOR THE FIRST TIME
Here is where our paths will diverge. As I mentioned earlier, the
process at this point consists of stepping through and correcting any
error messages we get one by one. Everybody's environment is different,
and so everybody will have different error messages. What follows is a
general guide on how to diagnose errors which may crop up as you are
starting services chrooted for the first time.
Correct Any Errors Appearing on the Console
First, just try running the service. You may get lucky and have it work
as is. Generally, if a daemon mode is available, choose to run it in
console mode instead. It will provide some kind of message, in most
cases, to tell you the service has started. If not, it may write some
error messages to the screen. First correct any error messages which
may appear on the console if the application does not start
successfully.
Look Through the Log Files for Errors
If no error messages are given on the console, or if you have corrected
them all, next check /var/log/syslog, /var/log/messages, and
/var/log/secure outside of the jail (which I hope is working, since you
have already set up the syslogd daemon to work with this application)
for any error messages which may have been sent to the logging dameon.
Quite often the service can be started using these error messages for
troubleshooting alone.
When All Else Fails
This is where it gets tricky. Assuming you can start the service when
not chrooted, but still can't start it in the chroot jail, there is
almost certainly a permissions problem or a missing file somewhere, but
you can't figure out where yet. You need to truss the program to find
any error messages returned by function calls which are not being
logged or written to the console. To do this, you must copy the strace
program from wherever it is located on your system to the static
binaries directory in the chroot jail. Place it in the static binaries
directory so it may be removed later easily. I suggest copying the less
and grep commands as well. You should already have the necessary
libraries in the jail to run these utilities, as they were likely
copied during the library installation process for the application
itself. If not, follow the library installation procedure above for
these tools as well.
To truss the application for any errors, run the following command,
assuming the main application binary is located in /chroot/[app]/bin.
(remember that the second argument to the chroot command is given
relative to the chroot root, not the system root):
Code:
strace chroot /chroot/[app] /bin/[app]
And you now have a whole bunch of gobbledygook to sort through for
error messages. This is where less and grep come in handy. You can just
pipe the whole damn thing though less, and examine it one screen at a
time, but here are some strings to grep for which may speed up your
search:
Code:
open
read
connect
ENOENT
ENOACCESS
I'm sure there are other good strings to search for, but I can't think
of any at the moment. Let me use MySQL as an example of how to use
strace to diagnose a startup error. After installing a basic
environment as described above, I attempt to start the mysql daemon
with the following command:
Code:
root@elizabeth:/chroot/mysql/var/mysql/mysql# chroot /chroot/mysql
/bin/mysqld -u root
051111 23:41:02 InnoDB: Started; log sequence number 0 43655
051111 23:41:02 [ERROR] Fatal error: Can't open and lock privilege
tables: Table 'mysql.host' doesn't exist
This is both good and bad. Bad, of course, because it's not working.
But this also means that all the necessary libraries are in place, else
we would see an error message complaining about missing libraries. My
first thought was that mysql.host must be a missing file somewhere, so
I checked the whole system for that file. Of course, it wasn't there.
mysql.host refers to the host table within the mysql database. Next, I
checked the /var/log/syslog, /var/log/messages, and /var/log/secure log
files for any useful information. No information about the error was
recorded in my logfiles, probably because the developers felt that the
console error message was sufficient. Unfortunately, I had no idea at
this point what was actually causing the error. So my next step was to
truss the mysqld daemon as it started up and check if there were any
files it could not find. I did so with the following command:
Code:
strace chroot /chroot/mysql /bin/mysqld -u mysql 2>&1 | grep open
This starts mysqld from within the chroot jail with the argument “-u
mysql”, which tells it to start as the mysql user. “2&>1” tells
bash to redirect the 2nd output stream to the 1st output stream (in
other words, redirect standard error to standard output), so I can
filter it with grep, which normally operates on standard input. Grep
then filters the output, displaying only those lines with the string
“open” in them. I got the following output:
Code:
root@elizabeth# strace chroot /chroot/mysql /bin/mysqld -u root
2>&1 | grep open
open("/etc/ld.so.cache", O_RDONLY) = 3
open("/lib/tls/libc.so.6", O_RDONLY) = 3
<snip>
open("./ib_logfile1", O_RDWR|O_LARGEFILE) = 7
open("/tmp/iboSN3lL", O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = 8
open("./mysql/host.frm", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such
file or directory)
open("./mysql/host.frm", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such
file or directory)
write(2, "051111 23:46:43 [ERROR] Fatal er"..., 108051111 23:46:43
[ERROR] Fatal error:
Can't open and lock privilege tables: Table 'mysql.host' doesn't exist
AHA! I can see the error message that was written to the console... but
it was not generated from a function call return error. This is merely
how MySQL interprets the error. More descriptive error messages would
have been nice. We can now see that in fact, the error is being caused
because MySQL expects to find a file called ./mysql/host.frm, which
means it is looking for it in a subdirectory of whatever directory it
is working in. I guessed, since this looks like a database file, that
it was in /var/mysql (within the jail), so it was looking at
/var/mysql/mysql, which is the mysql system database, located in the
database directory. MySQL stores a fair amount of information in its
own, self-titled database. I then searched my system for the host.frm
file. It was in /usr/local/mysql/var/mysql/mysql. /usr/local/mysql,
which is where I installed MySQL originally, so I would have an idea of
what the jail was supposed to look like for MySQL to run in it. I then
copied the entire MySQL database from /usr/local/mysql/var/mysql/mysql
to /chroot/mysql/var/mysql/mysql, and after doing so, the mysqld daemon
started without error.
Of course, there were other steps involved in setting up the MySQL
server, such as installing the database (which I apparently did in the
wrong place), but these are specific to MySQL. This document is
intended as a general guide to chrooting, which is why those steps were
not included.
This tutorial is licensed under the terms of the Creative Commons
Attribution-NonCommercial-NoDerivs 2.5 license
Original Tutorial
Submitted by Striek for TheTAZZone-TAZForum
Originally posted on June 11th, 2006 here
Do not use, republish, in whole or in part, without the consent of
the Author. TheTAZZone policy is that Authors retain the rights to the
work they submit and/or post...we do not sell, publish, transmit, or
have the right to give permission for such...TheTAZZone merely retains
the right to use, retain, and publish submitted work within it's
Network.