Pb segfault - stack trace

Source : http://www.proftpd.org/docs/howto/Debugging.html

Debugging Segfaults - (ProFTPd exemple)

If you see a message like the following in your logs:

golem.castaglia.org (127.0.0.1[127.0.0.1]) - ProFTPD terminating (signal 11)

then you have most likely encountered a segfault. This is a particular bad error, and can be hard to track down.

In version 1.3.1rc1, ProFTPD gained the ability to provide better logging to help track down these sorts of bugs. To enable this ability, you will need to configure proftpd with something like:

$ ./configure --enable-devel=stacktrace ...

and run your proftpd like normal. If a segfault occurs, the logs should show something like this:

golem.castaglia.org (127.0.0.1[127.0.0.1]) - ProFTPD terminating (signal 11)
golem.castaglia.org (127.0.0.1[127.0.0.1]) - FTP session closed.
golem.castaglia.org (127.0.0.1[127.0.0.1]) - -----BEGIN STACK TRACE-----
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [0] ./proftpd [0x809b1e1]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [1] ./proftpd(call_module+0x53) [0x8072c63]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [2] ./proftpd(strftime+0x14cf) [0x8051bef]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [3] ./proftpd(pr_cmd_dispatch+0x167) [0x8051f2f]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [4] ./proftpd(strftime+0x1fd3) [0x80526f3]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [5] ./proftpd [0x8053e12]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [6] ./proftpd [0x805484d]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [7] ./proftpd [0x8057975]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [8] ./proftpd(main+0x9d1) [0x8058625]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [9] /lib/i686/libc.so.6(__libc_start_main+0x93) [0x40076507]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - [10] ./proftpd(strcpy+0x31) [0x8051001]
golem.castaglia.org (127.0.0.1[127.0.0.1]) - -----END STACK TRACE-----

These stack symbols are generated automatically by glibc, and may or may not contain the function names.

The key here for tracking down the location of the segfault is that [0] frame, and the memory address: 0x809b1e1. Using that address and a very handy command called addr2line, you can determine the location of that address in the source code:

$ addr2line -e ./proftpd 0x809b1e1

In this example, I saw:

$ addr2line -e ./proftpd 0x809b1e1
/home/tj/proftpd/cvs/proftpd/modules/mod_auth.c:1723

which is the location of test code added to trigger the segfault.

This feature is specific to glibc systems, so keep that in mind. Other things to mention about this feature: it disables the changing of the process title that proftpd normally does. The obtaining of the stack symbols that glibc does uses the process title, and if this feature did not disable that process title changing, the output would be much less legible. Also, it does not work if the process which receives the SIGSEGV signal is chrooted. (Blame glibc for its bad handling of being chrooted.) To work around this, I would recommend using the mod_vroot module, just for a short period of time, in order to get a useful automatic stack trace.