Debug et Core dumped

01/2002

Expérience, Histoire vécue:

Un développement interne/ externe par plusiers SSII à base d'Uniface, Pro*C et C. Plus de 50000 lignes infligeant des core dumped toutes les 10 minutes, le tout basé sur

Ma mission, tenter d'estimer en 2 semaines la faisabilité d'un correctif.

La première partie du process a été de nous trouver des outils pour nous aider à débuguer. On détaille ces outils et leurs utilisations. En faite on recherche des outils pour le debuggage de malloc. Il en existe un certain nombre (ex: mpatrol, njamd, etc.) On note quelques difficultes pour les install suivanteselectric fence 2.2.2 et gdb sur cette OS/ gcc.

On spécifie les options de compilation -O0 -g3 -Wall (Warning maximum, et option pour inclure le source dans le binaire). On utilise gdb par ddd, on ouvre le binaire, on ouvre ensuite le core; on peut aussi prendre un process en cours de fonctionnement (très utile). On voit alors l'état de la pile juste après le plantage. Le problème est que les premières fuites mémoire se passe en général bien avant le plantage, ie le plantage est une conséquence de l'instabilité. On installe alors Electric Fence de manière à exiger un core dumped dès la première violation de mémoire. Dans le Makefile on ajonte -lefence pour l'utilisation de la mémoire protégée par la cloture electrique.

Avec ces réglages on lit le core, et là ..... La pile contient les fonctions malloc dans /usr/lib/libc.* et celle de oracle dans $ORACLE_HOME/lib/libclntsh.so

On lit alors des docs(ex: http://www.openldap.org/lists/openldap-devel/199909/msg00005.html (local)) sur SunOs et les applications multi-Thread. Ces docs mentionnent l'importance d'utiliser d'autre implémentation des fonctions d'allocation mémoire de /usr/libc.*, en particulier ces docs recommende l'utilisation du mtmalloc (faire man mtmalloc)dans mtmalloc.h. Pour utiliser ces définitions de fonctions d'allocations mémoires il suffit de rajouter l'option de compilation -lmtmalloc.

En somme on doit rajouter -O0 -g3 -Wall -lefence -lmtmalloc, avoir gdb, ddd et Electric Fence.

Maintenant on a une photo précise de la pile dès débordement, on peut donc corriger et débugger. A la suite de l'installation de notre outils de débuggage adapté à notre situation, 2 semaines supplémentaires on suffit à atteindre une fiabilisation de l'application supèrieur à 95% du temps d'exploitation. La décision de la direction à été de poursuivre la correction. Mission accomplie.

Cours: exemple njamd (http://sourceforge.net/docman/display_doc.php?docid=2573&group_id=18683 (local)

Doc Plomberie, Fuite de memmoire : electric fence, memprof:

Other tolls

dbx

Il existe des interfaces X (A GUI for several command-line debuggers.) pour GDB, DBX, JDB, WDB, XDB, the Perl debugger, and the Python debugger. par exemplecode_medic ou DDD:

The Data Display Debugger (DDD) is a popular GUI for command-line debuggers like GDB, DBX,
JDB, WDB, XDB, the Perl debugger, and the Python debugger. DDD allows you to view source texts
and provides an interactive graphical data display, in which data structures are displayed as graphs. You
can use your mouse to dereference pointers or view structure contents, which are updated every time the
program stops. DDD can debug programs written in Ada, C, C++, Chill, Fortran, Java, Modula, Pascal,
Perl, and Python. DDD provides machine-level debugging; hypertext source navigation and lookup;
breakpoint, watchpoint, backtrace, and history editors; array plots; undo and redo; preferences and
settings editors; program execution in the terminal emulation window, debugging on a remote host, an
on-line manual, extensive help on the Motif user interface, and a command-line interface with full
editing, history and completion capabilities.

dbx est un débogueur symbolique textuel.
Il permet de déterminer l'origine d'un plantage en étudiant le fichier core pour analyser la pile d'appel du programme :
dbx mon_prog core
On utilise la commande where au prompt qui apparait. En retour s'affiche la file d'appels des fonctions employées ainsi que le nom des fichiers
sources et les numéros des lignes ou ont eu lieu ces appels :
(dbx) where
dbx travaille de manière optimale sur un code compilé avec les options -O0 et -g3. Cela a pour effet d'inhiber les optimisations et de rajouter des
informations de déboguage dans le code objet. Ces informations permettent à dbx de voir les variables locales et de déterminer les numéros des
lignes du source. Autrement dbx ne pourra fournir que peu d'informations

dbx est un débogueur symbolique textuel.
dbx travaille de manière optimale sur un code compilé avec l'option -g (option -g3
recommandée) et sans optimisation (option -O0 recommandée). Cela a pour effet de rajouter
des informations de déboguage dans le code objet. Ces informations permettent à dbx de
voir les variables locales et de déterminer les numéros des lignes du source.

Introduction
dbx permet de déterminer l'origine d'un plantage en étudiant le fichier core généré pour
analyser la pile d'appel du programme :
ex : dbx mon_prog core.
On utilise la commande where au prompt qui apparait. En retour s'affichent la file d'appels
des fonctions employées ainsi que le nom des fichiers sources et les numéros des lignes où
ont eu lieu ces appels jusqu'à la fonction responsable de l'erreur :
(dbx) where
> 0 foo2(i = 5) ["/usr/tmp/test.c":44, 0x1000109c]
1 foo(i = 4) ["/usr/tmp/test.c":38, 0x1000105c]
2 main(argc = 1, argv = 0xffffffad78) ["/usr/tmp/test.c":55, 0x10001104]
3 __start() ["/shamu/crt1text.s":137, 0x10000ee4]
Dans ce cas, le programme test s'est planté à la ligne 44 du fichier source test.c en cours
d'exécution de la routine foo2, appelée à la ligne 38 de la routine foo, elle-même appelée à la
ligne 55 de la function main. Par la suite on utilisera d'autres commandes de dbx pour
obtenir les valeurs de variables et connaitre les raisons du plantage.

Voir le site http://www.crihan.fr/CRIHAN/calcul/par/doc/Doc/Descript_log/Debogueurs/dbx.html (local)

ddd (existe en package debian ie dpkg -l "*ddd*")

ljljk

TotalView: http://www.etnus.com/Products/TotalView/latest/index.html

Si ca crashe et que vous recuperez un ficher core, lancez en interactif totalview http://www.llnl.gov/computing/tutorials/workshops/workshop/totalview/MAIN.html (local):


Ca vous permettra de voir ou ca a plante et de contacter les bonnes personnes.

Liens:

extrait sur http://www.tuxfinder.com par d'une recherche sur google de dbx debug download

1. code_medic
A UNIX debugging environment.
Code Medic is a free graphical debugger that provides access to the power of gdb with an intuitive front end. It currently supports opening multiple source windows at once, setting/clearing breakpoints while the program is running, watching variables change in the variable tree as you step through code (even with nested structs), text searching through source, and integration with Code Crusader to provide a rapid, efficient develop-debug cycle.

2. ddd
A GUI for several command-line debuggers.
The Data Display Debugger (DDD) is a popular GUI for command-line debuggers like GDB, DBX, JDB, WDB, XDB, the Perl debugger, and the Python debugger. DDD allows you to view source texts and provides an interactive graphical data display, in which data structures are displayed as graphs. You can use your mouse to dereference pointers or view structure contents, which are updated every time the program stops. DDD can debug programs written in Ada, C, C++, Chill, Fortran, Java, Modula, Pascal, Perl, and Python. DDD provides machine-level debugging; hypertext source navigation and lookup; breakpoint, watchpoint, backtrace, and history editors; array plots; undo and redo; preferences and settings editors; program execution in the terminal emulation window, debugging on a remote host, an on-line manual, extensive help on the Motif user interface, and a command-line interface with full editing, history and completion capabilities.

3. ddd-doc
DDD documentation
This package contains various DDD documentation and a manual page.

4. ddd-dynamic
X interface to the GDB, DBX and XDB debuggers
Fully dynamically linked DDD binary that uses Motif 1.2 shared library - lesstif v0.88.0 or higher will work. The Data Display Debugger (DDD) is a common graphical user interface for GDB, DBX, and XDB, the popular UNIX debuggers. Besides ``classical'' front-end features such as viewing source texts, DDD provides a graphical data display, where data structures are displayed as graphs. A simple mouse click dereferences pointers or views structure contents, updated each time the program stops. Using DDD, you can reason about your application by viewing its data, not just by viewing it execute lines of source code. Other DDD features include: debugging of programs written in C, C++, Ada, Fortran, Java, Perl, Pascal, Modula-2, or Modula-3; machine-level debugging; hypertext source navigation and lookup; breakpoint, backtrace, and history editors; preferences and settings editors; program execution in terminal emulator window; debugging on remote host; on-line manual; interactive help on the Motif user interface; GDB/DBX/XDB command-line interface with full editing, history, search, and completion capabilities. DDD has been designed to compete with well-known commercial debuggers. For more info on DDD see http://www.cs.tu-bs.de/softech/ddd/

5. ddd-python
X interface to the GDB, DBX and XDB debuggers - The python debugger
Data Display Debugger - python debugger.

6. ddd-semistatic
X interface to the GDB, DBX and XDB debuggers
Dynamically linked DDD binary with Motif 1.2 library statically linked in, as supplied from lestiff v0.88.1. The Data Display Debugger (DDD) is a common graphical user interface for GDB, DBX, and XDB, the popular UNIX debuggers. Besides ``classical'' front-end features such as viewing source texts, DDD provides a graphical data display, where data structures are displayed as graphs. A simple mouse click dereferences pointers or views structure contents, updated each time the program stops. Using DDD, you can reason about your application by viewing its data, not just by viewing it execute lines of source code. Other DDD features include: debugging of programs written in C, C++, Ada, Fortran, Java, Perl, Pascal, Modula-2, or Modula-3; machine-level debugging; hypertext source navigation and lookup; breakpoint, backtrace, and history editors; preferences and settings editors; program execution in terminal emulator window; debugging on remote host; on-line manual; interactive help on the Motif user interface; GDB/DBX/XDB command-line interface with full editing, history, search, and completion capabilities. DDD has been designed to compete with well-known commercial debuggers. For more info on DDD see http://www.cs.tu-bs.de/softech/ddd/

7. ddd-static
X interface to the GDB, DBX and XDB debuggers
Completely statically linked DDD binary. The Data Display Debugger (DDD) is a common graphical user interface for GDB, DBX, and XDB, the popular UNIX debuggers. Besides ``classical'' front-end features such as viewing source texts, DDD provides a graphical data display, where data structures are displayed as graphs. A simple mouse click dereferences pointers or views structure contents, updated each time the program stops. Using DDD, you can reason about your application by viewing its data, not just by viewing it execute lines of source code. Other DDD features include: debugging of programs written in C, C++, Ada, Fortran, Java, Perl, Pascal, Modula-2, or Modula-3; machine-level debugging; hypertext source navigation and lookup; breakpoint, backtrace, and history editors; preferences and settings editors; program execution in terminal emulator window; debugging on remote host; on-line manual; interactive help on the Motif user interface; GDB/DBX/XDB command-line interface with full editing, history, search, and completion capabilities. DDD has been designed to compete with well-known commercial debuggers. For more info on DDD see http://www.cs.tu-bs.de/softech/ddd/

8. ElectricFence
Electric Fence C memory debugging library
Electric Fence is a libary that can be used for C programming and debugging. Package contain shared library libefence which can be loaded by LD_PRELOAD without relinking debuged program. Package contain also ef shell script which preloads libefence and runs program passed as parameter.

9. ElectricFence-static
Satatic Electric Fence library
Satatic Electric Fence library.

10. lslk
A lock file lister.
Lslk is a lock file lister. Lslk attempts to list all of the locks on the executing system's local files (i.e., on the active inodes). Install lslk if you need a utility for listing file locks.

11. ltrace
Tracks runtime library calls from dynamically linked executables.
Ltrace is a debugging program which runs a specified command until the command exits. While the command is executing, ltrace intercepts and records both the dynamic library calls called by the executed process and the signals received by the executed process. Ltrace can also intercept and print system calls executed by the process. You should install ltrace if you need a sysadmin tool for tracking the execution of processes.

12. memprof
Tool for memory profiling and leak detection
Memprof is a tool for profiling memory usage and detecting memory leaks. It can be used with existing binaries without need for recompilation.

13. mpr
Poor man's memory profile
mpr can be used to find malloc/realloc memory leaks and memory allocation statistics and patterns - it does not find memory corruption errors. It uses a simple, brute force strategy - log all memory alloc/free requests to a file and then post-process this log file once the program has terminated.

14. mtrace
Tool for tracing malloc()s
mtrace is a tool that was included with the glibc 2.1.2 library that is used to trace malloc()'s. This package include the perl script mtrace and libraries needed to use it.

15. strace
Tracks and displays system calls associated with a running process.
The strace program intercepts and records the system calls called and received by a running process. Strace can print a record of each system call, its arguments and its return value. Strace is useful for diagnosing problems and debugging, as well as for instructional purposes. Install strace if you need a tool to track the system calls made and received by a process.

16. xxgdb
An X Window System graphical interface for the GNU gdb debugger.
Xxgdb is an X Window System graphical interface to the GNU gdb debugger. Xxgdb provides visual feedback and supports a mouse interface for the user who wants to perform debugging tasks like the following: controlling program execution through breakpoints, examining and traversing the function call stack, displaying values of variables and data structures, and browsing source files and functions. Install the xxgdb package if you'd like to use a graphical interface with the GNU gdb debugger. You'll also need to have the gdb package installed.