Browse Prior Art Database

Method to Provide Changes to the Aix Operating System Kernel For Multi-Process Debugging

IP.com Disclosure Number: IPCOM000101252D
Original Publication Date: 1990-Jul-01
Included in the Prior Art Database: 2005-Mar-16
Document File: 3 page(s) / 88K

Publishing Venue

IBM

Related People

Bohrer, KA: AUTHOR [+2]

Abstract

A method of changing the AIX* operating system is described which will allow AIX to support any method of debugging a program that is already running or the child of a program that is in debug mode. It will also support any method of debugging a program that performs an exec (overlays itself with a new program).

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 52% of the total text.

Method to Provide Changes to the Aix Operating System Kernel For Multi-Process Debugging

       A method of changing the AIX* operating system is
described which will allow AIX to support any method of debugging a
program that is already running or the child of a program that is in
debug mode.  It will also support any method of debugging a program
that performs an exec (overlays itself with a new program).

      Since AIX supports debuggers only when the program being
debugged is a child of the debugger, in order to provide debuggers
that can debug a program that is already running, or programs that
are not a child process of the debugger, changes are made to AIX
operating system by the new method.

      The method provides the following new system functions and
changes.
      proc.h:  Add new fields.
      a.  p_dpid - Debugger process id.  The debugger process id may
or may not be the parent of the program being debugged.  A new
ptrace(a) call will be made to set the p_dpid. This field will be
used by the kernel to know that an attached debugger is running and
allows a non-parent to be a program debugger. A parent may still be
the debugger.
      b.  p_dext - Debug extension flag.  A new ptrace() call will
turn this flag on and off.   This flag will indicate to the kernel
that multi-process debugging functions, such as fork() and exec...()
require special action.  When the flag is set, then the debugger
expects to debug the forked or execed program.
     ptrace - Letters represent new ptrace calls.  Numbers will be
assigned later.
a.  ptrace(a):  Add function that debugger is attached. This function
is similar to ptrace(0) except an external program makes the
ptrace(a) call as the program debugger.
      b.  ptrace(b):  Add function that debugger is to detach from a
process.
      c.  ptrace(c):  Add a function to get the program name. Used
when attaching a debugger to a program or when an exec has occurred.
      d.  ptrace(d):  Add request for new debugger to attach (re-at
tach).  Used to switch debugger.  Normally used when a program forks
and a different debugger is to be attached.
      e.  ptrace(e):  Add call to set multi-process debugging active
or not active.  This function sets flag p_dext (debug extended) in
proc.h.  May...