Browse Prior Art Database

Secure execution environment for untrusted programs using debugger technology Disclosure Number: IPCOM000009145D
Publication Date: 2002-Aug-09
Document File: 2 page(s) / 4K

Publishing Venue

The Prior Art Database

Related People

Wayne Gramlich: INVENTOR [+2]


[ IPCOM000000002S originally published 2001-05-01 16:55 UTC ] This disclosure describes a way to execute untrusted programs safely, including programs from an unknown source that may contain a computer virus, worm, trojan horse, or other malicious code. Using a debugger, any program's interaction with the computer may be monitored and limited, eliminating the need to scan the program for malicious content such as a computer virus. A running program must interact with the system it is running on. A malicious program may use these interactions to modify the system in undesired ways. Java, Python, and Tcl, among others, use a technology called "sandbox", where an untrusted program is run as a subset of another program, with the executing program limiting the interactions this program can perform. However, the tight coupling of the trusted and untrusted code, and the complexity of the trusted code, allow numerous opportunities for errors that the untrusted code can exploit. Any system where trusted and untrusted code share a process space (memory resources) is likely to be vulnerable. This invention, which we may call "padded cell", uses debugger technology to run untrusted programs safely. Debuggers are designed to run and control programs (which frequently contain errors that lead to bad effects) without letting the programs affect the rest of the system. If the operating system supports it, a debugger will set up a memory space that the untrusted code can run in safely, and can be programmed to intercept all attempts by the program to communicate with the rest of the system. This disclosure will describe how to implement this technology on the Unix operating system. Programmers skilled in other operating systems and debuggers will easily be able to adapt this invention to their system, although some systems (e.g. Windows 95) cannot provide the necessary separation between memory spaces. On Unix, this could be implemented through the "/proc" interface which is used by debuggers. Depending on the debugger implementation, it may be desirable to modify the debugger program, write a separate program using much of the debugger source code, or configure the existing debugger. None of these options requires exceptional skill; the last option will be assumed in this file. Common forms of attack include: 1) Creating files on the system that will later be used maliciously. 2) Reading files containing data that should be secret. 3) Opening network connections (e.g. to report to the owner, or carry out an attack on another system). 4) Interacting with other programs in a way that exploits their weaknesses (e.g. corrupting their memory with malicious instructions). 5) Misleading the user (e.g. putting up a window to encourage them to supply their password). All of these attacks require a program to invoke "system calls". For example, the first three require an "open" call. The debugger can be configured to intercept or trap calls such as "open" and prevent them from executing, or make more complex decisions as to whether to allow the call to succeed. If the call should fail, the debugger can make it look to the program like the call failed naturally (e.g. the desired file didn't exist). If the program is non-malicious and well-written it can continue running without the desired resource. It may be more efficient not to trap certain system calls. Unix implementations vary as to whether they support this; Solaris does. For example, if the program is allowed to open a file at all, "read" calls to that file need not be trapped. A "wrapper" can invoke and configure the debugger and run the code with a single command, or be invoked automatically. This may be part of a system-wide security strategy. The padded cell may provide various levels of permission according to access type or target, such as which directory of the file system. It may prevent or restrict X Window attacks, for example by using an X protocol proxy such as xfwp or Xnest. It can restrict the use of terminal emulators to avoid CURSES attacks. It may provide the user with various levels of options when the program does something suspicious. It is more flexible than chroot and more widely applicable than access control lists (ACL) (e.g. used by Multics). It may also provide extra security for trusted programs that handle untrusted data, such as web browsers, mail readers, network port daemons (mitigating the risks of buffer overrun attacks), and for interpreters for any language including Java, JavaScript, Visual Basic, and VBScript. The disclosure is continued, and these points are developed more fully, in the attached text. [ 000000002S 02S 2S ]

This text was extracted from an ASCII text file.
This is the abbreviated version, containing approximately 50% of the total text.

A "wrapper" can be built that allows the running of untrusted code with a single command: the wrapper starts and configures the debugger to build the padded cell, then tells the debugger to execute the code inside the cell.  This wrapper can be invoked automatically by programs, such as mail readers and web browsers, that frequently run untrusted code. 

The padded cell may provide various levels of permission to the untrusted code, including: no access to the file system; read-only access; read and delete; read, write, and delete.  These decisions can be made for any part of the file system individually, by file, directory, or directory tree, since the debugger can see the name of any file the code tries to access.

An untrusted program may try to access the windowing system and draw on the screen.  In Unix, this is done via the X Window protocol, which is accessed through a TCP/IP stream (even on local displays).  X presents a security risk both from misleading displays (such as fake password dialogs) and because X servers are complex and tend to run in privileged mode.  Attempts to open an X connection can be prevented entirely by trapping the connect system call.  If limited X functionality is desired, the communication over the connection can be piped through an X protocol proxy, such as xfwp to restrict traffic or Xnest to restrict screen access.  mmap system calls, sometimes used for faster direct frame buffer access, can be disabled.  Security of X systems is beyond the scope of this disclosure, but is not a new problem and much prior art exists elsewhere.

In order to avoid some of the security problems associated with terminal emulators (e.g.  vulnerability to CURSES), the padded cell software can restrict the terminal emulator to be one of the safe ones.

If a program tries to do something unexpected, the wrapper/debugger may ask the user whether to allow it, or simply disallow it automatically....