source: trunk/ash/TOUR@ 3327

#       $NetBSD: TOUR,v 1.8 1996/10/16 14:24:56 christos Exp $
#       @(#)TOUR        8.1 (Berkeley) 5/31/93

NOTE -- This is the original TOUR paper distributed with ash and
does not represent the current state of the shell.  It is provided anyway
since it provides helpful information for how the shell is structured,
but be warned that things have changed -- the current shell is
still under development.

================================================================

                       A Tour through Ash

               Copyright 1989 by Kenneth Almquist.


DIRECTORIES:  The subdirectory bltin contains commands which can
be compiled stand-alone.  The rest of the source is in the main
ash directory.

SOURCE CODE GENERATORS:  Files whose names begin with "mk" are
programs that generate source code.  A complete list of these
programs is:

        program         intput files        generates
        -------         ------------        ---------
        mkbuiltins      builtins            builtins.h builtins.c
        mkinit          *.c                 init.c
        mknodes         nodetypes           nodes.h nodes.c
        mksignames          -               signames.h signames.c
        mksyntax            -               syntax.h syntax.c
        mktokens            -               token.h
        bltin/mkexpr    unary_op binary_op  operators.h operators.c

There are undoubtedly too many of these.  Mkinit searches all the
C source files for entries looking like:

        INIT {
              x = 1;    /* executed during initialization */
        }

        RESET {
              x = 2;    /* executed when the shell does a longjmp
                           back to the main command loop */
        }

        SHELLPROC {
              x = 3;    /* executed when the shell runs a shell procedure */
        }

It pulls this code out into routines which are when particular
events occur.  The intent is to improve modularity by isolating
the information about which modules need to be explicitly
initialized/reset within the modules themselves.

Mkinit recognizes several constructs for placing declarations in
the init.c file.
        INCLUDE "file.h"
includes a file.  The storage class MKINIT makes a declaration
available in the init.c file, for example:
        MKINIT int funcnest;    /* depth of function calls */
MKINIT alone on a line introduces a structure or union declara-
tion:
        MKINIT
        struct redirtab {
              short renamed[10];
        };
Preprocessor #define statements are copied to init.c without any
special action to request this.

INDENTATION:  The ash source is indented in multiples of six
spaces.  The only study that I have heard of on the subject con-
cluded that the optimal amount to indent is in the range of four
to six spaces.  I use six spaces since it is not too big a jump
from the widely used eight spaces.  If you really hate six space
indentation, use the adjind (source included) program to change
it to something else.

EXCEPTIONS:  Code for dealing with exceptions appears in
exceptions.c.  The C language doesn't include exception handling,
so I implement it using setjmp and longjmp.  The global variable
exception contains the type of exception.  EXERROR is raised by
calling error.  EXINT is an interrupt.  EXSHELLPROC is an excep-
tion which is raised when a shell procedure is invoked.  The pur-
pose of EXSHELLPROC is to perform the cleanup actions associated
with other exceptions.  After these cleanup actions, the shell
can interpret a shell procedure itself without exec'ing a new
copy of the shell.

INTERRUPTS:  In an interactive shell, an interrupt will cause an
EXINT exception to return to the main command loop.  (Exception:
EXINT is not raised if the user traps interrupts using the trap
command.)  The INTOFF and INTON macros (defined in exception.h)
provide uninterruptable critical sections.  Between the execution
of INTOFF and the execution of INTON, interrupt signals will be
held for later delivery.  INTOFF and INTON can be nested.

MEMALLOC.C:  Memalloc.c defines versions of malloc and realloc
which call error when there is no memory left.  It also defines a
stack oriented memory allocation scheme.  Allocating off a stack
is probably more efficient than allocation using malloc, but the
big advantage is that when an exception occurs all we have to do
to free up the memory in use at the time of the exception is to
restore the stack pointer.  The stack is implemented using a
linked list of blocks.

STPUTC:  If the stack were contiguous, it would be easy to store
strings on the stack without knowing in advance how long the
string was going to be:
        p = stackptr;
        *p++ = c;       /* repeated as many times as needed */
        stackptr = p;
The folloing three macros (defined in memalloc.h) perform these
operations, but grow the stack if you run off the end:
        STARTSTACKSTR(p);
        STPUTC(c, p);   /* repeated as many times as needed */
        grabstackstr(p);

We now start a top-down look at the code:

MAIN.C:  The main routine performs some initialization, executes
the user's profile if necessary, and calls cmdloop.  Cmdloop is
repeatedly parses and executes commands.

OPTIONS.C:  This file contains the option processing code.  It is
called from main to parse the shell arguments when the shell is
invoked, and it also contains the set builtin.  The -i and -j op-
tions (the latter turns on job control) require changes in signal
handling.  The routines setjobctl (in jobs.c) and setinteractive
(in trap.c) are called to handle changes to these options.

PARSING:  The parser code is all in parser.c.  A recursive des-
cent parser is used.  Syntax tables (generated by mksyntax) are
used to classify characters during lexical analysis.  There are
three tables:  one for normal use, one for use when inside single
quotes, and one for use when inside double quotes.  The tables
are machine dependent because they are indexed by character vari-
ables and the range of a char varies from machine to machine.

PARSE OUTPUT:  The output of the parser consists of a tree of
nodes.  The various types of nodes are defined in the file node-
types.

Nodes of type NARG are used to represent both words and the con-
tents of here documents.  An early version of ash kept the con-
tents of here documents in temporary files, but keeping here do-
cuments in memory typically results in significantly better per-
formance.  It would have been nice to make it an option to use
temporary files for here documents, for the benefit of small
machines, but the code to keep track of when to delete the tem-
porary files was complex and I never fixed all the bugs in it.
(AT&T has been maintaining the Bourne shell for more than ten
years, and to the best of my knowledge they still haven't gotten
it to handle temporary files correctly in obscure cases.)

The text field of a NARG structure points to the text of the
word.  The text consists of ordinary characters and a number of
special codes defined in parser.h.  The special codes are:

        CTLVAR              Variable substitution
        CTLENDVAR           End of variable substitution
        CTLBACKQ            Command substitution
        CTLBACKQ|CTLQUOTE   Command substitution inside double quotes
        CTLESC              Escape next character

A variable substitution contains the following elements: