A.2 The IEEE 1003.2 POSIX Shell Standard

There have been many attempts to standardize UNIX. Hardware companies' monolithic attempts at market domination, fragile industry coalitions, marketing failures, and other such efforts are the stuff of history-and the stuff of frustration.

Only one standardization effort has not been tied to commercial interests: the Portable Operating System Interface, known as POSIX. This effort started in 1981 with the /usr/group (now UniForum) Standards Committee, which produced the /usr/group Standard three years later. The list of contributors grew to include the Institute of Electrical and Electronic Engineers (IEEE) and the International Organization for Standardization (ISO).

The first POSIX standard was published in 1988. This one, called IEEE P1003.1, covers low-level issues at the system call level. IEEE P1003.2, covering the shell, utility programs, and user interface issues, was ratified in September 1992 after a six-year effort.

The POSIX standards were never meant to be rigid and absolute. The committee members certainly weren't about to put guns to the heads of operating system implementors and force them to adhere. Instead, the standards are designed to be flexible enough to allow for both coexistence of similar available software, so that existing code isn't in danger of obsolescence, and the addition of new features, so that vendors have the incentive to innovate. In other words, they are supposed to be the kind of third-party standards that vendors might actually be interested in following.

As a result, most UNIX vendors currently comply with POSIX 1003.1. Now that POSIX 1003.2 is available, the most important shells will undoubtedly adhere to it in the future. The Korn shell is no exception; it's expected to be 100% POSIX compliant in its next release, which should be within the next two years. Therefore you should pay attention to the differences between the current Korn shell and 1003.2 if you write shell code that you would like to be portable in the not-too-distant future.

POSIX 1003.2 itself consists of two parts. The first, 1003.2, addresses shell script portability; it defines the shell and the standard utilities. The second, 1003.2a, called the User Portability Extensions (UPE), defines standards of interactive shell use and interactive utilities like the vi editor. The combined document-on the order of 2000 pages-is available through the IEEE; for information, call (800) 678-IEEE.

The committee members had two motivating factors to weigh when they designed the 1003.2 shell standard. On the one hand, the design had to accomodate, as much as possible, existing shell code written under various Bourne-derived shells (the Version 7, System V, BSD, and Korn shells). These shells are different in several extremely subtle ways, most of which have to do with the ways certain syntactic elements interact with each other.

It must have been quite difficult and tedious to spell out these differences, let alone to reach compromises among them. Throw in biases of some committee members towards particular shells, and you might understand why it took six years to ratify 1003.2.

On the other hand, the shell design had to serve as a standard on which to base future shell implementations. This implied goals of simplicity, clarity, and precision-objectives that seem especially elusive in the context of the above problems.

The designers found one way of ameliorating this dilemma: they decided that the standard should include not only the features included in the shell, but also those explicitly omitted and those included but with unspecified functionality. The latter category allows some of the existing shells' innovations to "sneak through" without becoming part of the standard, while listing omitted features helps programmers determine which features in existing shell scripts won't be portable to future shells.

The POSIX standard is primarily based on the System V Bourne shell, which is a superset of the Version 7 shell discussed earlier in this Appendix. Therefore you should assume that Korn shell features that aren't present in the Bourne shell also aren't included in the POSIX standard.

However, the Korn shell did contribute a few of its features to the POSIX standard, including:

• $((...)) syntax for arithmetic expressions.

• $(...) syntax for command substitution, except that the $(<filename) shorthand for $(cat filename) isn't supported.

• Tilde expansion (originally derived from the C shell).

The following Korn shell features are left "unspecified" in the standard, meaning that their syntax is acceptable but their functionality is not standardized:

• The ((...)) syntax for arithmetic conditionals. The arithmetic test operators introduced in Chapter 5, (e.g., -eq, -lt), however, are included.

• The [[...]] syntax for conditional tests. The external test or [...] utility should be used instead.

• The syntax for defining functions that this book uses. The other syntax shown in Chapter 4 (fname() instead of function fname) is supported; see below.

• The select control structure.

• Code blocks ({...}) are supported, but for maximum portability, the curly braces should be quoted (for reasons too complicated to go into here).

• Signal numbers are only allowed if the numbers for certain key signals (INT, TERM, and a few others) are the same as on the most important historical versions of UNIX. In general, shell scripts should use symbolic names for signals.

The POSIX standard supports functions, as shown above, but the semantics are weaker: functions do not have local traps or options, it is not possible to define local variables, and functions can't be exported.

The POSIX standard also has a few new features:

• The command lookup order has been changed to allow certain built-in commands to be overridden by functions-since aliases aren't included in the standard. (Recall that we had to use an alias as "decoy" for a function that superseded the cd command in Chapter 5.) Built-in commands are divided into two sets by their positions in the command lookup order: some are processed before functions, some after. Specifically, the built-in commands break, : (do nothing), continue, . (dot), eval, exec, exit, export, readonly, return, set, shift, trap, and unset take priority over functions.

• A new built-in command, command, allows you to use built-in commands other than the above even if there are functions of the same name.

• A new keyword, !, takes the logical "not" of a command's exit status: if command returns exit status 0, ! command returns 1; if command returns a non-0 value, ! command returns 0. ! can be used with &&, ||, and parentheses (for nested subshells) to create logical combinations of exit statuses in conditionals.

• The command unset -v is used instead of unset (without an option) to remove the definition of a variable. This provides a better syntactic match with unset -f, for unsetting functions.

Finally, because the POSIX standard is meant to promote shell script portability, it explicitly avoids mention of features that only apply to interactive shell use-including aliases, editing modes, control keys, and so on. The UPE covers these. It also avoids mentioning certain key implementation issues: in particular, there is no requirement that multitasking be used for background jobs, subshells, etc. This was done to allow portability to non-multitasking systems like MS-DOS, so that, for example, the MKS Toolkit (see below) can be POSIX compliant.