pedagogy : POSIX Shell Command Language / SCL

WIP:

When teaching (shell command language) / This shit should be taught systematically. But no ... "let then figure it out" ... yeah, that's how you waste a lot of time.

study chart for the POSIX Shell Command Language 

- in today's "things you didn't need to see" 

Download PDF 

0. Portable Operating System Interface (POSIX) vs. Linux ; kernel vs. shells

• POSIX is a standard of architecture for computer operating systems, the current version of which is POSIX.1-2024
• POSIX refers mainly to user-space concepts, and does not discuss kernel architectures
• Linux is not fully-compliant with POSIX, but tracks closely
• the rest of this article assumes the Linux environment
• a program is said to be (in a space) if the program (has access to that memory space)
• the security model of memory spaces, is typically described as concentric rings, where inner-circles have higher security than outer spaces
• in this context, an analogy of (seeds and nuts) is used, introducing the terms (kernel) and (shell)
• a (kernel) is the central part of a computer's software operating system, which is highly-secured, and highly-privileged
• (kernel-space) refers to secured-memory which is accessible only to a kernel 
• typically, programs running in kernel-space are not accessible to a computer's deprivileged human users
• human users may acquire privileges, which escalate access to kernel-space programs; these are typically referred to as "administrator privileges" held by an "administrative role" 
• (user-space) refers to insecured-memory which is accessible to programs other than the kernel
• a (shell) is a program located in user-space, and functions as a mediator between programs in kernel-space and programs in user-space
• deprivileged human users typically have access only to programs in user-space - so deprivileged human users can talk to user-space programs, and user-space programs can talk to kernel-space programs ; sometimes, but not always, that talking is mediated by a shell
• shells are conveniences, which provide standardised interfaces for user-space programs to talk to kernel-space programs ; shells are therefore necessary for efficient, safe, architecture ... but not necessary if efficiency and safety are compromisable
• the rest of this article assumes the use of text-based shells, a.k.a. command-line interfaces (CLI)

1. Introductions to Linux CLI tools for reflection

• (type type) should be the ( introduction to the ) first tool for reflection, 
• (help help) should be the second ( built-ins use this, in (bash) but not (dash) )
• (info info) should be the third ( GNU introduced this )
• (man man) the fourth ( ancient UNIX )

2. Introductions to Linux CLI shells

• (\$\$) is an alias to the current shell's process' identification number (PID)
• (ps -p \$\$) uses that PID to find the ( command which created the current shell's process ), and prints it under the column (CMD).
• The most common shell environments are 
• ( bash ) the Bourne Again Shell, which has many convenience features
• (bash) defaults to a POSIX non-complaint mode, but can also be run in POSIX-compliant mode under a special flag
• run ( ls -la `which bash` ) or ( stat `which bash` ), to check that it is a "standard" file
• ( sh ) the traditional name for the shell, which on most systems is an alias for ( dash ) the Debian Almquist Shell
• (dash) is POSIX-compliant
• run ( ls -la `which sh` ), or ( stat `which sh` ), to check that (sh) is a "symbolic link" file which points to (dash)

3. Introduction to POSIX file systems

• Files : POSIX.1-2024 defines seven types of "file" in (stat.h).
• (1) a "standard" file is a convenient name for a blob of data, which might be returned by a filesystem via various implementations.
• There exist specific executable programs named (file), but this is not the general case being discussed here.
• Historically, this comes from a time when hand-held files were used, to contain punch-cards.
• (2) a "directory" file is a convenient name for part of a ( pathname ), see Indices > Secondary Index below
• (3) a "symbolic link" file is a pointer from one ( pathname ) to another ( pathname )
• (4-7) "block special", "character special", "FIFO special", and "socket" files refer to various data sources, such as devices and networking abstractions, which are given ( pathnames ) for convenience
• Indices
• Primary index : serial number
• Each file may be referenced by an ( inode / index node), which is an address of the blob of data, on the storage media. POSIX calls this the "file serial number".
• Secondary index : address in a tree
• Hierarchical/tree-structured file-systems were introduced by the Multics operating system in 1969.
• This introduces the concept of nested "directories" a.k.a. "folders", analogous to the concept of physical encapsulation.
• An address in a hierarchical file-system is called a ( pathname ), in POSIX.
• Links
• (hard links) are links between a ( pathname ) and an ( inode ).
• When a file is given a new pathname ( due to changes in its directory location, or changes to its own name ), the new pathname refers to the same old inode, so you can find the same data. There are no changes to (other hard links), and so the change of a pathname is transparent/irrelevant from the point of view of (other hard links). 
• (soft links) are links between a ( pathname ) and another ( pathname ).
• When a file is given a new pathname, any (soft links) pointing to the old pathname will no longer be able to locate the file's inode, and the inode's data. This is referred to as a broken link.

4. Introduction to Linux CLI shell utilities

• POSIX.1-2024 Standard Utilities
• IMPORTANT : ( section 12.2. Utility Argument Guidelines ) explains how to write utilities 
• IMPORTANT : ( section 12.1. Utility Argument Syntax ) explains how to read the documentation of utilities presented by ( info and man )
• Utility Argument Syntax
• Notation
• [entity] indicates that "entity" is optional
• Examples
• [-A] an "option" a.k.a. "flag"
• [-ABC] a "group of inclusive options"
• "options" which take "optional option-arguments" must be at the end of such a list
• [-A|-B] a "group of mutually exclusive options"
• Mandatory Option Arguments
• [-A option_argument] an "option" with a "mandatory option-argument" as TWO arguments
• Recommended guideline
• [-A option_argument]... an "option" with a "mandatory option-argument" which can be input multiple times
• [-Aoption_argument] an "option" with a "mandatory option-argument" as ONE argument
• Allowed for legacy compatibility, but not recommended.
• Optional Option Arguments
• [-A[option_argument]] an "option" with "optional option-argument" as ONE argument
• Recommended
• If the option_argument is missing from the optstring, the utility SHALL behave as if the entire option is missing from the optstring
• UNSPECIFIED : an "option" with "optional option-argument" as TWO arguments
• [operand] an "operand" which follows after all (if any) "options"
• [operand]... an "operand" which can be input multiple times 
• Standard Utilities SHALL be implemented
• Built-in Utilities
• Implies there is no need to execute a (separate executable file), as the utility is built into the (shell) itself.
• Special Built-in Utilities SHALL be Built-ins
• An error in a (special built-in utility) SHALL cause the (shell) to abort
• (variable assignments) preceding the invocation SHALL affect the (current execution environment)
• Accessibility to the (exec) family of POSIX.1-2024 functions is NOT mandatory
• List of Special Built-in Utilities
• ... which SHALL recognise the "--" argument
• dot 
• exec 
• export 
• readonly 
• set
• trap 
• unset 
• ... which MAY OR MAY NOT recognise the "--" argument
• eval
• ... which shall NOT recognise/use the "--" argument
• break
• colon
• continue
• exit
• return
• shift
• times
• Regular Built-in Utilities 
• An error in a (regular built-in utility) will NOT cause the (shell) to abort
• (variable assignments) preceding the invocation shall NOT affect the (current execution environment)
• Intrinsic Utilities
• NOT subject to a PATH search during ( command and search execution )
• CANNOT be overridden with a utility from PATH
• ? Accessibility to the (exec) family of POSIX.1-2024 functions is NOT mandatory 
• List of Intrinsic Built-in Utilities : OFTEN provided as (Regular built-in utilities)
• alias
• bg
• cd
• command
• fc
• fg
• getopts
• hash
• jobs
• kill
• ? Accessibility to the (exec) family of POSIX.1-2024 functions is MANDATORY
• read
• type
• ulimit
• umask
• unalias
• wait
• Non-Special Built-in, Non-Intrinsic, Standard Utilities
• ? SUBJECT to a PATH search during ( command and search execution )
• CAN be overridden with a utility from PATH
• Accessibility to the (exec) family of POSIX.1-2024 functions is MANDATORY 
• MAY OR MAY NOT be provided as (Regular built-in utilities)
• List of Non-Special Bult-in, Non-Intrinsic, Standard Utilities
• admin
• ar
• asa
• at
• awk
• basename
• batch
• bc
• c17
• cal
• cat
• cflow
• chgrp
• chmod
• chown
• cksum
• cmp
• comm
• compress
• cp
• crontab
• csplit
• ctags
• cut
• cxref
• date
• dd
• delta
• df
• diff
• dirname
• du
• echo
• ed
• env
• ex
• expand
• expr
• false
• file
• find
• fold
• fuser
• gencat
• get
• getconf
• gettext
• grep
• head
• iconv
• id
• ipcrm
• ipcs
• join
• lex
• link
• ln
• locale
• localedef
• logger
• logname
• lp
• ls
• m4
• mailx
• make
• man
• mesg
• mkdir
• mkfifo
• more
• msgfmt
• mv
• newgrp
• ngettext
• nice
• nl
• nm
• nohup
• od
• paste
• patch
• pathchk
• pax
• pr
• printf
• prs
• ps
• pwd
• readlink
• realpath
• renice
• rm
• rmdel
• rmdir
• sact
• sccs
• sed
• sh
• sleep
• sort
• split
• strings
• strip
• stty
• tabs
• tail
• talk
• tee
• test
• time
• timeout
• touch
• tput
• tr
• true
• tsort
• tty
• uname
• uncompress
• unexpand
• unget
• uniq
• unlink
• uucp
• uudecode
• uuencode
• uustat
• uux
• val
• vi
• wc
• what
• who
• write
• xargs
• xgettext
• yacc
• zcat
• (in bash, but not dash) 
• compgen / expansion
• HISTFILE / history / !

5. Introduction to POSIX definitions which influence POSIX SCL

• (POSIX regular expression character classes), with the notation [:classname:]