Last Updated: 2025-04-29 Tue 13:14

CMSC216 Project 4: Going Commando

CODE DISTRIBUTION: p4-code.zip

VIDEO OVERVIEW: https://youtu.be/So6ApzumTBI

CHANGELOG:

Tue Apr 29 01:07:03 PM EDT 2025

Several buggy tests have been corrected and students working on the project are advised to run make update to get the updated versions.

Post 1038 and Post 1049 identified a bug in Problem 3 Test 15 which expected wrong output due to some snaky shell quoting issues. The update resolves this so that the test expects the correct output.

Post 1032 identified a situation where a combination of incorrect behavior in the student code and a poorly written test may cause students to be booted from a login session (GRACE or a local graphical session). This was due to the kill() function being used in Problem 2 Test 8 without checking if student code had changed the PID away from the default -1 value. If it had not been changed, then ALL processes belonging to a student would be killed. The test has been adjusted to avoid this and print an error instead.

Mon Apr 28 04:29:36 PM EDT 2025

The Makeup Problem section has been completed in the project specification. It outline the optional source <file.txt> builtin command which may be completed for MAKEUP credit. Test cases are now in the code pack and available for use.

Post 1022 reported an intermittent bug which occasionally causes Problem 1 Test #3 to fail with a "directory not found" error. This was traced back to the same directory being used in another test. The files test_commando.c / test_commando1.org have been patched to fix the bug.

All students how have downloaded project files should run

  >> make update

to get required fixes and new files for testing the Makeup problem.

Sun Apr 27 09:36:13 AM EDT 2025
Made several clarifying changes to the function documentation to resolve omissions raised in Post 1012, Post 1010 and a few others.
Sat Apr 26 10:53:36 AM EDT 2025

Some modest changes have been made to the tests and grading to ease the understanding of test results. If you've started working on the project already, run

  >> make update     # update provided files

to get the changes.

The specific changes are:

  • Automated tests for Problem 1 goes from 15 to 20 tests; 1 large test was broken into several parts so the same functionality is tested just in distinct parts that makes it easier to interpret output
  • Credit for Problem 1 is upped from 30 to 35 points for the new tests
  • Credit for Problem 3 is reduced from 40 to 35 points with Style points going from 10 to 5 points.

1 Introduction: A Simple Shell

Command line shells allow one to access the capabilities of a computer using simple, interactive means. Type the name of a program and the shell will bring it into being, run it, and show output. The name "shell" is indicative of the program providing a thin convenience "layer" around more core facilities of a computing and operating system: run programs easily and see what they do. Familiarizing yourself with a basic shell implementation teaches about this interface layer and will make working in full-blown shells more palatable.

The goal of this project is to write a simple, quasi-command line shell called commando. The shell will be less functional in many ways from standard shells like bash (default on most Linux machines), zsh (default on MacOS) and tcsh (default on GRACE), but will have some properties that distinguish it such as maintaining a history of all programs run and their output. Like most interesting projects, commando uses a variety of system calls together to accomplish its overall purpose. Most of these will be individually discussed in lecture but the interactions between them is what inspires real danger and romance.

Completing commando will educate an implementer on the following systems programming topics.

  • Basic C Memory Discipline: A variety of strings and structs are allocated and de-allocated during execution which will require attention to detail and judicious use of memory tools like Valgrind.
  • fork() and exec(): Text entered that is not recognized as a built-in is treated as an command (external program) to be executed. This spawns a child process which executes the new program.
  • open(), dup2(), mkdir(), read(): Rather than immediately print child output to the screen, child output is redirected into files and then retrieved on request by commando.
  • wait() and waitpid(), blocking and nonblocking: Child processes usually take a while to finish so the shell will check on their status every so often

2 Download Code and Setup

As in labs, download the code pack linked at the top of the page. Unzip this which will create a folder and create your files in that folder.

File State Notes
Makefile Provided Build project, run tests
commando_funcs.c CREATE Functions to deal with the cmd_t and cmdctl_t structs (Problems 1/2)
commando_main.c CREATE main() function for commando interactive shell (Problem 3)
commando.h Provided Header file which contains required structs, defines, and prototypes
test_commando.org Testing Test definitions for all commando
test_commando.c Testing Low-level function tests for Problems 1/2
testy Testing Test running script
test_standardize_pids Testing Filter to standardize PID printing during testing
test_standardize_all Testing Filter to standardize other output during testing
data/ Data Directory containing below files used in testing

3 Opening Demo

The best way to get a sense of any program is to see how it behaves. In the below demonstration, commando is first built then started. Input is entered in commando after its prompt on lines that look like this:

[COMMANDO]>> commands here

Other lines contain output from the program. To the right of the demo after the # symbol are comments on what is happening.

>> make commando                                            ## compile commando
gcc -Wall -Werror -g -c commando_main.c
gcc -Wall -Werror -g -c commando_funcs.c
gcc -Wall -Werror -g -o commando commando_main.o commando_funcs.o

>> ./commando                                               ## run commando
=== COMMANDO ver 2 ===
Type 'help' for supported commands
[COMMANDO]>> help                                           ## print help on builtins
help                       : show this message
exit                       : exit the program
directory                  : show the directory used for command output
history                    : list all jobs that have been started giving information on each
pause <secs>               : pause for the given number of seconds which may be fractional
info <cmdid>               : show detailed information on a command
show-output <cmdid>        : print the output for given command number
finish <cmdid>             : wait until the given command finishes running
finish-all                 : wait for all running commands to finish
source <filename>          : (MAKEUP) open and read input from the given file as though it were typed
command <arg1> <arg2> ...  : non-built-in is run as a command and logged in history

[COMMANDO]>> directory                                      ## print the directory used for command output
Output directory is 'commando-dir'                          ## this is the default value

[COMMANDO]>> history                                        ## show the history of commands; none so far
CMDID PID       BYTES STATE           COMMAND

[COMMANDO]>> cat data/quote.txt                             ## run a command as a child process

[COMMANDO]>> history                                        ## show the history of entered commands, one currently running
CMDID PID       BYTES STATE           COMMAND
    0 #222616       - RUNNING         cat
---> COMPLETE:     0 #222616     125 EXIT(0)         cat    
1 commands finished                                         ## an alert that a previously started command ended

[COMMANDO]>> history                                        ## after the above command finishes, state has changed
CMDID PID       BYTES STATE           COMMAND
    0 #222616     125 EXIT(0)         cat

[COMMANDO]>> info 0                                         ## detailed information on the command
CMD 0 INFORMATION
command: cat data/quote.txt 
argv[]:
  [ 0]: cat
  [ 1]: data/quote.txt
cmdstate: EXIT(0)
input: <NONE>
output: commando-dir/0000-cat.out
output size: 125 bytes

[COMMANDO]>> show-output 0                                  ## output for the completed command
Object-oriented programming is an exceptionally bad idea which could
only have originated in California.

-- Edsger Dijkstra

[COMMANDO]>> ls -l data                                     ## start another command

[COMMANDO]>>                                                ## after a momement, press enter
---> COMPLETE:     1 #222643    1097 EXIT(0)         ls
1 commands finished                                         ## to see the command has finished

[COMMANDO]>> history                                        ## show history
CMDID PID       BYTES STATE           COMMAND
    0 #222616     125 EXIT(0)         cat
    1 #222643    1097 EXIT(0)         ls

[COMMANDO]>> show-output 1                                  ## print output for last command
total 96
-rw-r--r-- 1 kauffman users 13893 Feb  5  2019 3K.txt
-rwxr--r-- 1 kauffman users    43 Apr 23 23:41 delay.sh
-rw-r--r-- 1 kauffman users  1511 Feb  5  2019 gettysburg.txt
-rwxr--r-- 1 kauffman users   326 Apr 24 15:17 make_unwriteable.sh
-rwxr-xr-x 1 kauffman users 15424 Apr 24 15:12 print_args
-rw-r--r-- 1 kauffman users   218 Feb  5  2019 print_args.c
-rw-r--r-- 1 kauffman users   125 Feb  5  2019 quote.txt
-rw-r--r-- 1 kauffman users   184 Feb  6  2020 README
-rw-r--r-- 1 kauffman users    72 Apr 21 14:02 script.txt
-rwxr--r-- 1 kauffman users   185 Apr 23 23:35 self_signal.sh
-rwxr--r-- 1 kauffman users    36 Apr 18 16:17 sleep_print.sh
drwxr-xr-x 2 kauffman users  4096 Apr 25 10:22 subdir
-rwxr--r-- 1 kauffman users   427 Feb  5  2019 table.sh

[COMMANDO]>> data/sleep_print 25.0 Long running             ## start command but mistype its name

[COMMANDO]>> history                                        ## show current history
CMDID PID       BYTES STATE           COMMAND
    0 #222616     125 EXIT(0)         cat
    1 #222643    1097 EXIT(0)         ls
    2 #222659       - RUNNING         data/sleep_print
---> COMPLETE:     2 #222659      95 FAIL_EXEC       data/sleep_print
1 commands finished                                         ## note the FAILE_EXEC - didn't run right

[COMMANDO]>> data/sleep_print.sh 25.0 Long running          ## try again with the correct name

[COMMANDO]>> history                                        ## command running now
CMDID PID       BYTES STATE           COMMAND
    0 #222616     125 EXIT(0)         cat
    1 #222643    1097 EXIT(0)         ls
    2 #222659      95 FAIL_EXEC       data/sleep_print
    3 #222666       - RUNNING         data/sleep_print.sh

[COMMANDO]>> info 3                                         ## show detailed information
CMD 3 INFORMATION
command: data/sleep_print.sh 25.0 Long running              ## full command line
argv[]:                                                     ## command line args
  [ 0]: data/sleep_print.sh
  [ 1]: 25.0
  [ 2]: Long
  [ 3]: running
cmdstate: RUNNING                                           ## current state: still running
input: <NONE>
output: commando-dir/0003-data_sleep_print.sh.out
output size: <CMD NOT FINISHED>                             ## no output as it is still running

[COMMANDO]>> wc -l data/gettysburg.txt                      ## launch another command

[COMMANDO]>> 
---> COMPLETE:     4 #222677      23 EXIT(0)         wc     ## it finishes quickly
1 commands finished

[COMMANDO]>> history                                        ## check history
CMDID PID       BYTES STATE           COMMAND
    0 #222616     125 EXIT(0)         cat
    1 #222643    1097 EXIT(0)         ls
    2 #222659      95 FAIL_EXEC       data/sleep_print
    3 #222666       - RUNNING         data/sleep_print.sh   ## still running
    4 #222677      23 EXIT(0)         wc

[COMMANDO]>> data/table.sh 10 3                             ## run another command
---> COMPLETE:     3 #222666      13 EXIT(0)         data/sleep_print.sh
1 commands finished                                         ## past command finishes

[COMMANDO]>>                                                ## wait a tick and press enter
---> COMPLETE:     5 #222688     380 EXIT(0)         data/table.sh
1 commands finished                                         ## most recent command finishes

[COMMANDO]>> info 4                                         ## information on 4th command
CMD 4 INFORMATION
command: wc -l data/gettysburg.txt 
argv[]:
  [ 0]: wc
  [ 1]: -l
  [ 2]: data/gettysburg.txt
cmdstate: EXIT(0)
input: <NONE>
output: commando-dir/0004-wc.out
output size: 23 bytes

[COMMANDO]>> show-output 4                                  ## output for 4th command
28 data/gettysburg.txt

[COMMANDO]>> info 5                                         ## infor for 5th command
CMD 5 INFORMATION
command: data/table.sh 10 3
argv[]:
  [ 0]: data/table.sh
  [ 1]: 10
  [ 2]: 3
cmdstate: EXIT(0)
input: <NONE>
output: commando-dir/0005-data_table.sh.out
output size: 380 bytes

[COMMANDO]>> show-output 5                                  ## output for command
i^1=      1  i^2=      1  i^3=      1
i^1=      2  i^2=      4  i^3=      8
i^1=      3  i^2=      9  i^3=     27
i^1=      4  i^2=     16  i^3=     64
i^1=      5  i^2=     25  i^3=    125
i^1=      6  i^2=     36  i^3=    216
i^1=      7  i^2=     49  i^3=    343
i^1=      8  i^2=     64  i^3=    512
i^1=      9  i^2=     81  i^3=    729
i^1=     10  i^2=    100  i^3=   1000

[COMMANDO]>> data/sleep_print.sh 5.0 A                      ## launch several commands that
                                                            ## will take 5 seconds to run
[COMMANDO]>> data/sleep_print.sh 5.0 B

[COMMANDO]>> data/sleep_print.sh 5.0 C

[COMMANDO]>> pause 5.5                                      ## pause commando for 5.5 seconds
Pausing for 5.500000 seconds
---> COMPLETE:     6 #222705       2 EXIT(0)         data/sleep_print.sh
---> COMPLETE:     7 #222710       2 EXIT(0)         data/sleep_print.sh
---> COMPLETE:     8 #222716       2 EXIT(0)         data/sleep_print.sh
3 commands finished                                         ## an coming back from pause, all commands finished

[COMMANDO]>> data/sleep_print.sh 5.0 D                      ## 9th command runs a long time

[COMMANDO]>> data/sleep_print.sh 1.0 D                      ## short run

[COMMANDO]>> data/sleep_print.sh 1.0 D                      ## short run

[COMMANDO]>> finish 9                                       ## block until 9th command finishes
---> COMPLETE:     9 #222765       2 EXIT(0)         data/sleep_print.sh   ## 9th command done
---> COMPLETE:    10 #222770       2 EXIT(0)         data/sleep_print.sh   ## top-of-loop update shows
---> COMPLETE:    11 #222777       2 EXIT(0)         data/sleep_print.sh   ## short run command also done
2 commands finished                                                        ## only 2 as 1st is an explicit finish

[COMMANDO]>> data/sleep_print.sh 5.0 X                      ## start several long-running commands

[COMMANDO]>> data/sleep_print.sh 5.0 X

[COMMANDO]>> data/sleep_print.sh 5.0 X

[COMMANDO]>> finish-all                                     ## wait for all to fininsh
---> COMPLETE:    12 #222793       2 EXIT(0)         data/sleep_print.sh
---> COMPLETE:    13 #222798       2 EXIT(0)         data/sleep_print.sh
---> COMPLETE:    14 #222801       2 EXIT(0)         data/sleep_print.sh
3 commands finished

[COMMANDO]>> exit                                           ## all done for now

>>                                                          ## back to a "real" shell

Things to Note

  • The child processes (commands) that commando starts do not show any output by default and run concurrently with the main process which gives back the [COMMANDO]>> prompt immediately. This is different from a normal shell such as bash which starts commands in the foreground, shows their output immediately, and will block the shell until a job finishes before showing the command prompt for additional input.
  • The output for all jobs is saved by commando and can be recalled at any time using the show-output <jobid> built-in.
  • Not all of the built-ins are shown in the demo but each will be discussed in later sections.
  • It should be clear that commando is not a full shell (no signals, built-in scripting, or pipes), but it is a mid-sized project which will take some organization. Luckily, this document prescribes a simple architecture to make the coding manageable.

4 Overall Architecture

4.1 cmd_t struct

The cmd_t type is defined in the commando.h header file. It is intended to encapsulate the state of a command, a running or completed child process. Fields within it describe aspects such as the name of the command being run, arguments to it, its exit status, and a file with the command output in it.

// states in which a cmd might be
typedef enum  {
  CMDSTATE_NOTSET = 0,                // hasn't been set
  CMDSTATE_INIT,                      // cmd struct just initialized but no child process
  CMDSTATE_RUNNING,                   // child process fork()/exec()'d and running
  CMDSTATE_EXIT,                      // child process exited normally
  CMDSTATE_SIGNALLED,                 // child process exited abnormally due to a signal
  CMDSTATE_UNKNOWN_END,               // cmd finished but for unknown reasons
  CMDSTATE_FAIL_INPUT=97,             // couldn't find input file for redirection
  CMDSTATE_FAIL_OUTPUT=98,            // couldn't find input file for redirection
  CMDSTATE_FAIL_EXEC=99,              // exec() failed after fork()
} cmdstate_t;

// struct that represents a command and its output 
typedef struct cmd {
  char cmdline[MAX_LINE];             // original command line entered by users, copied from input
  char *argv[MAX_ARGS];               // argv[] for running child, NULL terminated, malloc()'d and must be free()'d
  int argc;                           // length of the argv[] array, used for sanity checks
  cmdstate_t cmdstate;                // indicates phase of cmd in life cycle (running, exited, etc.)
  pid_t pid;                          // PID of running child proces for the cmd; -1 otherwise
  int exit_code;                      // value returned on exit from child process or negative signal number
  char input_filename[MAX_NAME];      // name of file containing input
  char output_filename[MAX_NAME];     // name of file containing output
  ssize_t output_bytes;               // number of bytes of output
} cmd_t;

4.2 cmdctl_t struct

The demo illustrates that commando tracks multiple commands and their child processes. This multiplicity is simplified somewhat with a data structure to add and iterate through all child jobs. This is the role of cmdctl_t: it's most important field is the array cmd_t *cmds which maintains all commands that are running or are complete. As the number of commands run increases, this array may "expand" which will require reallocation and copying of its contents to new memory. The struct also maintains some global information such as the directory that is used to store output for commands.

Most functions in commando_funcs.c pertain to operating on a cmdctl_t struct often taking an index of a individual command to operate on. Function including adding a command, starting a command running, updating a command to see if it has finished, and printing information and output for a command.

// control structure to track cmd history, output location, etc.
typedef struct {                
  char cmddir[MAX_NAME];              // name of the directory where cmd output files will be placed
  int cmds_count;                     // number of elements in the cmds array
  int cmds_capacity;                  // capacity of the cmds array
  cmd_t *cmds;                        // array of all cmds that have run / are running
} cmdctl_t;

4.3 main() function ::

The file commando_main.c will contain a main() function which loops over input provided by the user either interactively or via standard input as is done in the automated tests. After setup, the program executes an infinite loop until no more input is available or an exit/quit built-in is used. The main() understand certain built-ins that allow checking on running commands or reporting output. Anything that doesn't match a built-in is fork()'d/exec()'d as a child process and encapsulated as a "command".

Several command line options are supported by main() to enable echoing and change the default output directory. These are described later.

4.4 Outline of Code

Below is an outline of the required functions. Additional functions may be added but were not needed in the instructor solution.

Several utility functions are provided at the beginning of the code outline which students can and should be used in the implementation.

// UPDATED: Mon Apr 28 04:33:13 PM EDT 2025

#include "commando.h"

////////////////////////////////////////////////////////////////////////////////
// PROVIDED DATA / FUNCTIONS
//
// The data and functions below should not be altered as they should
// be used as-is in the project.
///////////////////////////////////////////////////////////////////////////////

// Prints out a message if the environment variable DEBUG is set;
// Running as `DEBUG=1 ./some_program` or `DEBUG=1 make test-prob1`
// will cause all Dprintf() messages to show while running without the
// DEBUG variable will not print those messages. An example:
// 
// >> DEBUG=1 ./commando              // enable debug messages
// === COMMANDO ver 2 ===
// Re-using existing output directory 'commando-dir'
// Type 'help' for supported commands
// [COMMANDO]>> ls
// |DEBUG| sscanf ret is 1 for line `ls`
// |DEBUG| pid 237205 still running
// [COMMANDO]>> exit
// |DEBUG| sscanf ret is 1 for line `exit`
// |DEBUG| end main loop
// |DEBUG| freeing memory
// 
// >> ./commando                      // no debug messages
// === COMMANDO ver 2 ===
// Re-using existing output directory 'commando-dir'
// Type 'help' for supported commands
// [COMMANDO]>> ls
// [COMMANDO]>> exit
void Dprintf(const char* format, ...) {
  if(getenv("DEBUG") != NULL){
    va_list args;
    va_start (args, format);
    char fmt_buf[2048];
    snprintf(fmt_buf, 2048, "|DEBUG| %s", format);
    vfprintf(stderr, fmt_buf, args);
    va_end(args);
  }
}

// Sleep the running program for the given number of seconds allowing
// fractional values.
void pause_for(double secs){
  int isecs = (int) secs;
  double frac = secs - ((double) isecs);
  long inanos = (long) (frac * 1.0e9);

  struct timespec tm = {
    .tv_nsec = inanos,
    .tv_sec  = isecs,
  };
  nanosleep(&tm,NULL);
}

// Splits `line` into tokens with pointers to each token stored in
// argv[] and argc_ptr set to the number of tokens found. This
// function is in the style of strtok() and destructively modifies
// `line`. A limited amount of "quoting" is supported to allow single-
// or double-quoted strings to be present. The function is useful for
// splitting lines into an argv[] / argc pair in preparation for an
// exec() call.  0 is returned on success while an error message is
// printed and 1 is returned if splitting fails due to problems with
// the string.
//
// EXAMPLES:
// char line[128] = "Hello world 'I feel good' today";
// char *set_argv[32];
// int set_argc;
// int ret = split_into_argv(line, set_argv, &set_argc);
// // set_argc: 4
// // set_argv[0]: Hello
// // set_argv[1]: world
// // set_argv[2]: I feel good
// // set_argv[3]: today
// // set_argv[4]: <NULL>
int split_into_argv(char *line, char *argv[], int *argc_ptr){
  int argc = 0;
  int in_token = 0;
  for(int i=0; line[i]!='\0'; i++){
    if(!in_token && isspace(line[i])){                 // skip spaces between tokens
      continue;
    }
    else if(in_token && (line[i]=='\'' || line[i]=='\"')){
      printf("ERROR: No support for mid-token quote at index %d\n",i);
      return 1;
    }
    else if(line[i]=='\'' || line[i]=='\"'){           // begin quoted token
      char start_quote_char = line[i];
      i++;                                             // skip first quote char
      argv[argc++] = line+i;                           // start of token
      for(;line[i] != start_quote_char;i++){
        if(line[i] == '\0'){
          printf("ERROR: unterminated quote in <%s>\n",line);
          return 1;
        }
      }
      line[i] = '\0';                                  // end quoted token
    }
    else if(in_token && !isspace(line[i])){            // still in a token, move ahead
      continue;
    }
    else if(!in_token && !isspace(line[i])){           // begin unquoted token 
      in_token = 1;
      argv[argc++] = line+i;
    }
    else if(in_token && isspace(line[i])){             // end unquoted token
      in_token = 0;
      line[i] = '\0';
    }
    else{
      printf("ERROR: internal parsing problem at string index %d\n",i);
      return 1;
    }
  }
  *argc_ptr = argc;
  argv[argc] = NULL;                                   // ensure NULL termination required by exec()
  return 0;
}

////////////////////////////////////////////////////////////////////////////////
// PROBLEM 1 Functions
////////////////////////////////////////////////////////////////////////////////

cmdctl_t *cmdctl_new(char *cmddir, int cmds_capacity);
// PROBLEM 1: Allocate a new cmdctl structure with the output
// directory `cmddir` and intitial `cmds_capacity` for its cmds[]
// array field.

void cmdctl_free(cmdctl_t *cmdctl);
// PROBLEM 1: De-allocate the given given cmdctl struct. Free all
// argv[] elements in each of the cmds[], free that array, then free
// the cmdctl struct itself.

int cmdctl_create_cmddir(cmdctl_t *cmdctl);
// PROBLEM 1: Creates the output directory names in cmdctl->cmddir. If
// cmddir does not exist, it is created as directory with permissions
// of User=read/write/execute then returns 1. If cmddir already exists
// and is a directory, prints the following message and returns 0:
//
//   Re-using existing output directory 'XXX'
//
// If a non-directory file named cmddir already exists, print an error
// message and return -1 to indicate the program cannot proceed. The
// error message is:
//
// ERROR: Could not create cmddir directory 'XXX'
//        Non-directory file with that name already exists
//
// with XXX substituted with the value of cmddir. 
//
// CONSTRAINT: This function must be implemented using low-level
// system calls. Use of high-level calls like system("cmd") will be
// reduced to 0 credit. Review system calls like stat() and mkdir()
// for use here. The access() system call may be used but keep in mind
// it does not distinguish between regular files and directories.

int cmdctl_add_cmd(cmdctl_t *cmdctl, char *cmdline);
// PROBLEM 1: Add a command to the cmdctl. If the capacity of the
// cmds[] array is insufficient, is size is doubled before adding
// using the realloc() function. 
//
//
// SETTING ARGV[] OF THE CMD 
//
// `cmdline` is the string typed in as a command. `cmdline` is
// separated into an argv[] array using the split_into_argv()
// function. All strings are duplicated into the argv[] array of the
// cmd_t and are later free()'d on memory de-allocation.  
//
// Example:
// cmdline: "data/sleep_print.sh 1 hello world"
// argv[]:
//   [ 0]: data/sleep_print.sh
//   [ 1]: 1
//   [ 2]: hello
//   [ 3]: world
//   [ 4]: NULL
//
// NOTE: Paramameter `cmdline` is copied into the cmd.cmdline[]
// field. This copy is left unchanged. When splitting, another local
// copy of the string is made. strcpy() is useful for this and
// strdup() is used for creating heap-allocated copies for cmd.argv[]
// to point at.
//
// HANDLING INPUT REDIRECTION
//
// If the last two elements of the command line indicate input
// redirection via "< infile.txt" then the argv[] and input_filename
// fields are set accordingly.  An example:
//
// cmdline: "wc -l -c < Makefile"
// argv[]:
//   [ 0]: wc
//   [ 1]: -l
//   [ 2]: -c
//   [ 3]: NULL      NOTE "<" and "Makefile" removed
// input_filename: Makefile
//
// SETTING OUTPUT FILENAME
//
// The output_filename field is set to a pattern like the following:
//
//     outdir/0017-data_sleep_print.sh.out
//     |      |     |                   +-> suffix .out
//     |      |     +-> argv[0] with any / characters replaced by _
//     |      +-> 4 digit cmdid (index into cmds[] array), 0-padded
//     +->cmddir[] directory name followed by "/"
//
// A copy of the argv[0] string is modified to replace slashes with
// underscores and the sprintf() family of functions is used for
// formatting.
//
// SETTING OTHER FIELDS
//
// Other fields are set as follows:
// - argc to the length of the argv[] array
// - cmdstate to the CMDSTATE_INIT 
// - pid, exit_code, output bytes to -1
//
// The function returns 1 if the added command caused the internal
// array of commands to expand and 0 otherwise. 
//
// CONSTRAINT: Do not use snprintf(), just plain sprintf(). Aim for
// simplicity and add safety at a later time.

void cmdctl_print_oneline(cmdctl_t *cmdctl, int cmdid);
// PROBLEM 1: Prints a oneline summary of the indicated command's
// current state. The format is:
//
// EXAMPLES:
//  A:5 B:1 C:-6    D:7     E:-15         F:remaining
// ------------------------------------------------------------
//    24 #007575       - RUNNING         data/sleep_print.sh
//   123 #007570      54 EXIT(0)         seq         
//     2 #007566      73 EXIT(1)         gcc         
//  2961 #007562      23 SIGNALLED(-15)  data/self_signal.sh
// ------------------------------------------------------------
// 123456789012345678901234567890123456789012345678901234567890
//
// - A: cmdid (index into the cmds[] array), 5 places, right aligned
// - B: # symbol
// - C: PID of command, 6 places 0 padded, left aligned
// - D: number of output bytes if not running, otherwise a "-", 7 places
//      right aligned
// - E: printed state with exit code/signal, 15 places, left aligned
// - F: argv[0] (command name), left aligned, printed for remainder of
//   line
//
// For E, the possible outputs are based on cmdstate_t values:
// - NOTSET
// - INIT
// - RUNNING
// - EXIT(<num>)      : positive number of normal exit
// - SIGNALLED(<num>) : negative number of signal
// - UNKNOWN_END
// - FAIL_INPUT
// - FAIL_OUTPUT
// - FAIL_EXEC
//
// NOTE: Most implementations will use a combination of printf() and
// possibly sprintf() to get the required widths. The trickiest part
// is getting the states EXIT() / SIGNALLED() aligned correctly for
// which an individual call to sprintf() to first format the numbers
// followed by a printf() with a width specifier is helpful.
//
// CONSTRAINT: Do not use snprintf(), just plain sprintf(). Aim for
// simplicity and add safety at a later time.

void cmdctl_print_all(cmdctl_t *cmdctl);
// PROBLEM 1: Prints a header and then iterates through all commands
// printing one-line summaries of them information on them. Prints a
// table header then repeatedly calls cmdctl_print_oneline(). Example:
//
// CMDID PID       BYTES STATE           COMMAND
//     0 #107561      22 SIGNALLED(-9)   data/self_signal.sh
//     1 #107562      23 SIGNALLED(-15)  data/self_signal.sh
//     2 #107566      73 EXIT(1)         gcc         
//     3 #107570      54 EXIT(0)         seq         
//     4 #107575       - RUNNING         data/sleep_print.sh

void cmdctl_print_info(cmdctl_t *cmdctl, int cmdid);
// PROBLEM 1: Prints detailed information about the given
// command. Example output is as follows.
//
// EXAMPLE 1: Exited command
// Call: cmdctl_print_info(cmdctl, 2);
// -------OUTPUT-------------
// CMD 2 INFORMATION
// command: seq 25
// argv[]:
//   [ 0]: seq
//   [ 1]: 25
// cmdstate: EXIT(0)
// input: <NONE>
// output: commando-dir/0002-seq.out
// output size: 66 bytes
// -------------------------
// 
// EXAMPLE 2: Running command
// Call: cmdctl_print_info(cmdctl, 4);
// -------OUTPUT-------------
// CMD 4 INFORMATION
// command: data/sleep_print.sh 25 hello world
// argv[]:
//   [ 0]: data/sleep_print.sh
//   [ 1]: 25
//   [ 2]: hello
//   [ 3]: world
// cmdstate: RUNNING
// input: <NONE>
// output: commando-dir/0004-data_sleep_print.sh.out
// output size: <CMD NOT FINISHED>
// -------------------------
//
// May have be some duplicate code in this function to that present in
// cmd_print_oneline() for handling the different cmdstate values.

void cmdctl_show_output(cmdctl_t *cmdctl, int cmdid);
// PROBLEM 1: If the given command is not finished (e.g. still
// RUNNING), prints the message
//
//   NO OUTPUT AVAILABLE: command has not finished running
//
// If the command is finished, opens the output file for the command
// and prints it to the screen using low-level UNIX I/O.
//
// CONSTRAINT: This routine must use low-level open() and read() calls
// and must allocate no more memory for the file contents than
// MAX_LINE bytes in a buffer. Use of fscanf(), fgetc(), and other C
// standard library functions is not permitted and scanning the
// entirety of the file into memory is also not permitted.

////////////////////////////////////////////////////////////////////////////////
// PROBLEM 2 Functions
////////////////////////////////////////////////////////////////////////////////

int cmdctl_start_cmd(cmdctl_t *cmdctl, int cmdid);
// PROBLEM 2: Start a child process that will run the indicated
// cmd. After creating the child process, the parent sets sets the
// `pid` field, changes the state to RUNNING and returns 0.
//
// The child sets up output redirection so that the standard out AND
// standard error streams for the child process is channeled into the
// file named in field `output_filename`. Note that standard out and
// standard error are "merged" so that they both go to the same
// `outfile_name`. This file should use the standard file
// creation/truncation options (O_WRONLY|O_CREAT|O_TRUNC) and be
// readable/writable by the user (S_IRUSR|S_IWUSR).
//
// If input_filename is not empty, input redirection is also set up
// with input coming from the file named in that field. Output
// redirection is set up first to allow any errors associated with
// input redirection to be captured in the output file for the
// command. 
//
// IMPORTANT: Any errors in the child during I/O redirection or
// exec()'ing print error messages and cause an immediate exit() with
// an associated error code. These are as follows:
//
// | CONDITION            | EXIT WITH CODE              | Message                                                                         |
// |----------------------+-----------------------------+---------------------------------------------------------------------------------|
// | Output redirect fail | exit(CMDSTATE_FAIL_OUTPUT); | No message printed                                                              |
// | Input redirect fail  | exit(CMDSTATE_FAIL_INPUT);  | ERROR: can't open file 'no-such-file.txt' for input : No such file or directory |
// | Exec failure         | exit(CMDSTATE_FAIL_EXEC);   | ERROR: program 'no-such-cmd' failed to exec : No such file or directory         |
//
// The "No such file..." message is obtained and printed via
// strerror(errno) which creates a string based on what caused a
// system call to fail.
//
// NOTE: When correctly implemented, this function should never return
// in the child process though the compiler may require a `return ??`
// at the end to match the int return type. NOT returning from this
// function in the child is important: if a child manages to return,
// there will now be two instances of main() running with the child
// starting its own series of grandchildren which will not end well...

int cmdctl_update_one(cmdctl_t *cmdctl, int cmdid, int finish);
// PROBLEM 2: Updates a single cmd to check for its completion. If the
// cmd is not RUNNING, returns 0 immediately.
//
// If the parameter `finish` is 0, uses wait()-family system calls to
// determine if the child process associated with the command is
// complete. If not returns 0 immediately - a NON-BLOCKING WAIT() is
// required to accomplish this.
//
// If the parameter `finish` is 1, uses a BLOCKING-style call to wait
// until the child process is actually finished before moving on.
//
// If the child process is finished, then diagnosis its exit status to
// determine a normal exit vs abnormal termination due to a
// signal. the W-MACROS() are used to determine this. The
// cmd->exit_code field is set to positive for a regular exit code or th
// negative signal number for an abnormal exit.
//
// The output_bytes is set by using a stat()-family call to determine
// the number of bytes in the associated output file.
//
// When the command finishes, an "ALERT" message is printed like the
// following which primarily uses the cmd_print_oneline() function.
//
// ---> COMPLETE:     2 #108624    2617 EXIT(0)         ls
// ---> COMPLETE:   129 #108725      22 SIGNALLED(-2)   data/self_signal.sh
// <string shown> <----------output from cmd_print_oneline()---------------->
//
// Returns 1 when the child process changes from RUNNING to a
// completed state.

int cmdctl_update_all(cmdctl_t *cmdctl, int finish);
// PROBLEM 2: Iterates through all commands and calls update on
// them. Passes the the `finish` parameter on to each call. Counts
// then number of commands that change state (from RUNNING to a
// finished state) and if 1 or more finish, prints a message like
//
//   4 commands finished
//
// after completing the loop. Returns the number of commands the that
// finished.

////////////////////////////////////////////////////////////////////////////////
// PROBLEM 3: Implement main() in commando_main.c
////////////////////////////////////////////////////////////////////////////////

int main(int argc, char *argv[]);
// Problem 3: main in command_main.c.
//
// * COMMAND LINE FORMS 
//
// The program must support four command line forms.
// | ./commando                        | argc=1 | No echo, default cmddir          |
// | ./commando --echo                 | argc=2 | Echoing on                       |
// | ./commando --dir <dirname>        | argc=3 | Set cmddir to argument           |
// | ./commando --echo --dir <dirname> | argc=4 | Echoing on and non-defaut cmddir |
//
// When the --echo option is provided, each line of input that is read
// in is first printed to the screen. This is makes testing easier.
//
// When the --dir <dirname> option is present, it sets the output
// directory used to store command output. If it is not present, the
// default cmddir name DEFAULT_CMDDIR is used; commando.h sets this to
// "command-dir".
//
// If a command line form above is not matched (arg 1 is not --echo,
// no <dirname> provided, 5 or more args given, etc.), then the
// following usage message should be printed:
//
//   usage: ./commando
//          ./commando --echo
//          ./commando --dir <dirname>
//          ./commando --echo --dir <dirname>
//   Incorrect command line invocation, use one of the above
//
//
// * INTERACTIVE LOOP
//
// After setting up a cmdctl struct, main() enters an interactive loop
// that allows users to use built-in and external commands.  Each
// iteration of the interactive loop prints a prompt then reads a line
// of text from users using the fgets() function. If echoing is
// enabled, the line read is immediately printed.
//
// A full line is read as it may contain many tokens that will become
// part of an external command to run.  The first "token" (word) on
// the line is extracted using a call to sscanf() and checked against
// the built-in commands below. If the word matches a known built-in,
// then the program honors it and iterates.
//
// If no built-in functionality matches, then the line is treated as
// an external command and run via a fork() / exec() regime from
// functions like cmdctl_add_cmd() and cmdctl_start_cmd().
//
// At the top of each interactive loop, all commands are updated via a
// call to cmdctl_update_all(...,0) which which will check to see if
// any commands have finished and print a message indicating as much
// before the prompt.
//
// Cautions on using fgets(): While useful, fgets() is a little tricky
// and implementer should look for a brief explanation of how it
// works. Keep in mind the following
// - It requires a buffer to read into; use one of size MAX_LINE
// - It will return NULL if the end of input has been reached
// - Any newlines in the input will appear in the buffer; use strlen()
//   to manually "chop" lines to replace the trailing \n with \0 when
//   processing the command
// - A handy invocation is something like
//     char firstok[...];
//     sscanf(inline, "%s", firstok);
//   which allows the first token (word) to be extracted from the
//   input line.  Other tokens can be similarly extracted using
//   sscanf(). 
//
// * STARTUP MESSAGES AND ERRORS
//
// When beginning, commando will print out the following:
//
//   === COMMANDO ver 2 ===
//   Re-using existing output directory 'commando-dir'
//   Type 'help' for supported commands
//   COMMANDO>>
//
// The last line is the prompt for input form the user.
//
// The first line indicates that there is already a directory present
// with output so old command output may get overwritten.  It is
// printed during cmdctl_create_cmddir().
//
// * BUILT-IN FUNCTIONALITY
//
// Typing 'help' will show a message about the built-in functionality.
//
// COMMANDO>> help
// help                       : show this message
// exit                       : exit the program
// directory                  : show the directory used for command output
// history                    : list all jobs that have been started giving information on each
// pause <secs>               : pause for the given number of seconds which may be fractional
// info <cmdid>               : show detailed information on a command
// show-output <cmdid>        : print the output for given command number
// finish <cmdid>             : wait until the given command finishes running
// finish-all                 : wait for all running commands to finish
// source <filename>          : (MAKEUP) open and read input from the given file as though it were typed
// command <arg1> <arg2> ...  : non-built-in is run as a command and logged in history
//
// These are all demonstrated in the project specification and
// generally encompass either an associated service function. The
// `source` built-in is optional for MAKEUP credit.

5 Problem 1: Basic Functionality

Functions in this problem are all in commando_funcs.c and are designed to acquaint implementers with the basic data types in the project along with providing some printing functionality.

5.1 Implementation Notes

Most Implementation Notes are provided in the Outline of Code. Here are a few additional items to consider

Use of realloc()

Aside from malloc() / calloc(), the C standard library provides one more useful memory allocation function called realloc(). A call to it looks like the following.

{
  int count = 10;
  data_t *ptr = malloc(sizeof(data_t)*count); // malloc()'d at some size
  ...;                                        // after a short time...
  count = count * 2;                          // need more space in the array
  data_t *old = ptr;                          // optionall save location
  ptr = realloc(ptr, sizeof(data_t)*count);   // allocate more space
  // ptr now refers to a larger area

  if(ptr == old){
    printf("Same location\n");  // expansion in place by extending a heap block
  }
  else{
    printf("New location!\n");  // data copied to new larger location and old
  }                             // now refers to a free()'d region
}  

realloc() gets a larger heap area for heap-allocated data as efficiently as possible.

  • It may be able to expand the data in place to the new size
  • It may have to move the data to a new location in which case the old area will be de-allocated

The reasons for the two cases will be easier to understand once heap implementation is discussed.

Use of sprintf()

There are situations in which one wants to craft a string using printf()-like formatting. For this sprintf() is useful: rather than printing to the screen, it arranges characters in a string (character array). A simple usage is:

{
  int meaning = 42;
  char *speaker = "Deep Thought";
  char strbuf[128];
  sprintf(strbuf,"%s: The meaning of life is %d",speaker,meaning);
  // strbuf: "Deep Thought: The meaning of life is 42"
}

sprintf() is a bit "dodgy" as it has the potential to overflow buffers but in our beginner setting we are not overly concerned with this. In the wild, opt for snprintf() which can prevent buffer overflow but for this project *use sprintf() and NOT snprintf().

5.2 Grading Criteria for Problem 1   grading 35

Weight Criteria
  AUTOMATED TESTS via make test-prob1
20 Tests provide via test_commando1.org and test_commando.c
  Runs all code under Valgrind to ensure that no memory errors are present.
  1 point per test passed
15 MANUAL INSPECTION
   
5 Code Style: Functions adhere to CMSC 216 C Coding Style Guide which dictates
  reasonable indentation, commenting, consistency of curly usage, etc.
   
1 cmdctl_new() / cmdctl_free()
  Effort to initialize fields to default values when allocating
  Clear traversal of cmds[] array and all argv[] elements to free them
   
2 cmdctl_create_cmddir()
  Use of the system call mkdir() to create the directory
  Checks for presence of an existing directory which is re-used
  Checks for existing non-directory file which is reported
   
3 cmdctl_add_cmd()
  Code to detect the need to expand the cmds[] array and use of realloc() to do so
  Use of the split_into_argv() function to split up the provided command line
  Processing of the argv[0] element to replace all / characters with _ characters
  Detection and processing of input redirection via ... < file.txt
  May use sprintf() but NOT snprintf()
   
2 cmdctl_print_oneline() cmdctl_print_all() cmdctl_print_info()
  Handles cases for each of the cmdsate values like RUNNING / EXIT / etc.
  May use sprintf() but NOT snprintf()
   
2 cmdctl_show_output()
  Uses low-level open() / read() calls for reading files
  Utilizes a buffer no larger than MAX_LINE bytes for input
  Demonstrates a clear input loop to read parts of the file, process it, then repeat

6 Problem 2: Starting and Finishing Commands

This problem implements functions to start, update, and finish commands. There are only two but they are interesting in that they utilize system calls we have studied.

6.1 Implementation Notes

Printing System Call Errors

When system calls like open() / fork() / etc. fail, they set a global variable which indicates the cause of an error. This can be reported in several ways shown in the following examples.

{
  char *filename = "myfile.txt";
  int ret = open(filename, O_RDONLY);
  if(ret == -1){
    printf("open() failed");
    // (A) fixed message, little information

    perror("open failed because"); 
    // (B) perror() appends a cause for the failure automatically

    fprintf(stderr, "open() on the file %s failed : %s\n",
            filename, strerror(errno));
    // (C) sterror(errno) returns a string which can be inserted into
    // a format string which conveys more information about the context
}

Each of these is appropriate to some contexts but (C) is the most flexible as it allows a string error message to be inserted at an arbitrary location in other output.

Starting Commands

Starting child processes via fork() has been discussed and should be used in cmdctl_start_cmd() function. Be cautious with this as errors in such code can have strange effects.

  • Ensure there are cases where the parent and child process execute differing code by checking the return value of fork()
  • If the child encounters errors, make sure to exit(code) from it: that is a standard C function that ends a process. This avoids the child potentially returning from the function and creating a second instance of commando running.
  • Utilize code patterns that have been studied in lecture and lab to have the child process redirect output and exec() a new program to change its identity

Finish Commands

Commands are started "asynchronously" and checked for completion on occasion. Calls to the cmdctl_update_one() function allow for flexible checking on the running command.

  • If called with finish=0, the command in question will simply be "checked" and if it is not finished, the function returns immediately
  • If called with finish=1, the calling process (parent) will block until the command (child process) completes.

To get these effects, utilize options to the waitpid() system call that adjust between Blocking/Nonblocking versions.

When a child finishes, its exit status is captured and used to diagnose several special cases and update the main processes cmdctl_t struct for the command.

6.2 Grading Criteria for Problem 2   grading 30

Weight Criteria
  AUTOMATED TESTS via make test-prob2
15 Tests provide via test_commando2.org and test_commando.c
  Runs all code under Valgrind to ensure that no memory errors are present.
  1 point per test passed
15 MANUAL INSPECTION
   
5 Code Style: Functions adhere to CMSC 216 C Coding Style Guide which dictates
  reasonable indentation, commenting, consistency of curly usage, etc.
   
5 cmdctl_start_cmd()
  Clear cases for parent and child code that are distinguished by the return value of fork()
  Parent code returns quickly updating the command to RUNNING
  Child proceeds to set up the command to run
  Code to open an output file with the indicated options / permissions
  Code handle output redirection and optionally input redirection in indicated
  Use of an exec()-family function to alter the child process to run the command
  Error checking and exit on problems with I/O redirection or exec()-ing
   
5 cmdctl_update_one() / cmdctl_update_all()
  Early checks of whether the command is still RUNNING with immediate return if not
  Use of waitpid() with appropriate options to affect either Blocking / Non-blocking updates
  Code to capture exit status of child process and inspect it for various cases
  Use of the stat() system call to obtain the size in bytes of the command output
  Alert message printed when the command is --> COMPLETE

7 Problem 3: Commando main()

The commando_main.c file wraps a simple text user interface around the functions from the previous problems. Implementers may add utility/helper functions in commando_mainc.c or commando_funcs.c if desired but these will not be directly tested and were not needed in the reference implementation.

NOTE ON PREVIOUS WORK: The interactive loop that main() implements is similar to that used in past assignments for this course (Lab02 and Project 2 in particular). It will require input and command echoing so a good starting point might be to start with those past solutions.

7.1 Interactive Loop

After setup, the main input loop will likely have the following basic structure.

  1. At the beginning of each loop, update the state of all commands via a call to cmdctl_update_all(). A single cmdctl_t struct is used to track the history and status of all past and present commands.
  2. Print the prompt [COMMANDO]>>
  3. Use a call to fgets() to read a whole line of text from the user. The header has a #define MAX_LINE ... which specifies the maximum length of lines what are supported and a character buffer of that size should be used.
  4. If no input remains (e.g. reach end of input), print End of input and break out of the reading loop.
  5. If echoing is enabled, Echo (print) the given input to the screen. This is required to be compatible with the Automated Tests.
  6. Use the sscanf() function to extract the 0th "token" from the line (e.g. the first word on the line). Study examples of how sscanf() works to read input from a string rather than a file or input but is otherwise similar to other scanf()-style functions.
  7. Examine the 0th token for supported built-ins like help, history, and so forth. Use strcmp() to determine if any match and make appropriate calls. This will be a long if/else-if chain of statements similar to other interactive programs covered in the course.
  8. If a built-in is matched, potentially use additional sscanf() calls to read more tokens from the string then execute behavior associated with the built-in.
  9. If no built-ins match, treat the line as a new command to run. Add a new cmd_t to the cmdctl_t and start it running.

Beginning of Loop Updates and Alerts

At the end of each iteration of the main loop of commando, each command should be checked for updates to its status. This is done via a calls to cmdctl_update_all(). This call should not block: if processes have not finished, commando should not wait for them. That means the underlying calls to waitpid() should use the WNOHANG option which causes waitpid() to return immediately if nothing has happened with the child process.

If the call to cmdctl_update_all() detects a change it should print an Alert message indicating the it has finished. If correctly implemented, underlying calls like cmdctl_update_one() will already print this message. If one or more commands finish, a message is also printed about how many commands finished due to the call to cmdctl_update_all() (also part of the specified behavior for this function). This gives the effect that commands that take a while to complete are reported as finished together. An example is below.

[COMMANDO]>> data/sleep_print.sh 2.5 Last     ## runs for a while
[COMMANDO]>> data/sleep_print.sh 1.0 First    ## runs a shorter time
[COMMANDO]>>                                  ## what a second then press enter
---> COMPLETE:     2 #222530       5 EXIT(0)         data/sleep_print.sh
---> COMPLETE:     3 #222535       6 EXIT(0)         data/sleep_print.sh
2 commands finished
[COMMANDO]>> ls data                          ## run a short command
[COMMANDO]>>                                  ## wait a second then press enter
---> COMPLETE:     4 #222542     242 EXIT(0)         ls
1 commands finished
[COMMANDO]>>

fgets() Usage

The basic usage of fgets() allows a whole line of text to be retrieved. A simple invocation looks like this.

#define CHAR_LIM
{
  FILE *infile = ...;           // might be stdin
  char inbuf[CHAR_LIM];

  char *ret = fgets(inbuf, CHAR_LIM, infile);

  if(ret == NULL){
    printf("Apparently no more input...\n");
  }
  else{
    // ret will point to inbuf so use either
    printf("inbuf is: '%s'\n", inbuf);
    int len = strlen(inbuf);
    printf("inbuf has %d chars\n",len);
    if(inbuf[len-1] == '\n'){
      inbuf[len-1] == '\0';        // "chop" the included newline
    }
    printf("inbuf without newline: '%s'\n",inbuf);
  }
}

This resource has additional information and examples: https://www.tutorialspoint.com/c_standard_library/c_function_fgets.htm

sscanf() Usage

sscanf() is useful to parse off parts of a string into individual tokens. Here is a simple invocation.

{
  char input_string = "Two is 2 and Four is 4.0";
  char tok1[32];
  int tok3;
  int ret = sscanf(input_string, "%s %*s %d", tok1, &tok3);
  // The * in %*s parses a token but does not store it so only 2
  // memory addresses are needed

  printf("tok1: %s\n",tok1);    // Two
  printf("tok3: %d\n",tok3);    // 2
}

Additional examples and information are here: https://www.tutorialspoint.com/c_standard_library/c_function_sscanf.htm

7.2 Startup, Basic Help, Exiting

On startup, a "heading" is printed to indicate that Commando is running and a brief line is printed to indicate the presence of the help built-in. Users can exit by typing the exit command. Below is a demo of these feature.

>> ./commando
=== COMMANDO ver 2 ===
Re-using existing output directory 'commando-dir'
Type 'help' for supported commands
[COMMANDO]>> help
help                       : show this message
exit                       : exit the program
directory                  : show the directory used for command output
history                    : list all jobs that have been started giving information on each
pause <secs>               : pause for the given number of seconds which may be fractional
info <cmdid>               : show detailed information on a command
show-output <cmdid>        : print the output for given command number
finish <cmdid>             : wait until the given command finishes running
finish-all                 : wait for all running commands to finish
source <filename>          : (MAKEUP) open and read input from the given file as though it were typed
command <arg1> <arg2> ...  : non-built-in is run as a command and logged in history
[COMMANDO]>> exit

>> 

Remember on exiting to de-allocate memory, primarily the cmdctl_t struct.

7.3 Command Line Forms and Usage

Commando supports 4 command line forms and if a user invokes the program in any other way, the program should print a usage message like below and exit with code 1 to indicate the program did not run correctly.

>> ./commando abc               ## no option 'abc'
usage: ./commando
       ./commando --echo
       ./commando --dir <dirname>
       ./commando --echo --dir <dirname>
Incorrect command line invocation, use one of the above

>> ./commando --dir             ## missing <dirname> for option
usage: ./commando
       ./commando --echo
       ./commando --dir <dirname>
       ./commando --echo --dir <dirname>
Incorrect command line invocation, use one of the above

>> echo $?                      ## exit code 1 when usage is printed
1

7.4 Command Echoing

To make testing and scripting easier to understand, commando should support command echoing which means to print back to the screen what a user has typed in. If the input source is coming from somewhere else as is the case in testing, this allows the entered commands to be seen in output.

On startup, commando should check for the --echo option and enable echoing if present. This is usually done by setting a variable somewhere to 1 and then printing lines of input each time they are read (or NOT printing them if the variable is 0). Refer to past assignments that demonstrated models for how to do this.

Examples of echoing vs NOT echoing.

>> ./commando                   ## NO echoing 
=== COMMANDO ver 2 ===
Type 'help' for supported commands
[COMMANDO]>> history            ## built-in entered
CMDID PID       BYTES STATE           COMMAND
[COMMANDO]>> directory          ## built-in entered
Output directory is 'commando-dir'
[COMMANDO]>> exit               ## built-in entered

>> ./commando --echo            ## ENABLE echoing
=== COMMANDO ver 2 ===
Type 'help' for supported commands
[COMMANDO]>> history            ## built-in entered
history                         ## and echoed
CMDID PID       BYTES STATE           COMMAND
[COMMANDO]>> directory          ## built-in entered
directory                       ## and echoed
Output directory is 'commando-dir'
[COMMANDO]>> exit               ## built-in entered
exit                            ## and echoed

>> 

7.5 Output Directory

Commando stores output from its commands in a directory. By default this is commando-dir as defined in commando.h

#define DEFAULT_CMDDIR "commando-dir" // default output directory for commands

However, the directory can be changed at startup via the --dir <dirname> option which will need to be taken from the command line arguments to commando. A typical strategy is to have a local variable set initially to the default but if --dir is present, change the variable to the specified value.

Whatever the output directory is, it is used to initialize a cmdctl_t which in turn uses it in cmdctl_create_cmddir(). That function may create the directory, re-use it if existing, or report and error which should cause commando to quit.

>> ./commando --dir special-dir        ## start with non-default directory 
=== COMMANDO ver 2 ===
Type 'help' for supported commands
[COMMANDO]>> exit

>> ./commando --dir special-dir        ## directory already exists
=== COMMANDO ver 2 ===
Re-using existing output directory 'special-dir'
Type 'help' for supported commands
[COMMANDO]>> exit

>> echo Dont overwrite me > other-dir  ## create a file 

>> ./commando --dir other-dir          ## won't overwrite the existing file so quit
=== COMMANDO ver 2 ===
ERROR: Could not create cmd directory 'other-dir'
       Non-directory file with that name already exists
Unable to run without an output directory, exiting

>> 

7.6 Built-ins

Within the interactive loop for commando there are a number of built-in commands. These are demonstrated here.

Help, History, Directory, Exit

These are all fairly self-explanatory based on the output of the help built-in and are shown below.

[COMMANDO]>> help                                           ## print help on builtins
help                       : show this message
exit                       : exit the program
directory                  : show the directory used for command output
history                    : list all jobs that have been started giving information on each
pause <secs>               : pause for the given number of seconds which may be fractional
info <cmdid>               : show detailed information on a command
show-output <cmdid>        : print the output for given command number
finish <cmdid>             : wait until the given command finishes running
finish-all                 : wait for all running commands to finish
source <filename>          : (MAKEUP) open and read input from the given file as though it were typed
command <arg1> <arg2> ...  : non-built-in is run as a command and logged in history

[COMMANDO]>> directory                                      ## print the directory used for command output
Output directory is 'commando-dir'                          ## this is the default value

[COMMANDO]>> exit                                           ## all done for now

>>                                                          ## back to a "real" shell

History, Info, Output

The history of all past and present commands is tracked and can be reported via history which relies on cmdctl_print_all().

The info built-in utilizes cmdctl_print_info(). Note that a command index must be provided and using sscanf() to get the index from the input line is recommended.

The show-output built-in retrieves output from the file associated with the command and displays it on screen using the cmdctl_show_output() function.

[COMMANDO]>> history            ## show current history
CMDID PID       BYTES STATE           COMMAND
    0 #222616     125 EXIT(0)         cat
    1 #222643    1097 EXIT(0)         ls
    2 #222659      95 FAIL_EXEC       data/sleep_print
    3 #222666       - RUNNING         data/sleep_print.sh   
    4 #222677      23 EXIT(0)         wc

[COMMANDO]>> info 4             ## information on 4th command
CMD 4 INFORMATION
command: wc -l data/gettysburg.txt 
argv[]:
  [ 0]: wc
  [ 1]: -l
  [ 2]: data/gettysburg.txt
cmdstate: EXIT(0)
input: <NONE>
output: commando-dir/0004-wc.out
output size: 23 bytes

[COMMANDO]>> show-output 4      ## output for 4th command
28 data/gettysburg.txt

Finishing Commands

All commands are launched asynchronously (background / child process / non-blocking / various other jargon). The finish <cmdid> built-in allows a user to block Commando until a command completes. It utilizes cmdctl_update_one() with appropriate options to accomplish this. Likewise, the built-in finish-all will use cmdctl_update_all() with appropriate options to complete all unfinished commands.

[COMMANDO]>> data/sleep_print.sh 5.0 D                      ## 9th command runs a long time

[COMMANDO]>> data/sleep_print.sh 1.0 D                      ## short run

[COMMANDO]>> data/sleep_print.sh 1.0 D                      ## short run

[COMMANDO]>> finish 9                                       ## block until 9th command finishes
---> COMPLETE:     9 #222765       2 EXIT(0)         data/sleep_print.sh   ## 9th command done
---> COMPLETE:    10 #222770       2 EXIT(0)         data/sleep_print.sh   ## top-of-loop update shows
---> COMPLETE:    11 #222777       2 EXIT(0)         data/sleep_print.sh   ## short run command also done
2 commands finished                                                        ## only 2 as 1st is an explicit finish

[COMMANDO]>> data/sleep_print.sh 5.0 X                      ## start several long-running commands

[COMMANDO]>> data/sleep_print.sh 5.0 X

[COMMANDO]>> data/sleep_print.sh 5.0 X

[COMMANDO]>> finish-all                                     ## wait for all to fininsh
---> COMPLETE:    12 #222793       2 EXIT(0)         data/sleep_print.sh
---> COMPLETE:    13 #222798       2 EXIT(0)         data/sleep_print.sh
---> COMPLETE:    14 #222801       2 EXIT(0)         data/sleep_print.sh
3 commands finished

Pausing

It is useful in testing to be able to have commando simply do nothing for a short time, accomplished with the pause <secs> built-in which utilizes the provided pause_for() function.

Note that the <secs> token is a floating point number so may be fractional (see examples below). Use a call to sscanf() to easily parse this parameter.

After the pause, the standard check of all child processes for state changes should occur which can cause some processes to print that they are done as shown in the following example.

[COMMANDO]>> pause 0.25         ## pause a quarter second
Pausing for 0.250000 seconds

[COMMANDO]>> pause 5.5          ## pause commando for 5.5 seconds
Pausing for 5.500000 seconds
---> COMPLETE:     6 #222705       2 EXIT(0)         data/sleep_print.sh
---> COMPLETE:     7 #222710       2 EXIT(0)         data/sleep_print.sh
---> COMPLETE:     8 #222716       2 EXIT(0)         data/sleep_print.sh
3 commands finished             ## after the pause, several commands finished and are reported

[COMMANDO]>> 

7.7 Running Commands

On inspecting the 0th token of an input line and finding that it does not match any of the built-ins for Commando, the line is interpreted as a Command: an external program to be run as a child process. The command is added to the history via cmdctl_add_cmd() and then started running.

>> ./commando
=== COMMANDO ver 2 ===
Re-using existing output directory 'commando-dir'
Type 'help' for supported commands

[COMMANDO]>> ls data

[COMMANDO]>> gcc --version
---> COMPLETE:     0 #237938     242 EXIT(0)         ls
1 commands finished

[COMMANDO]>> data/table.sh
---> COMPLETE:     1 #237941     228 EXIT(0)         gcc
1 commands finished

[COMMANDO]>> 
---> COMPLETE:     2 #237947    1140 EXIT(0)         data/table.sh
1 commands finished

[COMMANDO]>> data/self_signal.sh 9

[COMMANDO]>> 
---> COMPLETE:     3 #237957      22 SIGNALLED(-9)   data/self_signal.sh
1 commands finished
[COMMANDO]>> history
CMDID PID       BYTES STATE           COMMAND
    0 #237938     242 EXIT(0)         ls
    1 #237941     228 EXIT(0)         gcc
    2 #237947    1140 EXIT(0)         data/table.sh
    3 #237957      22 SIGNALLED(-9)   data/self_signal.sh

[COMMANDO]>> 

7.8 Grading Criteria for Problem 3   grading 35

Weight Criteria
  AUTOMATED TESTS via make test-prob1
15 Tests provide via test_commando3.org and test_commando.c
  Runs all code under Valgrind to ensure that no memory errors are present.
  1 point per test passed
25 MANUAL INSPECTION
   
5 Code Style: Functions adhere to CMSC 216 C Coding Style Guide which dictates
  reasonable indentation, commenting, consistency of curly usage, etc.
   
2 Handling of command line arguments to enable echoing, change output directory
2 Use of fgets() / sscanf() to get input process the 0th token, obtain additional tokens for some built-ins
2 Clear conditional structure to check for built-ins like help, pause, finish, ...
2 Compact handling of built-in cases that leverage functions from commando_funcs.c; no duplicate code
2 Handling of end of input (not just exit) to break from the interactive loop
5 Cases and code to address each of the 10 built-in / command forms

8 MAKEUP Problem: source built-in

It is handy in any shell to be able to execute commands that come from a file. Many shells support this via a source file.txt command and Commando follows this convention.

The general idea is as follows.

  1. Users type source file.txt
  2. Internally, Commando will stop reading Standard Input and attempt open file.txt
  3. If successful, input is read from that file line by line just as if were typed.
  4. When there is no more input in the file, it is closed and Commando proceeds to again read from Standard Input.

To make it clear that input is coming from a sourced script, two things change while reading from the file:

  1. Command echoing is turned on so any input from the script appears on the screen. Echoing is restored to its former on/off state once the script is done.
  2. The prompt changes to reflect input from the file.
    • [COMMANDO]>> is the standard prompt
    • [data/script.txt]>> indicates input is coming from a script after executing the builtin source data/script.txt.

8.1 Example Script Session

>> ./commando
=== COMMANDO ver 2 ===
Re-using existing output directory 'commando-dir'
Type 'help' for supported commands
[COMMANDO]>> help                        ## show the help message which includes
...                                      ## the `source` command
source <filename>          : (MAKEUP) open and read input from the given file as though it were typed
...
[COMMANDO]>> source data/script.txt      ## read and execute all input from a script
Sourcing file 'data/script.txt'          ## print a message about the script
[data/script.txt]>> data/sleep_print.sh 1 hello world
[data/script.txt]>> ls -l                ## prompt changes to reflect the script file
[data/script.txt]>> seq 25
[data/script.txt]>> wc -l Makefile
[data/script.txt]>> pause 1.5
Pausing for 1.500000 seconds
---> COMPLETE:     0 #315311      12 EXIT(0)         data/sleep_print.sh
---> COMPLETE:     1 #315312    2415 EXIT(0)         ls
---> COMPLETE:     2 #315313      66 EXIT(0)         seq
---> COMPLETE:     3 #315314      13 EXIT(0)         wc
4 commands finished
[data/script.txt]>>                      ## no more input in the script
End of input                           
Closing source file 'data/script.txt'    ## transition back to stadard inpu
[COMMANDO]>> history                     ## back to standard input; read more commands
CMDID PID       BYTES STATE           COMMAND
    0 #315311      12 EXIT(0)         data/sleep_print.sh
    1 #315312    2415 EXIT(0)         ls
    2 #315313      66 EXIT(0)         seq
    3 #315314      13 EXIT(0)         wc
[COMMANDO]>> exit                        ## exit commando

8.2 Details of Implementation

There are a few details and constraints and the source builtin:

  1. Messages should be printed at the beginning of the script execution and at the end of execution.

       [COMMANDO]>> source data/script.txt
       Sourcing file 'data/script.txt'
       ...
       ...
       Closing source file 'data/script.txt'
       [COMMANDO]>>
    
  2. While reading commands from the script, the prompt changes from the default to [COMMANDO]>> to the name of the script:

       [COMMANDO]>> source data/script.txt
       Sourcing file 'data/script.txt'
       [data/script.txt]>> data/sleep_print.sh 1 hello world
       [data/script.txt]>> ls -l
       ...
    

    The prompt reverts to the default when the script finishes.

  3. The script may end with either the exit command or just with the end of the file. Both of these should cause a return to reading from Standard Input. When finishing the script due to end of input the normal End of Input message should be printed.

       [COMMANDO]>> source data/script1.txt
       Sourcing file 'data/script1.txt'
       [data/script1.txt]>> help
       ...
       [data/script1.txt]>> exit       ## script ends with exit
       Closing source file 'data/script1.txt'
       [COMMANDO]>> 
       ...
       [COMMANDO]>> source data/script.txt
       Sourcing file 'data/script.txt'
       [data/script.txt]>> data/sleep_print.sh 1 hello world
       ...
       [data/script.txt]>> 
       End of input                    ## script just ends, no exit
       Closing source file 'data/script.txt'
       [COMMANDO]>> 
    
  4. No "recursive source" is supported: if a sourced file has the source second.txt command in it, print an error message and ignore the line.

       [COMMANDO]>> source data/script6.txt
       Sourcing file 'data/script6.txt'
       
       [data/script6.txt]>> history
       CMDID PID       BYTES STATE           COMMAND
       
       [data/script6.txt]>> directory
       Output directory is 'commando-dir'
       
       [data/script6.txt]>> source data/script1.txt    ## not supported, 
       Already reading a source file 'data/script6.txt'; recursive source not supported
       
       [data/script6.txt]>> pause 0.1
       Pausing for 0.100000 seconds
       
       [data/script6.txt]>> exit
       Closing source file 'data/script6.txt'
       
       [COMMANDO]>> 
    
  5. Otherwise all other builtins and commands should behave as they would if executed from standard input (including blank line handling).

8.3 Design Description

Implementers may use whatever mechanism they wish to implement the temporary execution of commands from a sourced file. However, they must include a description of their design in the comments in commando_main.c.

  • Start a block comment with the exact string MAKEUP DESIGN (test 10 checks for this)
  • Describe how the design of the source builtin works including extra variables, functions, changes to the original main() etc. that are required.
  • Specifically describe in the comment these items (and anything else of importance)
    • (A) RELEVANT VARIABLES / STATE: what variables are used to set up handling sourced files and distinguish between reading input from standard in and files
    • (B) HANDLING source BUILTIN: what code is added to handle the source case and set up handling of input from it.
    • (C) HANDLING exit / End of Input IN SCRIPTS: how is the end of a sourced script handled and reading from Standard In restored.
    • (D) HANDLING AND RESTORING ECHOING: echoing should be on when a script is processed but restored to its previous state (on/off); how is this handled?
    • (E) PREVENTING RECURSIVE source: how is a source command within a sourced file handled and prevented from executing?
    • (F) DUPLICATED CODE: was it necessary to repeat any code amid main() to support sourcing - e.g. to handle control flow / input data / other items the same exact code was copied and pasted to several spots. Try to limit this as too much copy-paste will be penalized. A good evaluation metric is to ask: If a new builtin was added, how many parts of main() would need to change to support it?
  • Use a single large comment to describe the design decisions in prose (written words). Comments can be placed elsewhere but the MAKEUP DESIGN comment should explain the broad strokes of the design.

Any implementation that does not have the MAKEUP DESIGN comment may have some or all credit for automated tests removed.

8.4 Grading Criteria for Makeup Problem   grading 15

MAKEUP Credit adds to the overall total of points earned across all projects and make make up for credit lost here or on past/future projects. It will not cause the overall credit across all projects to exceed the 100% of the weight allotted in the syllabus (e.g. max of 500 / 500 points on all 5 projects).

Weight Criteria
  AUTOMATED TESTS via make test-makeup
10 Tests provide via test_commado_makeup.org and test_commando.c
  Runs all code under Valgrind to ensure that no memory errors are present.
  1 point per test passed
  Penalties applied if Manual Inspection criteria are not met
5 MANUAL INSPECTION
  MAKEUP DESIGN Comment is present and includes written information on the design
  Description includes information on (A)-(F) parts
  Reasonably clean design that limits the amount of copy/paste code to support source
  A missing or poor MAKEUP DESIGN will reduce credit for Automated Tests

9 Project Submission

9.1 Submit to Gradescope

Refer to the Project 1 instructions and adapt them for details of how to submit to Gradescope. In summary they are

  1. Type make zip in the project directory to create p4-complete.zip
  2. Log into Gradescope, select Project 4, and upload p4-complete.zip

9.2 Late Policies

You may wish to review the policy on late project submission which will cost 1 Engagement Point per day late. No projects will be accepted more than 48 hours after the deadline.

https://www.cs.umd.edu/~profk/216/syllabus.html#late-submission


Web Accessibility
Author: Chris Kauffman (profk@umd.edu)
Date: 2025-04-29 Tue 13:14