# crashbang
A tool to aid in testing the stability of programs
Crashbang is a tool that allows you to easily run a program through several
iterations, speeding up the process of testing a program's stability. It also
gathers data about the iterations and can provide statistics about successful
exits versus crashes. Data is presented upon test run completion and program
output can be saved to a file for later analysis. There are also facilities for
limited automated testing.
Crashbang is essentially a beefed up loop. It runs a program through the
specified number of iterations and detects what status the program exited with.
Non-zero exit status constitutes an error, so those are flagged as failures,
while zero is counted as a success. If a timeout is specified and the program
doesn't exit before the timeout is reached, crashbang will automatically close
the program and the iteration is counted as a success. Crashbang can also detect
crashed caused by a signal (e.g. SIGIOT, Segmentation faults, etc.) and will
display the recieved signal in the output file.
## Usage
Basic usage is to prepend `crashbang` to the program you want to run. This will
run through 10 iterations of the program, waiting for the program to naturally
terminate before proceeding to the next one.
```
crashbang program-name
```
### Adding Arguments
Arguments to pass to the tested program are specified using the -a flag, and
must be wrapped in quotation marks.
```
crashbang program-name -a "--first-argument --second-argument=true"
```
### Options
#### Iterations
Crashbang supports any number of iterations. Iterations can be set using the
`-i` flag. The default number of iterations is 10.
```
crashbang program-name -i 100
```
#### Timeout
A timeout can also be specified. This is helpful when creating large automated
testing runs. With the timeout specified, Crashbang waits for natural program
termination up to the specified timeout. If the timeout is reached, the program
is stopped and the iteration is counted as a success, then a new iteration is
started. If the program exits with 0 prior to the timeout, the iteration is also
considered successful. Crashes before the timeout are still treated as failures,
and if the program exits for any reason before the timeout is reached, the
remainder of the timeout is skipped and the next iteration begins immediately.
Timeouts can be specified in seconds with the '-t' flag:
```
crashbang program-name -t 10
```
## Output
Upon completion or termination of a test run, Crashbang will display the
gathered statistics from the test run, including the program name, the total
number of iterations, the number of successful iterations, the number of crashes
detected, and the failure percentage.
Crashbang currently handles the KeyboardInterrupt exception to allow it to
display statistics for completed iterations if the user uses Ctrl-C to exit the
test run early. The output will reflect the total number of completed iterations
in addition to the successes and crashes.
### Output File
Crashbang also outputs a file with the console output (stdout and stderr) of any
failed iterations. It also outputs the iteration number and the exit status ( or
signal, if the termination was caused by a signal).
## Planned Features
The following features are currently not implemented, but are planned for
inclusion:
* Performance/timing information
* Status and ETA for completion of a test run
* Expanded output control
* Hooks for other automated testing tools