# AccessLog Parser and CLI
Web server access log parser and CLI tool with added features for web archive replay logs.
## Installation
This package requires Python 3.6 or above.
Install the latest version of the released package and `accesslog` CLI tool from PyPi:
```
$ pip install accesslog
```
Alternatively, build and install the development version of the package:
```
$ git clone https://github.com/oduwsdl/accesslog-parser.git
$ cd accesslog-parser
$ python3 setup.py install
$ accesslog --version
```
## Input Parsing
TODO
## Record Filtering
TODO
## Output Formatting
TODO
## CLI Reference
```
$ accesslog -h
usage: accesslog [options] [FILES ...]
A tool to parse Common Log formatted access logs with various derived fields.
positional arguments:
files Log files (plain/gz/bz2) to parse (reads from the STDIN, if empty or '-')
optional arguments:
-h, --help Show this help message and exit
-v, --version Show version number and exit
-d, --debug Show debug messages on STDERR
-e FIELDS, --nonempty FIELDS
Skip record if any of the provided fields is empty (comma separated list)
-i FIELDS, --valid FIELDS
Skip record if any of the provided field values are invalid
('all' or comma separated list from 'host,request,status,size,referrer')
-m FIELD~RegExp, --match FIELD~RegExp
Skip record if field does not match the RegExp (can be used multiple times)
-t TFORMAT, --origtime TFORMAT
Original datetime format of logs (default: '%d/%b/%Y:%H:%M:%S %z')
-f FORMAT, --format FORMAT
Output format string (see available formatting fields below)
-j FIELDS, --json FIELDS
Output NDJSON with the provided fields (use 'all' for all fields except 'origline')
formatting fields:
{origline} Original log line
{host} IP address of the client
{identity} Identity of the client, usually '-'
{user} User ID for authentication, usually '-'
{origtime} Original date and time (typically in '%d/%b/%Y:%H:%M:%S %z' format)
{epoch} Seconds from the Unix epoch (derived from origtime)
{date} UTC date in '%Y-%m-%d' format (derived from origtime)
{time} UTC time in '%H:%M:%S' format (derived from origtime)
{datetime} 14 digit datetime in '%Y%m%d%H%M%S' format (derived from origtime)
{request} Original HTTP request line
{method} HTTP method (empty for invalid request)
{path} Path and query (scheme and host removed, empty for invalid request)
{prefix} Memento endpoint path prefix (derived from path)
{mtime} 14 digit Memento datetime (derived from path)
{rflag} Memento rewrite flag (derived from path)
{urir} Memento URI-R (derived from path)
{httpv} HTTP version (empty for invalid request)
{status} Returned status code
{size} Number of bytes returned
{referrer} Referer header (empty, if not logged)
{agent} User-agent header (empty, if not logged)
{extras} Any additional logged fields
Default FORMAT: '{host} {date} {time} {method} {path} {status} {size} "{referrer}" "{agent}"'
```
