# `FAWOC` the FAst WOrd Classifier
FAWOC is a TUI program for manually labelling a list of words.
It has been developed to support the efficient clustering of documents based on topic modeling algorithms such as Dirichlet Latent Allocation.

The programs reads a CSV file containing the terms and allows the fast association of labels to the terms.
Each term is presented to the user, who can associate to the term one of the labels with the press of a key.
Some statistics are provided in the user interface to have a clue about the number of classified terms and the remaining ones.
The terms are sorted according to their frequency in the set of documents, which is a value that must be made available to FAWOC.
## Example of usage
```
fawoc terms.csv
```
The input file `terms.csv` needs to have at least one column with the header (first column) called `term`.
## Available commands and keybindings
The following labels are currently supported:
* k keyword
* n noise
* r relevant
* x not-relevant
* s stopword
* p postponed
* a autonoise
Other keys allow to save and quit:
* w save immediately
* q quit
FAWOC automatically saves the changes on closing.
Moreover, it autosaves the changes every 10 classified words.
## Logging
FAWOC writes profiling information into the file `profiler.log` with the relevant operations that are carried out.
## Files
FAWOC reads the terms from a tsv file with the following structure:
* `id`: identification number of the term. Must be unique. For backward compatibility this column may be missing. In this case, FAWOC assigns an id to each term that will be saved in a newly created column `id` on the first save;
* `term`: the term itself. For backward compatibility with old files, this column can be called `keyword`. This name is deprecated, and FAWOC will change it to `term` on the first save;
* `label`: a string describing the label assigned to the term.
FAWOC will load other information from two service files.
These files are named after the input file, by removing its extension, adding the suffix `fawoc_data` and then adding the proper extension.
The service files are:
* `*_fawoc_data.tsv`: it contains static information about each term. It is saved only on FAWOC closing. Currently, it is used to load the number of occurrences of each term;
* `*_fawoc_data.json`: it contains information used by FAWOC to correctly handle the undo command.
The `--no-info-file` command line option can be used to tell FAWOC to not load (and save) the `*_fawoc_data.tsv`.
With this option, FAWOC will not display the count value.
### `*_fawoc_data.tsv`
The format of this tsv file is:
* `id`: identification number of the term;
* `term`: the term itself. This field is not directly ised by FAWOC, and it is here only the make this file more readable;
* `count`: the number of occurrences of the term.
For backward compatibility with old files, if the `*_fawoc_data.tsv` file is missing, FAWOC searches for the `count` column in the input file.
If this column is found, then FAWOC will use this value, otherwise the value -1 will be used.
A new file `*_fawoc_data.tsv` is created on the first save with the loaded values of `count`.
### `*_fawoc_data.json`
The format of this JSON file is a dictionary.
The keys of this dictionary are the id of the terms.
The values are dictionaries with the following format:
* `order`: number indicating the order in which each term is classified;
* `related`: related term selected at the moment the term is classified.
For backward compatibility with old files, if the `*_fawoc_data.json` is missing, FAWOC searches the `order` and the `related` fields in the input file.
If they are not found, then FAWOC will not be able to handle the undo of the classifications made before.
Each new classification will have its own entry in a newly created `*_fawoc_data.json`.