# Astronomer Nova
### (Formerly Airflow V2 Upgrader)
This project aims to automate common tasks when upgrading DAGs from Airflow 1 to Airflow 2,
making use of [RedBaron](https://redbaron.readthedocs.io/en/latest/) to parse and manipulate the Python syntax tree.
# Features
- Automatically upgrade DAG source code from Airflow 1 to Airflow 2
- Scan Dags and report changes (Coming Soon)
# Installation
You can install the Astronomer Nova CLI through Pip
```shell
pip install astronomer-nova
```
The simplest way to get started with Astronomer Nova is by using the astro cli.
Simply add `astronomer-nova` to your list of dependencies in `requirements.txt`.
Then you can run it with the following commands:
```shell
astro dev start
astro dev bash
nova
```
## Installation from sources
To install astronomer-nova in [development mode](https://pip.pypa.io/en/latest/cli/pip_install/#install-editable):
```shell
pip install --editable .[lint,test]
```
To build from sources:
```
python -m build
```
OR
```
python -m pip wheel .
```
## Running tests:
```shell
python -m pytest
```
# How to Use
```shell
nova dags/ dags/upgraded/
```
For more information, you can access the help menu with:
```shell
nova --help
```
# How it works
Astronomer Nova performs the upgrade by:
1. Building an import map for classes in `airflow.hooks`, `airflow.operators`, `airflow.sensors`, and `airflow.providers`, which maps the classes to the current Airflow version import path.
2. Modifying Python files in a DAG directory to update
- Imports
- DAG access controls
- XCOM push
- XCOM pull
3. Writing updated DAG to either the same DAG directory or a parameter output directory with `_upgraded` added to the filename.
The script is configured to run against `./dags` and to write output to `./dags/upgraded` by default.
## Caveats
In order to load the Airflow import map, Airflow provider packages must be installed. There are often cross-dependencies within Airflow providers (eg `airflow.operators.s3_to_hive_operator` required the Hive provider), so make sure the `requirements.txt` file includes all necessary .
The script currently tries to correct imports that do not need to be changed going from Airflow 1 to 2, and logging does not include which files are currently being acted against.
The code uses Astronomer-centric opinions, like removing DAG-level RBAC. In the future, I'd rather have these decisions configurable at runtime, as well as generating per-DAG reports on the DAGs that can be used for further remediation, as well as understanding of DAG requirements, like the use of Connections and Variables.
# Contribute
- Issue Tracker: https://github.com/astronomer/astronomer-nova/issues
- Source Code: https://github.com/astronomer/astronomer-nova
# Contact
cabella@astronomer.io
