معرفی شرکت ها
dagrules-0.1.0
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
dagrules - dbt DAG rule creator and validator
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | dagrules-0.1.0 |
| نام | dagrules |
| نسخه کتابخانه | 0.1.0 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | Sterling Paramore |
| ایمیل نویسنده | gnilrets@gmail.com |
| آدرس صفحه اصلی | https://github.com/gnilrets/dagrules |
| آدرس اینترنتی | https://pypi.org/project/dagrules/ |
| مجوز | MIT |
# dagrules
dagrules is a tool that allows you to write your own dbt dag rules and check
that your dbt project is conforming to those rules.
## Overview
While the dbt community has established some excellent guidelines for
[how to structure dbt
projects](https://discourse.getdbt.com/t/how-we-structure-our-dbt-projects/355),
those conventions are not automatically enforced. Those
conventions are simply guidelines, and each team may decide on a
slightly different set of conventions that work best for their
particular set up. dagrules was developed to allow you to write your
own conventions in a simple `yaml` document, and have those
conventions enforced via your CI system.
To use dagrules, all you need is a dbt project and a `dagrules.yml`
file located in the root of the dbt project (e.g.,
`dbt/dagrules.yml`). The yaml file should look like (for a more
complete example, see [tests/dagrules.yml](test/dagrules.yml):
````yaml
---
version: '1'
rules:
- name: The name of my rule
subject:
... # How to select nodes to check that they satisfy the rules
must:
... # Define the rules that must be followed
- name: Another one of my rules
...
````
### Installation and running rules
dagrules can be installed using pip:
````bash
pip install dagrules
````
And then run `dagrules` with the `--check` argument from your dbt project root:
````bash
dagrules --check
````
dagrules assumes that it is being executed from the dbt project root and that there is
a `target/manifest.json` file already present (so the dbt project must be compiled
any time the dag is changed before dagrules can be run). These defaults can
be overridden by setting the `DBT_ROOT` and `DAGRULES_YAML` environment variable to
point to other locations.
## Subjects
For every rule, a subject should be declared that defines how to
select nodes of the dbt dag to use for rule validation. Omitting the
subject means that the rule will be applied to every dbt model.
dagrules currently supports two ways to select subjects: 1) by node
type (source, snapshot, model) and 2) by tags. For example, the
follow subject includes all models that are tagged "staging":
````yaml
rules:
- name: All staging models must ...
subject:
type: model
tags: staging
must:
...
````
## Tag selection
Tag selection applies both to `subject` and `must` section of the
dagrules yaml spec. Tags can be defined several ways.
**Single string** - Selecting with a single tag can be expressed as a simple string
````yaml
tags: staging
````
**List of tags: match any** - A list of tags can also be specified, and
dagrules will match nodes with **any** of the tags in the list. The
example below will match nodes having either `base` or `intermediate`
tags.
````yaml
tags:
- base
- intermediate
````
**Include: match all with exclusions** - When you need to select nodes
that match **all** tags in a list, and possibly exclude nodes with
some tags as well, you can use include/exclude. The example below
will select any nodes that have both "staging" and "finance" tags, but
that don't also have the `base` tag.
````yaml
tags:
include:
- staging
- finance
exclude:
- base
````
The arguments to `include` and `exclude` can either be a list or single strings.
**Combine any/all** - We can also combine **any** and **all** syntaxes
at once. The following will select all nodes that are either
"non-base staging", "core", or "mart" models.:
````yaml
tags:
- include: staging
exclude: base
- core
- mart
````
## Musts
"Musts" define the rules that must be adhered to by the subjects defined in the `subject`
section. Multiple "musts" may be included in a rule definition, and all must be
satisfied for the rule to pass.
**Match name** - The `match-name` rule requires that each subject adhere to a
particular naming pattern. dagrules currently only supports regular expression matching.
For example, the following rule enforces that all snapshot models must be named with
a `snap_` prefix:
````yaml
rules:
- name: Snapshot must be prefixed with snap_
subject:
type: snapshot
must:
match-name: /snap_.*/
````
**Have tags** - The `have-tags-any` rule requires that all selected models must have
one of any of the listed tags. The following example specifies that all nodes in the dag
must have at least one of the tags listed:
````yaml
rules:
- name: All models must be tagged either snapshot, base, staging, intermediate, core, mart
# Omit subject to include all nodes
must:
have-tags-any:
- snaphost
- base
- staging
- intermediate
- core
- mart
````
**Have parent or child relationship** - The `have-child-relationship`
and `have-parent-relationship` rules require that the subjects have a
certain kind of relationship to either their **immediate** children or
parents. The types of relationship can involve:
* `cardinality` - The cardinality of the relationship between a subject and its child/parent
can either be `one_to_one` or `one_to_many` (default). If `one_to_one` is selected,
that a subject may only have one child/parent.
* `required` - Indicates whether a child/parent relationship is required or not. The default
is `True`, meaning that if a relationship is defined, all subject must have at least
one child or parent node. If `False`, then a subject may have 0 children/parents.
* `require-tags-any` - Contains a list of tags that the parent/child
must have (with syntax defined in the "Tag selection" section
above).
* `require-node-type` - Indicates the node type (source, snapshot, model) that the child/parent
must be in order to pass.
* `select-tags-any` - Contains a list of tags that restricts the selection of parents/children
involved in the rule.
* `select-node-type` - Indicates that only the parents/children with the specified node
type are to be considered when checking the rule.
For example,
````yaml
rules:
- name: Snapshots must have 0 or 1 children, which must all be base models
subject:
type: snapshot
must:
have-child-relationship:
cardinality: one_to_one
required: false
require-tags-any:
- base
- name: Intermediate models may only depend on non-base staging, core, mart, or other intermediate models
subject:
tags:
include: intermediate
must:
have-parent-relationship:
require-tags-any:
- include: staging
exclude: base
- core
- mart
- intermediate
````
## Contributing
We welcome contributors! Please submit any suggests or pull requests in Github.
### Developer setup
Create an appropriate python environment. I like [miniconda](https://conda.io/miniconda.html),
but use whatever you like:
conda create --name dagrules python=3.9
conda activate dagrules
Then install pip packages
pip install pip-tools
pip install --ignore-installed -r requirements.txt
run tests via
inv test
and the linter via
inv lint
زبان مورد نیاز
| مقدار | نام |
|---|---|
| >=3 | Python |
نحوه نصب
نصب پکیج whl dagrules-0.1.0:
pip install dagrules-0.1.0.whl
نصب پکیج tar.gz dagrules-0.1.0:
pip install dagrules-0.1.0.tar.gz