معرفی شرکت ها
dataflows-0.3.8
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
A nifty data processing framework, based on data packages
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | dataflows-0.3.8 |
| نام | dataflows |
| نسخه کتابخانه | 0.3.8 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | Adam Kariv |
| ایمیل نویسنده | adam.kariv@gmail.com |
| آدرس صفحه اصلی | https://github.com/datahq/dataflows |
| آدرس اینترنتی | https://pypi.org/project/dataflows/ |
| مجوز | MIT |
#  DataFlows
[](https://travis-ci.org/datahq/dataflows)
[](https://coveralls.io/r/datahq/dataflows?branch=master)
[](https://gitter.im/dataflows-chat/Lobby)
DataFlows is a simple and intuitive way of building data processing flows.
- It's built for small-to-medium-data processing - data that fits on your hard drive, but is too big to load in Excel or as-is into Python, and not big enough to require spinning up a Hadoop cluster...
- It's built upon the foundation of the Frictionless Data project - which means that all data produced by these flows is easily reusable by others.
- It's a pattern not a heavy-weight framework: if you already have a bunch of download and extract scripts this will be a natural fit
Read more in the [Features section below](#features).
## QuickStart
Install `dataflows` via `pip install.`
(If you are using minimal UNIX OS, run first `sudo apt install build-essential`)
Then use the command-line interface to bootstrap a basic processing script for any remote data file:
```bash
# Install from PyPi
$ pip install dataflows
# Inspect a remote CSV file
$ dataflows init https://raw.githubusercontent.com/datahq/dataflows/master/data/academy.csv
Writing processing code into academy_csv.py
Running academy_csv.py
academy:
# Year Ceremony Award Winner Name Film
(string) (integer) (string) (string) (string) (string)
---- ---------- ----------- -------------------------------- ---------- ------------------------------ -------------------
1 1927/1928 1 Actor Richard Barthelmess The Noose
2 1927/1928 1 Actor 1 Emil Jannings The Last Command
3 1927/1928 1 Actress Louise Dresser A Ship Comes In
4 1927/1928 1 Actress 1 Janet Gaynor 7th Heaven
5 1927/1928 1 Actress Gloria Swanson Sadie Thompson
6 1927/1928 1 Art Direction Rochus Gliese Sunrise
7 1927/1928 1 Art Direction 1 William Cameron Menzies The Dove; Tempest
...
# dataflows create a local package of the data and a reusable processing script which you can tinker with
$ tree
.
├── academy_csv
│ ├── academy.csv
│ └── datapackage.json
└── academy_csv.py
1 directory, 3 files
# Resulting 'Data Package' is super easy to use in Python
[adam] ~/code/budgetkey-apps/budgetkey-app-main-page/tmp (master=) $ python
Python 3.6.1 (default, Mar 27 2017, 00:25:54)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from datapackage import Package
>>> pkg = Package('academy_csv/datapackage.json')
>>> it = pkg.resources[0].iter(keyed=True)
>>> next(it)
{'Year': '1927/1928', 'Ceremony': 1, 'Award': 'Actor', 'Winner': None, 'Name': 'Richard Barthelmess', 'Film': 'The Noose'}
>>> next(it)
{'Year': '1927/1928', 'Ceremony': 1, 'Award': 'Actor', 'Winner': '1', 'Name': 'Emil Jannings', 'Film': 'The Last Command'}
# You now run `academy_csv.py` to repeat the process
# And obviously modify it to add data modification steps
```
## Features
* Trivial to get started and easy to scale up
* Set up and run from command line in seconds ...
* `dataflows init` => `flow.py`
* `python flow.py`
* Validate input (and esp source) quickly (non-zero length, right structure, etc.)
* Supports caching data from source and even between steps
* so that we can run and test quickly (retrieving is slow)
* Immediate test is run: and look at output ...
* Log, debug, rerun
* Degrades to simple python
* Conventions over configuration
* Log exceptions and / or terminate
* The input to each stage is a Data Package or Data Resource (not a previous task)
* Data package based and compatible
* Processors can be a function (or a class) processing row-by-row, resource-by-resource or a full package
* A pre-existing decent contrib library of Readers (Collectors) and Processors and Writers
## Learn more
Dive into the [Tutorial](TUTORIAL.md) to get a deeper glimpse into everything that `dataflows` can do.
Also review this list of [Built-in Processors](PROCESSORS.md), which also includes an API reference for each one of them.
نیازمندی
| مقدار | نام |
|---|---|
| >=1.53.5 | tabulator |
| >=1.5.0 | datapackage |
| >=1.20 | tableschema |
| >=0.0.9 | kvfile |
| - | click |
| - | jinja2 |
| - | awesome-slugify |
| - | inquirer |
| - | tabulate |
| - | tableschema-sql |
| - | xmljson |
| >=3 | bitstring |
| - | python-dateutil |
| - | openpyxl |
| <2 | sqlalchemy |
| - | pylama |
| - | pylama-quotes |
| <2.5 | pyflakes |
| - | mock |
| - | pytest |
| - | pytest-cov |
| - | coverage |
| - | lxml |
| - | plyvel |
نحوه نصب
نصب پکیج whl dataflows-0.3.8:
pip install dataflows-0.3.8.whl
نصب پکیج tar.gz dataflows-0.3.8:
pip install dataflows-0.3.8.tar.gz