معرفی شرکت ها
fibberio-1.2.0
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
-
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | fibberio-1.2.0 |
| نام | fibberio |
| نسخه کتابخانه | 1.2.0 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | sethjuarez |
| ایمیل نویسنده | me@sethjuarez.com |
| آدرس صفحه اصلی | https://github.com/sethjuarez/fibberio |
| آدرس اینترنتی | https://pypi.org/project/fibberio/ |
| مجوز | - |
# fibber
(This is still under development)
Teaching machine learning things is hard. The idea behind this library is to generate data in such a way that certain principles can be highlighted without resorting to "finding" the perfect dataset to do so.
Currently the library can be installed using `pip`:
```
pip install fibberio
```
Once the library is installed in your python environment, you can start generating data by:
```
fibber -t .\tests\data\programmers.json -o .\sandbox\programmers.csv -c 10000
```
where `-t` is the Task Description file and `-o` is the output file. To specify the record count, the `-c` flag is used. Successfully running the command should show the following:
```
Generating 10000 items using "programmers.json"
-----------------------------------------------
FirstName LastName age style desc accept
count 10000 10000 10000.000000 10000 10000.000000 10000
unique 966 1000 NaN 2 NaN 2
top Remy Anthony NaN tabs NaN False
freq 29 21 NaN 6642 NaN 5378
mean NaN NaN 35.985700 NaN 21.736883 NaN
std NaN NaN 4.983832 NaN 10.526532 NaN
min NaN NaN 18.000000 NaN 5.010000 NaN
25% NaN NaN 33.000000 NaN 12.580000 NaN
50% NaN NaN 36.000000 NaN 20.070000 NaN
75% NaN NaN 39.000000 NaN 34.660000 NaN
max NaN NaN 57.000000 NaN 36.800000 NaN
Saving csv to C:\projects\fibberio\sandbox\programmers.csv
Task complete
```
The [programmers.json](./tests/data/programmers.json) file is a good starting point for understanding task descriptions.
# Task Description
The best way to understand how it works is to look at a task description:
```json
{
"sources": [
{
"id": "names",
"pandas": {
"path": "./full_names.csv",
"read_csv": {
"encoding": "unicode_escape",
"engine": "python"
}
}
}
],
"features": [
{
"id": "first_name",
"source": {
"id": "names",
"target": "FirstName"
}
},
{
"id": "age",
"normal": {
"mean": 36,
"stddev": 5,
"precision": 0
}
},
{
"id": "style",
"discrete": {
"tabs": 2,
"spaces": 1
}
},
]
}
```
There are two specific sections:
1. **Sources** - external reference data
2. **Features** - columns to generate
## Sources
The `sources` section contains a dictionary containing references to external files with data that can be sampled later as features.
```json
{
"id": "names",
"pandas": {
"path": "./full_names.csv",
"read_csv": {
"encoding": "unicode_escape",
"engine": "python"
}
}
}
```
The `id` is the identifier used to reference this data source later in the features. [`read_csv`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) in this case is the call to the pandas [`read_csv`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) function call with the enclosed dictionary representing the `**kwargs` passed to that function. In theory, any pandas call to load any file type can be used here (although as of the time of this writing, `read_csv` is the only one that has been tried).
The `path` to the data file (in the case above [./full_names.csv](./tests/data/full_names.csv)) is *in relation to the task description file* unless the full path is specified.
## Features
The `features` section contains the features the system should generate along with their corresponding distributions:
```json
"features": [
{
"id": "first_name",
"source": {
"id": "names",
"target": "FirstName"
}
},
{
"id": "age",
"normal": {
"mean": 36,
"stddev": 5,
"precision": 0
}
},
{
"id": "style",
"discrete": {
"tabs": 2,
"spaces": 1
}
}
]
```
In this example there are exactly three features:
1. **first_name** - this references the `names` source and samples from the `FirstName` column
2. **age** - this samples from the `normal` distribution with three parameters passed in to the `Normal` class as `**kwargs`
3. **style** - this samples from a discrete distribution that will generate `tabs` and `spaces` in a 2 to 1 ratio
The standard definition for a feature therefore consists of:
```json
{
"id": "feature_id"
"distribution_class": {
[... distribution args ...]
}
}
```
Where the `feature_id` represents the id of the feature and the column name (this can be overriden in certain samplers). The `distribution_class` is the name of a `Distribution` class which is instantiated with the corresponding args.
Essentially, if the Distribution class is instantiated by:
```
distribution_class(prop1=2, prop2=seismic)
```
then the corresponding `kwargs` should look like
```
{
"prop1": 2,
"prop2": "seismic"
}
```
and get instantiated by
```
distribution_class(**kwargs)
```
I am optimizing for readibility as opposed to brevity. This requires the class to have an `__init()__` with default named parameters.
The optional `conditional` part of the feature is described next.
## Conditionals
Feature conditionals allow for conditional sampling based on the parent distribution. Here's an example:
```json
"features": [
{
"id": "age",
"normal": {
"mean": 36,
"stddev": 5,
"precision": 0
}
},
{
"id": "score",
"conditional": {
"marginal": "age",
"posterior": [
{
"value": "[14, 65)",
"uniform": {
"low": 5,
"high": 25,
"itype": "float",
"precision": 2
}
},
{
"value": "[65, *)",
"normal": {
"mean": 35,
"stddev": 0.5
}
},
{
"value": "*",
"uniform": {
"low": 5,
"high": 25,
"itype": "float",
"precision": 2
}
}
]
}
}
]
```
This describes `score` feature conditioned on the `age` feature (as the marginal). Since the parent distribution is continuous, the conditional subdivisions should be represented by ranges:
- $[a, b]$ the closed interval ${ x \in \mathbb{R}: a \le x \le b }$
- $[a, b)$ the interval ${ x \in \mathbb{R}: a \le x \lt b }$
- $(a, b]$ the interval ${ x \in \mathbb{R}: a \lt x \le b }$
- $(a, b)$ the open interval ${ x \in \mathbb{R}: a \lt x \lt b }$
with `*` representing a catch within the range interval or as the "catch-all" - these are processed in order and an exception is raised if none of the criteria fit.
The task processes each top level feature and then passes the generated value to the conditional which evaluates each range and generates from the distribution which "catches" the generated top level value.
This also is true for discrete probability distributions:
```json
"features": [
{
"id": "style",
"discrete": {
"tabs": 234,
"spaces": 2332,
"agile": 21,
"scrum": 128
},
},
{
"id": "score",
"conditional": {
"marginal": "score",
"posterior": [
{
"value": "tabs",
"uniform": {
"low": 5,
"high": 25,
"itype": "float",
"precision": 2
}
},
{
"value": "*",
"normal": {
"mean": 12,
"stddev": 3
}
}
]
}
}
]
```
In this case, the conditional `score` feature will sample from the `uniform` distribution if "tabs" is generated for the `style` feature, otherwise the catch-all `*` will sample from the `normal` distribution.
These dependencies can be chained:
```json
"features": [
{
"id": "style",
"discrete": {
"tabs": 234,
"spaces": 2332,
"agile": 21,
"scrum": 128
},
},
{
"id": "score",
"conditional": {
"marginal": "style",
"posterior": [
{
"value": "tabs",
"uniform": {
"low": 5,
"high": 25,
"itype": "float",
"precision": 2
}
},
{
"value": "*",
"normal": {
"mean": 12,
"stddev": 3
}
}
]
}
},
{
"id": "accepted",
"conditional": {
"marginal": "score",
"posterior": [
{
"value": "[14, 65)",
"uniform": {
"low": 5,
"high": 25,
"itype": "float",
"precision": 2
}
},
{
"value": "[65, *)",
"normal": {
"mean": 35,
"stddev": 0.5
}
},
{
"value": "*",
"uniform": {
"low": 5,
"high": 25,
"itype": "float",
"precision": 2
}
}
]
}
}
]
```
Notice that in this case, the first conditional required discrete values while the second used ranges. An exception is raised if there is a mismatch.
The main idea is that every Feature has a `distribution` and optional dependant `conditional`
نیازمندی
| مقدار | نام |
|---|---|
| >=8.0.3,<9.0.0 | click |
| >=6.0,<7.0 | PyYAML |
| >=0.8.1,<0.9.0 | parsimonious |
| >=1.22.2,<2.0.0 | numpy |
| >=1.4.1,<2.0.0 | pandas |
زبان مورد نیاز
| مقدار | نام |
|---|---|
| >=3.9,<4.0 | Python |
نحوه نصب
نصب پکیج whl fibberio-1.2.0:
pip install fibberio-1.2.0.whl
نصب پکیج tar.gz fibberio-1.2.0:
pip install fibberio-1.2.0.tar.gz