معرفی شرکت ها
dummy-file-generator-1.1.6
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
dummy csv, flat, json text file generator, typical usage scenario can be load / stress / performance testing of file-processing data tools
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | dummy-file-generator-1.1.6 |
| نام | dummy-file-generator |
| نسخه کتابخانه | 1.1.6 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | datahappy1 |
| ایمیل نویسنده | - |
| آدرس صفحه اصلی | https://github.com/datahappy1/dummy_file_generator |
| آدرس اینترنتی | https://pypi.org/project/dummy-file-generator/ |
| مجوز | - |
# dummy_file_generator
## version 1.1.21
### Dummy .csv, flat text or json files generator written in Python 3.7
This tool is able to generate dummy csv, flat text or json files based on the configuration settings you setup for your project(s).
- [How to install and run the tool as CLI](#how-to-install-and-run-the-tool-as-CLI)
- [How to install and run the tool as an imported package](#how-to-install-and-run-the-tool-as-an-imported-package)
- [How to setup a new dummy file generator project](#how-to-setup-a-new-dummy-file-generator-project)
- [How to add a new source dataset for your project](#How-to-add-a-new-source-dataset-for-your-project)
- [Developer information](#Developer-information) (for further tool development)
# How to install and run the tool as CLI
One common usage scenario can be load / stress / performance testing of file-processing data tools, allowing you to generate the files needed from a command line.
### To install:
1) `git clone https://github.com/datahappy1/dummy_file_generator c:\dummy_file_generator\`
2) Set PYTHONPATH to c:\dummy_file_generator\ [tutorial](https://bic-berkeley.github.io/psych-214-fall-2016/using_pythonpath.html)
### To run:<br />
The CLI tool needs these **MANDATORY** arguments defining:
- projectname `--projectname` or `-pn` based on the projectname, the dummy file project specific settings from `dummy_file_generator/configs/config.json` file are loaded ,
- absolutepath `--generated_file_path` or `-gp` defining the full output file path to the file you are about to generate
>Provided arguments have higher precedence than fallback values in `settings.py`
The CLI tool can further consume these **OPTIONAL** arguments defining:
- filesize `--filesize` or `-fs` defining the desired size (in kBs) of the output file
- rowcount `--rowcount` or `-rc` defining the desired row count of the output file
>Note if you do NOT specify the filesize and do NOT specify the rowcount, the default `row_count` value from
`settings.py` will be used ( or the value you provide in the `default_rowcount` optional argument)
The CLI tool also supports these **OPTIONAL** arguments that can be used to override values in `settings.py`:
- logging_level `--logging_level` or `-ll` defining the Python logging level
- default_rowcount `--default_rowcount` or `-drc` defining the rowcount fallback value when neither row_count,neither file_size set
- file_encoding `--file_encoding` or `-fen` defining the generated files encoding
- file_line_ending `--file_line_ending` or `-fle` defining the file line ending
These two **OPTIONAL** arguments are typically needed when running the tool as an imported package, but you can use them even with
this tool running as CLI:
- data_files_location `--data_files_location` or `-dfl` defining the path to the source .txt data files
- config_json_path `--config_json_path` or `-cjp` defining the custom path to your config.json file
#### Example how to run the tool with the -fs argument to set the desired filesize of 256 kB :
1) `cd c:\dummy_file_generator\dummy_file_generator`
2) `python c:\dummy_file_generator\dummy_file_generator\__main__.py -pn dummy1 -gp c:\myfiles\dummy1file.csv -fs 256`
#### Example how to run the tool with the -rc argument to set the desired rowcount of 1000 rows :
1) `cd c:\dummy_file_generator\dummy_file_generator`
2) `python c:\dummy_file_generator\dummy_file_generator\__main__.py -pn dummy1 -gp c:\myfiles\dummy1file.csv -rc 1000`
# How to install and run the tool as an imported package
One common usage scenario can be load / stress / performance testing of file-processing data tools, where you can generate dummy text files during the test fixtures / setup.
### To install:
1) `pip install dummy-file-generator`
>You are strongly encouraged to use the Python virtual environment or Pipenv
### To run:<br />
The dummy file generator imported package needs these **MANDATORY** arguments defining:
- projectname `--projectname` or `-pn`, based on the project name, the dummy file specific settings from `config.json` file are loaded
- generated_file_path `--generated_file_path` or `gp` defining the full output file path to the file you are about to generate
>Provided arguments have higher precedence than fallback values in `settings.py`
The dummy file generator imported package can further consume these **OPTIONAL** arguments defining:
- filesize `--filesize` or `-fs` defining the desired size (in kBs) of the output file
- rowcount `--rowcount` or `-rc` defining the desired row count of the output file
>Note if you do NOT specify the filesize and do NOT specify the rowcount, the `DEFAULT_ROW_COUNT` value from
`settings.py` will be used ( you can override the `DEFAULT_ROW_COUNT` value in `settings.py` using the `default_rowcount` optional argument)
- data_files_location `--data_files_location` or `-dfl` defining the path to the source .txt data files
- config_json_path `--config_json_path` or `-cjp` defining the custom path to your `config.json` file
- logging_level `--logging_level` or `-ll` defining the Python logging level
- default_rowcount `--default_rowcount` or `-drc` defining the rowcount fallback value when neither row_count,neither file_size set
- file_encoding `--file_encoding` or `-fen` defining the generated files encoding
- file_line_ending `--file_line_ending` or `-fle` defining the file line ending
>In the example below, `project_scope_kwargs` arguments `project_name`, `data_files_location`, `config_json_path` and `default_rowcount` are used
to instantiate a DummyFileGenerator class instance.
`file_scope_kwargs` arguments `generated_file_path`, `file_size`, `file_encoding` and `file_line_ending` are used to setup the generated file properties.
Once there is a instance of DummyFileGenerator, you can use it to generate as many files as needed while only using
the `write_output_file` method and it's specific `file_scope_kwargs` arguments
#### Example how to run :
```
from dummy_file_generator import DummyFileGenerator as Dfg, DummyFileGeneratorException
logging_level = "INFO"
project_scope_kwargs = {
"project_name": "dummy1",
"data_files_location": "c:\\dfg_files\my_data_files",
"config_json_path": "c:\\dfg_files\my_configs\config.json",
"default_rowcount": None,
}
try:
dfg = Dfg(logging_level, **project_scope_kwargs)
except DummyFileGeneratorException as DFG_ERR:
raise DFG_ERR
file_scope_kwargs = {
"generated_file_path": "C:\dfg\\bin\\file1.csv",
"file_size": 1024,
#"row_count": 1000,
"file_encoding": "utf8",
"file_line_ending": "\n",
}
try:
dfg.write_output_file(**file_scope_kwargs)
except DummyFileGeneratorException as DFG_ERR:
raise DFG_ERR
```
# How to setup a new dummy file generator project
You need to generate dummy files based on the content of the text files in your `data_files` folder, and these source text files need to have this plain text format:
This tool picks random item from each of the files configured for your project in config.json and uses these values to populate the data for "columns" for each written row.
### - How to generate a .csv file
If you need to generate a dummy .csv file containing 3 columns for Names, Dates and IDs,
the project JSON object in your config.json would need to be setup like:
{
"project_name":"dummy1",
"file_type":"csv",
"header":true,
"csv_value_separator": ",",
"csv_quoting": "ALL",
"csv_quote_char": "'",
"csv_escape_char": "\\",
"columns":[
{
"column_name":"Name",
"datafile":"first_names.txt"
},
{
"column_name":"Date",
"datafile":"dates.txt"
},
{
"column_name":"ID",
"datafile":"ids.txt"
}
]
}
This configuration generates a file like this sample:
'Name','Date','ID'
'Hank','2004-05-22','23432'
'Joe','2000-03-12','445'
### - How to generate a .txt flat file:
If you need to generate a dummy .txt flat file containing 3 columns for Names, Dates and IDs with specific column lengths defined,
the "project" JSON object in your config.json would need to be setup like:
{
"project_name":"dummy2",
"file_type":"flat",
"header":true,
"columns":[
{
"column_name":"Name",
"column_len":10,
"datafile":"first_names.txt"
},
{
"column_name":"Date",
"column_len":12,
"datafile":"dates.txt"
},
{
"column_name":"ID",
"column_len":9,
"datafile":"ids.txt"
}
]
}
This configuration generates a file like this sample:
Name Date ID
Hank 2004-05-22 23432
Joe 2000-03-12 445
### - How to generate a .json file:
If you need to generate a dummy .json file containing 3 columns for Names, Dates and IDs,
the "project" JSON object in your config.json would need to be setup like:
{
"project_name":"dummy3",
"file_type":"json",
"columns":[
{
"column_name":"Name",
"datafile":"first_names.txt"
},
{
"column_name":"Date",
"datafile":"dates.txt"
},
{
"column_name":"ID",
"datafile":"ids.txt"
}
]
}
This configuration generates a file like this sample:
[{"Name": "Hank", "Date": "2004-05-22", "ID": "23432"},
{"Name": "Joe", "Date": "2000-03-12", "ID": "445"}]
If you need to generate a more complex dummy .json file containing 3 columns for Names, Dates, IDs
and an array-like column Identifiers containing one IDs array element and an object containing ID1 and ID2 attributes,
the "project" JSON object in your config.json would need to be setup like:
{
"project_name": "dummy4",
"file_type": "json",
"columns": [
{
"column_name": "Name",
"datafile": "first_names.txt"
},
{
"column_name": "Date",
"datafile": "dates.txt"
},
{
"column_name": "ID",
"datafile": "ids.txt"
},
{
"column_name": "Identifiers",
"__array_columns": [
{
"datafile": "ids.txt"
},
{
"columns": [
{
"column_name": "ID1",
"datafile": "ids.txt"
},
{
"column_name": "ID2",
"datafile": "ids.txt"
}
]
}
]
}
]
}
This configuration generates a file like this sample:
[{"Name": "Hank", "Date": "2004-05-22", "ID": "23432", "Identifiers": ["445", {"ID1": "11111", "ID2": "145546566345"}]},
{"Name": "Joe", "Date": "2000-03-12", "ID": "445", "Identifiers": ["11111", {"ID1": "145546566345", "ID2": "156765"}]}]
>JSON file configuration allows only one level deep nested objects, that have to be defined in the __array_columns array
# How to add a new source dataset for your project
Whenever you need to add a new source .txt file in the data_files folder, just add it to your `data_files` folder.
**The filename needs to correspond with the datafile value in your config.json file.**
If running as a standalone CLI tool, the `data_files` folder is located here:
`dummy_file_generator/data_files`
When running as an imported package, the `data_files` folder is where ever you specify it to be
using the argument `data_files_location`.
Now you can use this new data file in your project setup in `config.json` file.
# Developer information
## testing using Pytest
Pytest unit and performance tests are also a part of this repository.
You can install Pytest using `pip install pytest`
### To run tests:
1) `cd c:\dummy_file_generator\dummy_file_generator`
2) `python -m pytest c:\dummy_file_generator\tests`
( In case when running from IDE, make sure the current working dir is set to `c:\\dummy_file_generator`)
نحوه نصب
نصب پکیج whl dummy-file-generator-1.1.6:
pip install dummy-file-generator-1.1.6.whl
نصب پکیج tar.gz dummy-file-generator-1.1.6:
pip install dummy-file-generator-1.1.6.tar.gz