===================
Project description
===================
`eztest <https://github.com/lgt1001/eztest>`_ is a Python package and execution used for performance/load testing.
Home page: `https://github.com/lgt1001/eztest <https://github.com/lgt1001/eztest>`_
Features
--------
Test:
* Normal: Run selected cases only once.
* Continuous: Run cases [repeat] times with [interval] seconds' sleeping.
* Simultaneous: Start [stress] threads and run cases in each thread, sleep [interval] seconds after all cases are finished, and then start testing again with [repeat] times.
* Concurrency: Start [stress] threads and each thread will continuously run cases with [interval] seconds' sleeping.
* Frequent: Start [stress] threads per [interval] seconds. And only can have [limit] available threads running.
Report:
* Start|Stop report server, and ``eztest`` can send and save report on remote server.
* Dump failure rate and average of time taken from remote report server.
* Calculate failure rate and average of time taken for report files generated by ``eztest``.
Control:
* Stop testing and report server.
Usage
-----
``eztest`` command::
$ eztest -h
usage: eztest [-h] [--version] {test,stop,calc,server,dump} ...
eztest
positional arguments:
{test,stop,calc,server,dump}
test Start eztest for target cases, classes, modules.
stop Stop eztest and its report server.
calc Calculate report files generated by eztest.
server Start|Stop|Restart report server.
dump Dump data from report server.
optional arguments:
-h, --help show this help message and exit
--version, -v show program's version number and exit
usage: eztest [-h] [--target TARGET] [--classes CLASSES [CLASSES ...]]
[--not-classes NOT_CLASSES [NOT_CLASSES ...]]
[--cases CASES [CASES ...]]
[--not-cases NOT_CASES [NOT_CASES ...]]
[--mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}]
[--stress STRESS] [--repeat REPEAT] [--interval INTERVAL]
[--limit LIMIT] [--starts STARTS] [--duration DURATION]
[--ends ENDS] [--mail-config MAIL_CONFIG]
[--report-folder REPORT_FOLDER] [--noreport] [--nolog]
[--calc CALC [CALC ...]] [--group-minutes GROUP_MINUTES]
[--version]
``eztest test`` command::
$ eztest test -h
usage: eztest test [-h] --target TARGET [--classes CLASSES [CLASSES ...]]
[--not-classes NOT_CLASSES [NOT_CLASSES ...]]
[--cases CASES [CASES ...]]
[--not-cases NOT_CASES [NOT_CASES ...]]
[--mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}]
[--stress STRESS] [--repeat REPEAT] [--interval INTERVAL]
[--limit LIMIT] [--starts STARTS] [--duration DURATION]
[--ends ENDS] [--report-folder REPORT_FOLDER]
[--report-server REPORT_SERVER] [--noreport] [--nolog]
[--mail-config MAIL_CONFIG]
optional arguments:
-h, --help show this help message and exit
Case Group:
Define arguments of case related.
--target TARGET, -t TARGET
Folder or file path, or a module, a __init__.py file is required under that folder/module.
--classes CLASSES [CLASSES ...], -cl CLASSES [CLASSES ...]
Class names to be tested. It will be considered if target is file.
--not-classes NOT_CLASSES [NOT_CLASSES ...], -ncl NOT_CLASSES [NOT_CLASSES ...]
Class names to be ignored. It will be considered if target is file.
--cases CASES [CASES ...], -c CASES [CASES ...]
Case names to be tested.
It can be whole case name or part of them(e.g.: "*a", "a*", "*a*").
--not-cases NOT_CASES [NOT_CASES ...], -nc NOT_CASES [NOT_CASES ...]
Case names to be ignored.
It can be whole case name or part of them(e.g.: "*a", "a*", "*a*").
Test Mode Group:
Define arguments of test mode related.
--mode {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent},
-m {0,1,2,3,4,normal,continuous,simultaneous,concurrency,frequent}
(a)0 or normal: Run selected cases only once.
(b)1 or continuous: Run cases [repeat] times with [interval] seconds' sleeping.
(c)2 or simultaneous: Start [stress] threads and run cases in each thread,
sleep [interval] seconds after all cases are finished,
and then start testing again with [repeat] times.
(d)3 or concurrency: Start [stress] threads and each thread will continuously run
cases with [interval] seconds' sleeping.
(e)4 or frequent: Start [stress] threads per [interval] seconds.
And only can have [limit] available threads running.
--stress STRESS, -s STRESS
Start [stress] threads in each round of testing. Default value is 1.
--repeat REPEAT, -r REPEAT
Repeat [repeat] times of testing. Default value is 1
--interval INTERVAL, -i INTERVAL
Sleep [interval] seconds after one round of testing. Default value is 0.
--limit LIMIT, -l LIMIT
Only can have [limit] count of running threads.
No limitation if this is less than or equals to [stress].
--starts STARTS, -st STARTS
Testing will be started at [starts]. It is datetime string(e.g.: "2014-01-02 03:04:05").
--duration DURATION, -d DURATION
Testing will continue with [duration] minutes. Will be ignored if 'ends' is provided.
--ends ENDS, -et ENDS
Testing will be stopped at [ends]. It is datetime string(e.g.: "2014-01-02 03:04:05").
Report/Log Group:
Define arguments of report or log related.
--report-folder REPORT_FOLDER, -rf REPORT_FOLDER
Report and log files will be saved under [report-folder].
--report-server REPORT_SERVER, -rs REPORT_SERVER
Report server.
The format is "host_name:port_number" or "host_name" with default port number 8765.
--noreport, -nr No report file will be generated if [noreport] is clarified.
--nolog, -nl No log file will be generated if [nolog] is clarified.
--mail-config MAIL_CONFIG, -mc MAIL_CONFIG
Mail configuration file which contains mail server information etc.
It should be INI format file(http://en.wikipedia.org/wiki/INI_file).
Will send report by mail only if mail-config is provided and report file is generated.
Section is "SMTP" and properties can be "server", "from_mail", "to_mails",
"cc_mails", "bcc_mails", "username", "password", "need_authentication" and "subject".
"server", "from_mail" and "to_mails" are mandatory.
"to_mails", "cc_mails" and "bcc_mails" can be multiple values separated by comma.
"need_authentication" is boolean, "username" and "password" are required if
"need_authentication" is True.
``eztest server`` command::
$eztest server -h
usage: eztest server [-h] {start,stop} ...
positional arguments:
{start,stop}
optional arguments:
-h, --help show this help message and exit
$eztest server start -h
usage: eztest server start [-h] [--port PORT] [--handler HANDLER]
optional arguments:
-h, --help show this help message and exit
--port PORT, -p PORT Port number.
--handler HANDLER, -hl HANDLER
Custom handler.
The format is: "file_path:handler_class_name", or "module_name:handler_class_name".
``eztest dump`` command::
$eztest dump -h
usage: eztest dump [-h] [--report-server REPORT_SERVER]
optional arguments:
-h, --help show this help message and exit
--report-server REPORT_SERVER, -rs REPORT_SERVER
Report server.
The format is "host_name:port_number" or "host_name" with default port number 8765.
``eztest calc`` command::
$eztest calc -h
usage: eztest calc [-h] --path PATH [PATH ...] [--group-minutes GROUP_MINUTES]
optional arguments:
-h, --help show this help message and exit
--path PATH [PATH ...], -p PATH [PATH ...]
Report folders or files to be calculated.
--group-minutes GROUP_MINUTES, -gm GROUP_MINUTES
Calculate by grouping case results with [group-minutes] minutes. Default is 60 minutes.
Examples
--------
Test examples::
# Normal testing
$ eztest --target examples/target_is_unittest/test_case.py
# Continuous testing and repeat 100 times
$ eztest --mode continuous --target examples/target_is_unittest/test_case.py --repeat 100 --nolog
# Simultaneous testing, start 50 threads and repeat 100 times
$ eztest --mode simultaneous --target examples/target_is_unittest/test_case.py --stress 50 --repeat 100 --nolog
# Concurrency testing, start 50 threads and run 1 hour
$ eztest --mode simultaneous --target examples/target_is_unittest/test_case.py --stress 50 --duration 60 --nolog
# Frequent testing, start 50 threads and run 1 hour
$ eztest --mode frequent --target examples/target_is_unittest/test_case.py --stress 50 --duration 60 --nolog
# Ignore cases
$ eztest --target examples/target_is_unittest/test_case.py --not-cases test_hello
# Target is a module with CASES defined.
$ eztest --target examples.target_is_module
# Send and save case report to remote server.
$ eztest --target examples.target_is_module --report-server localhost:8765
# Stop testing or report server
$ eztest stop
Report related examples::
# Start report server.
$ eztest start --port 8765
# Stop report server.
$ eztest stop
# Dump testing summary from remote report server
$ eztest dump --report-server localhost:8765
# Calculate failure rate and average of time taken for report files.
$ eztest --calc "/tmp/a.csv" "/tmp/b.csv" --group-minutes 30
# Calculate failure rate and average of time taken for files under report folder.
$ eztest --calc "/tmp/reports" --group-minutes 30
Prerequisites
-------------
- C Python 2.7, 3.2 and higher.
- psutil https://pypi.org/project/psutil/
Authors
-------
lgt
License
-------
GNU GPL v2, see http://www.gnu.org/licenses/gpl-2.0.html