معرفی شرکت ها
es-logger-6.5
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
Framework for Creating Logstash events from Jenkins Jobs
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | es-logger-6.5 |
| نام | es-logger |
| نسخه کتابخانه | 6.5 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | JP Sullivan (j3p0uk) |
| ایمیل نویسنده | jonpsull@cisco.com |
| آدرس صفحه اصلی | https://github.com/CiscoDevNet/es-logger |
| آدرس اینترنتی | https://pypi.org/project/es-logger/ |
| مجوز | - |
ES-Logger
=========
 [](https://pypi.org/project/es-logger/) [](https://gitter.im/es-logger)
The es-logger project intends to build a pluggable data processor that will take data from
Jenkins builds and push that data into Elasticsearch via Logstash. Overview slides are available
on slideshare in [Pipeline Analytics](https://www.slideshare.net/JonPaulSullivan/pipeline-analytics)
# Installation and Running
Install the framework using:
python ./setup.py install
Install from source for developing using:
python ./setup.py develop
Install from pypi repository with:
pip install es-logger
After that, set environment variables needed, and just execute es-logger!
## Jenkins Connection
* JENKINS_URL - Used to prefix the location to gather Jenkins data from using the
[python-jenkins](https://python-jenkins.readthedocs.io/en/latest/) library
* JENKINS_USER - Username to access Jenkins
* JENKINS_PASSWORD - Password to access Jenkins. Can also be an API key
## Job Information
* ES_JOB_NAME - The full job name with all paths, added to the JENKINS_URL
* ES_BUILD_NUMBER - The build number to collect data for
## Plugins
* PROCESS_CONSOLE_LOGS - Console log processor plugins to run, space separated list
* GATHER_BUILD_DATA - Build data gatherer plugins to run, space separated list
* GENERATE_EVENTS - Event generator plugins to run, space separated list
# Plugins
The plugin infrastructure is provided by the
[stevedore library](https://pypi.python.org/pypi/stevedore).
There are following types of plugin supported:
## Console Log Processors
These plugins will have a *process* function called that is passed the full console log
of the job. It can perform any actions desirable, and shall return data, which will be
added to the es_info['console_log_plugins'][**plugin_name**] structure that is pushed to
logstash. A list or a dict is preferred as a return type.
## Build Data Gatherers
These plugins will have a *gather* function that is passed the EsCollector object. This
should provide any data needed for the plugin to gather any additional data that should
be added to the es_info. This could include test results from an external system not
available to Jenkins, parsed logs from an external piece of hardware, etc.
The return should be a dictionary, which will be added to the
es_info['build_data'][**plugin_name**] structure that is pushed to logstash.
## Event Targets
Once the data is collected by Es-Logger and constructed into a series of events,
these events need to be sent to a target. A target is any location that will accept
json structured events.
### Logstash Target
The logstash target
* LOGSTASH_SERVER - HTTP server configured to receive json messages, as per the sample
configuration
* LS_USER - User to connect to Logstash with
* LS_PASSWORD - Password to connect to Logstash with
### AWS SQS Target
* Credentials - Uses boto3, so ensure you have credentials set appropriately
See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html
* SQS_QUEUE - Name of the queue to send the data into
## Event Generators
An event generator is intended to process the Jenkins information and generate a number of
events based off that data. This allows for the creation of unique events per-host or
per-test if the Jenkins job operates on multiple resources. The structures returned are
expected to be very small, so surfacing data points to form the basis of visualisations. A
small number of default fields will be added by the main program to identify the origination
of the data, and the event data will become contained in a top-level key named after the plugin.
The return from the generator should be a list of events that will be posted to logstash
# Example Execution
Here is a sample execution of es-logger against a public Jenkins repo.
It uses the public Jenkins that Mono Project builds code upon at
[jenkins.mono-project.com](https://jenkins.mono-project.com/job/test-mono-mainline-linux/label=debian-9-amd64/)
```
export JENKINS_URL=https://jenkins.mono-project.com/
export ES_JOB_NAME=test-mono-mainline-linux
export GENERATE_EVENTS="junit commit"
ES_BUILD_NUMBER=5308/label=debian-9-amd64 es-logger --no-post -c 100
```
Each execution can get a single jobs data. To iterate over many, a simple bash loop can
suffice, or you can import es-logger into a python script and use it directly, similarly
to how it is used in [es\_logger/cli.py](es_logger/cli.py).
```
for i in $(seq 110 116)
do
ES_BUILD_NUMBER=${i} es-logger --no-post -c 100
done
```
```
usage: es-logger [-h] [--no-dump | --no-post] [-c CONSOLE_LENGTH] [-e] [-p]
[-t TARGET] [--debug]
Read data from a completed Jenkins job and push it to a logstash instance.
Behaviour is controlled through a number of environment variables as follows:
What data to gather:
PROCESS_CONSOLE_LOGS Which ConsoleLogProcessor plugins to use in processing
GATHER_BUILD_DATA Which GatherBuildData plugins to use in processing
GENERATE_EVENTS Which EventGenerator plugins to use in processing
Where to gather data from:
JENKINS_URL The url to access Jenkins at
JENKINS_USER The username for Jenkins access
JENKINS_PASSWORD The password or API token for Jenkins access
What to gather data from:
ES_JOB_NAME The "Full Project Name" style job name for the job to process
ES_BUILD_NUMBER The build number for the job to process
Target Variables:
Logstash Target Environment Variables:
LOGSTASH_SERVER The server to send events to
LS_USER The user for logstash access
LS_PASSWORD The password for logstash access
optional arguments:
-h, --help show this help message and exit
--no-dump Do not dump events to the console
--no-post Do not post events to any targets
-c CONSOLE_LENGTH, --console-length CONSOLE_LENGTH
Restrict the console length in the event to this
number of characters
-e, --events-only Do not dump or post the main job event, only events
from EventGenerator plugins
-p, --list-plugins List all plugins available
-t TARGET, --target TARGET
A target to send events to
--debug Print debug logs to console during execution
```
نحوه نصب
نصب پکیج whl es-logger-6.5:
pip install es-logger-6.5.whl
نصب پکیج tar.gz es-logger-6.5:
pip install es-logger-6.5.tar.gz