Event Metrics
-------------
An embedded, event-time metric collection library built for serving system
--------------------------------------------------------------------------
Metric systems like prometheus aggregate metric at "processing time", whenever the scraper is
able to scrape the metric. ``event_metrics`` capture and record metrics at event time, when the
event happens.
Features
^^^^^^^^
Comparing to other metric system, the ``event_metrics`` library:
* Write data to sqlite3 database to keep low-memory footprint
* Minimal dependency (only requires numpy for numeric aggregation)
* Aggregate with the full data by default (no reservior sampling)
* Allow select from past duration with timedelta windowing
* Timestamp all observation by default
* Small API footprint: ``observe`` and ``query`` is all you need to know
* Compute raw timeseries with different aggregation strategies:
* Scalers: last, min, max, mean, count, sum
* Buckets for histogram
* Percentiles for summary
* Array and timestamps for native python wrangling
* Metrics can be labeled with arbitrary key value pair and querying supports
multidimensional label matching.
Install
^^^^^^^
* Install from source: ``pip install -e .``
* PyPI package is work in progress
Usage
^^^^^
.. code-block:: python
from event_metrics import MetricConnection
conn = MetricConnection("/tmp/event_metrics_demo")
conn.observe("latency", 1.2)
conn.increment("counter", -1)
# labeling
conn.observe("latency", 2.0, labels={"service": "myapp", "other": "label"})
# querying
(conn.query("latency", labels={"service":"myapp"})
# select from past duration using one of the following
.from_beginning()
.from_timestamp(...)
.from_timedelta(...)
# perform aggregation using one of the following
.to_scaler(agg="last/min/max/mean/count/sum") # counter, guage
.to_buckets(buckets=[1, 5, 10], cumulative=False) # histogram
.to_percentiles(percnetiles=[50, 90, 99]) # summary
.to_array() # -> value array
.to_timestamps() # -> timestamp array
.to_timestamps_array() # -> 2 array, (timestamp, value array)
)
Speed
^^^^^
The library is *fast enough*. It can ingest about 34,000 data point
per seconds:
You can run ``pytest tests -k bench`` to generate benchmark on local hardware:
.. code-block::
-------------------------------------------------- benchmark: 1 tests --------------------------------------------------
Name (time in us) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
------------------------------------------------------------------------------------------------------------------------
test_bench_ingestion 25.3340 297.9320 28.8541 9.1582 26.8090 0.8650 521;814 34.6571 6496 1
------------------------------------------------------------------------------------------------------------------------
