Elastic Git
===========
Adventures in an declarative object-y thing backed by Git and using
Elasticsearch as a query backend.
.. note:: Here be massive dragons.
.. image:: https://travis-ci.org/universalcore/elastic-git.svg?branch=develop
:target: https://travis-ci.org/universalcore/elastic-git
:alt: Continuous Integration
.. image:: https://coveralls.io/repos/universalcore/elastic-git/badge.png?branch=develop
:target: https://coveralls.io/r/universalcore/elastic-git?branch=develop
:alt: Code Coverage
.. image:: https://readthedocs.org/projects/elastic-git/badge/?version=latest
:target: https://elastic-git.readthedocs.org
:alt: Elastic-Git Documentation
Usage
-----
.. code-block:: python
from elasticgit import EG
from elasticgit.models import Model, IntegerField, TextField
workspace = EG.workspace('/Users/sdehaan/Desktop/test-repo/')
workspace.setup('Simon de Haan', 'simon@praekeltfoundation.org')
"""
# The model looks like this
class Person(Model):
age = IntegerField('The Age')
name = TextField('The Name')
"""
from elasticgit.tests.base import TestPerson as Person
person1 = Person({'age': 10, 'name': 'Foo'})
workspace.save(person1, 'Saving Person 1')
person2 = Person({'age': 20, 'name': 'Bar'})
workspace.save(person2, 'Saving Person 2')
person3 = Person({'age': 30, 'name': 'Baz'})
workspace.save(person3, 'Saving Person 3')
Data is now persisted in a git repository and is queryable via elasticsearch:
.. code-block:: python
>>> from elasticgit import EG
>>> from elasticgit.tests.base import TestPerson as Person
>>> workspace = EG.workspace('/Users/sdehaan/Desktop/test-repo/')
>>> for person in workspace.S(Person).filter(age__gte=20):
... print person.name, person.age
...
Bar 20
Baz 30
Check the ``examples/`` directory for some more code samples.
.. code-block:: bash
$ python -m examples.basic_usage
e6cb25f00870472fa5223d76dc361667 Baz 30
2bd470372243411c9abd8fdcb969dcf5 Bar 20
Schema Management
-----------------
We've followed the example of Apache Avro_ when it comes to schema evolution.
Avro compatible schemas can be generated from the command line.
Model definitions can be rebuilt from Avro_ JSON schema files.
A sample model file:
.. code-block:: python
class TestFallbackPerson(Model):
age = IntegerField('The Age')
name = TextField('The name', fallbacks=[
SingleFieldFallback('nick'),
SingleFieldFallback('obsolete'),
])
nick = TextField('The nickname', required=False)
obsolete = TextField('Some obsolete field', required=False)
Generating the Avro_ spec file
.. code-block:: bash
$ python -m elasticgit.tools dump-schema \
> elasticgit.tests.base.TestFallbackPerson > avro.json
$ python -m elasticgit.tools load-schema avro.json > models.py
The generated model file:
.. code-block:: python
# NOTE:
#
# This is an automatically generated Elasticgit Model definition
# from an Avro schema. Do not manually edit this file unless you
# absolutely know what you are doing.
#
# timestamp: 2014-10-14T18:51:23.916194
# namespace: elasticgit.tests.base
# type: record
# name: TestFallbackPerson
#
from elasticgit import models
class TestFallbackPerson(models.Model):
name = models.TextField(u"""The name""", fallbacks=[models.SingleFieldFallback('nick'),models.SingleFieldFallback('obsolete'),])
age = models.IntegerField(u"""The Age""")
obsolete = models.TextField(u"""Some obsolete field""")
_version = models.ModelVersionField(u"""Model Version Identifier""")
nick = models.TextField(u"""The nickname""")
uuid = models.TextField(u"""Unique Identifier""")
We're using ConfModel_'s fallbacks feature and encode this in Avro_'s
Schema as ``aliases``. This allows you to fall back to older names for
fields:
.. code-block:: python
>>> TestFallbackPerson({'obsolete': 'oldest name', 'age': 10}).name
'oldest name'
>>> TestFallbackPerson({'nick': 'older name', 'age': 10}).name
'older name'
>>> TestFallbackPerson({'name': 'current name', 'age': 10}).name
'current name'
.. _Avro: http://avro.apache.org/docs/1.7.7/spec.html
.. _ConfModel: http://confmodel.rtfd.org/