docker-compose addons
=====================
A set of command line tools to supplement the features already available
in docker-compose. These tools generally focus on development or testing
environment use-cases.
.. image:: https://img.shields.io/pypi/v/compose-addons.svg
:target: https://pypi.python.org/pypi/compose-addons
:alt: Latest PyPI version
.. image:: https://travis-ci.org/dnephin/compose-addons.svg?branch=master
:target: https://travis-ci.org/dnephin/compose-addons
:alt: Travis CI
.. contents::
:backlinks: none
Install
-------
Currently the only install option is pip
.. code:: sh
pip install compose-addons
dcao-include
------------
Given a docker-compose.yml file, fetch each configuration in the include
section and merge it into a base docker-compose.yml. If any of the included
files have include sections continue to fetch and merge each of them until
there are no more files to include.
Use Cases
~~~~~~~~~
- If you have a service-oriented architecture, where each of your services
is developed and deployed in a separate code repo, and each has its own
docker-compose.yml. When you want to create a full testing or development
environment for an individual service, you need to include all the
downstream services. Instead of duplicating the topology of each
downstream service, you can include the ``docker-compose.yml`` from the
downstream service. Including (instead of duplicating) this topology
allows you to change dependencies in a single place without worrying
about breaking the test suite of dependent services.
- If the scope of your composition can change based on the task you're
performing. Your application might have a "core" set of services that are
always run, and some adhoc, or administrative services that are only run
sometimes. You can split your composition into two (or more) files.
The core ``docker-compose.yml`` would only contain the core services. The
``compose-admin.yml`` would include the ``docker-compose.yml`` and add
extra services which could link to or use volumes from the core services,
without having to duplicate any of the service configuration.
- If your composition varies by environment (dev vs prod). Similar to the
case above, the core ``docker-compose.yml`` would remain the same for all
environments, but ``docker-compose-dev.yml`` could include the "core"
services, and add additional service, like database or proxies.
Working with Includes
~~~~~~~~~~~~~~~~~~~~~
``dcao-include`` works with a configuration that is different from the
``docker-compose`` config in a few ways:
- an optional top level ``include`` key, which contains a list of urls (which
may be local file paths, http(s) urls, or s3 paths)
- a required top level ``namespace`` key, which is used by a config to link
to services in an included file. For example, if a config includes
http://example.com/compositions/servicea.yaml which has a ``namespace``
of ``servicea``, all "public" services in ``servicea.yaml`` should start
with ``servicea.``.
- since configuration can be included from a remote url, or different
directories, the configuration should not include anything that depends
on the host. There should be no ``build`` keys in any service, and no
host volumes.
Example
~~~~~~~
An example composition file with includes:
.. code:: yaml
include:
- http://example.com/compositions/servicea.yaml
- http://example.com/compositions/serviceb.yaml
namespace: core
web:
image: example/service_a:latest
links: ['servicea.web', 'serviceb.api']
**servicea.yaml** might look something like this
.. code:: yaml
namespace: servicea
servicea.web:
image: services/a:latest
**serviceb.yaml** might look something like this
.. code:: yaml
namespace: serviceb
serviceb.api:
image: services/b:latest
Usage
~~~~~
To use ``dcao-include`` with ``docker-compose`` you have a couple options:
Use it with a pipe to stdin:
.. code:: sh
dcao-include compose-with-includes.yml | docker-compose -f - up -d
Use it once to generate a new file:
.. code:: sh
dcao-include -o docker-compose.yml compose-with-includes.yml
docker-compose up -d
docker-compose ps
dcao-namespace
--------------
Given a standard ``docker-compose.yml`` file, add a namespace key, and prefix
all instances of service names with that namespace. This command is used to
prepare a standard ``docker-compose.yml`` file for being used as an include
by ``dcao-include``. This could be considered the "export" step required
before a compose file can be included by another project.
Example
~~~~~~~
Given a ``docker-compose.yml``:
.. code:: yaml
web:
image: example.com/web:latest
links: ['db']
volumes_from: ['configs']
db:
image: example.com/db:latest
configs:
image: example.com/configs:latest
running ``dcao-namespace docker-compose.yml myservice`` would produce
.. code:: yaml
namespace: myservice
myservice.web:
image: example.com/web:latest
links: ['myservice.db:db']
volumes_from: ['myservice.configs']
myservice.db:
image: example.com/db:latest
myservice.configs:
image: example.com/configs:latest
Usage
~~~~~
First generate the namespaced config
.. code:: sh
dcao-namespace -o myservice.yml docker-compose.yml myservice
Next you'll want to make ``myservice.yml`` available to other services. In this
example we'll assume we're using an s3 bucket
.. code:: sh
aws s3 cp myservice.yml s3://some-bucket/compose-registry/myservice.yml
Now we can use that configuration as an include in another service. In a
different services ``compose-with-includes.yml`` (which will be consumed by
``dcao-include``)
.. code:: sh
include:
- s3://some-bucket/compose-registry/myservice.yml
dcao-merge
----------
Merge ``docker-compose.yml`` configuration files by overriding values in the
base configuration with values from other files. It is used to transform a
configuration without having to duplicate any fields that should remain
consistent.
Use Cases
~~~~~~~~~
- Often in development you'll want to include code using a volume for faster
iteration, but for testing on a CI you want to include the source in the
container with ``ADD`` (or ``COPY``). You could use an ``overrides-dev.yml`` to add
volumes to the configuration, and skip that step during CI.
- If the composition is running on a shared host each developer needs to use a
different host port. This variation can be included in a file maintained by
each developer outside of version control.
- If a ``docker-compose.yml`` contains ``build`` directives for local
development, but needs ``image`` directives in other environments (testing,
stage, prod, etc), merge can be used to rewrite ``build`` to ``image`` with
the correct image tag.
Usage
~~~~~
To rewrite a configuration to use image instead of build, and remove any host
specific configuration:
.. code:: sh
dcao-merge -o export.yml docker-compose.yml compose-overrides.yml
Where ``docker-compose.yml`` is:
.. code:: yaml
web:
build: .
links: ['db']
volumes: ['./logs:/app/logs']
db:
build: database/
and ``compose-overrides.yml``:
.. code:: yaml
web:
image: example.com/web:latest
volumes: []
db:
image: example.com/db:latest
would produce an ``export.yml``
.. code:: yaml
web:
image: example.com/web:latest
links: ['db']
volumes: []
db:
image: example.com/db:latest