Checkoutmanager
===============
Makes bzr/hg/git/svn checkouts in several places according to a
``.checkoutmanager.cfg`` config file (in your home directory).
The advantage: you've got one command with which you can update all your
checkouts. And with which you can ask for a list of uncommitted changes. And
you can rebuild your entire checkout structure on a new machine just by
copying the config file (this was actually the purpose I build it for: I had
to change laptops when I switched jobs...).
Checkoutmanager works on linux, osx and windows.
Starting to use it
------------------
Starting is easy. Just ``pip install checkoutmanager`` and run
``checkoutmanager``.
- The first time, you'll get a sample configuration you can copy to
``.checkoutmanager.cfg`` in your home directory.
- The second time, you'll get a usage message. (You'll want to do
``checkoutmanager co`` to grab your initial checkouts).
Generic usage
-------------
What I normally do every morning when I get to work is ``checkoutmanager
up``. This grabs the latest versions of all my checkouts from the server(s).
So an ``svn up`` for my subversion checkouts, a ``hg pull -u`` for mercurial
and so on.
From time to time, I'll do a ``checkoutmanager st`` to show if I've got some
uncommitted files lying around somewhere. Very handy if you've worked in
several directories throughout the day: it prevents you from forgetting to
check in that one bugfix for a whole week.
A new project means I add a single line to my config file and run
``checkoutmanager co``.
Checkoutmanager allows you to spread your checkouts over multiple
directories. It cannot mix version control systems per directory, however.
As an example, I've got a ``~/buildout/`` directory with my big svn website
projects checked out there. And a directory with my svn work python
libraries. And a ``~/hg/`` dir with my mercurial projects. And I've made
checkouts of several config directories in my home dir, such as
``~/.emacs.d``, ``~/.subversion`` and so on. Works just fine.
Commands
--------
Available commands:
exists
Print whether checkouts are present or missing
up
Grab latest version from the server.
st
Print status of files in the checkouts
co
Grab missing checkouts from the server
missing
Print directories that are missing from the config file
out
Show changesets you haven't pushed to the server yet
in
Show incoming changesets that would be pulled in with 'up'. For some
version control systems, this depends on the English output of the
respective commands and is therefore inherently fragile.
Hidden commands
---------------
A few commands are hidden because they are seldom used and are only
useful for subversion.
upgrade
This upgrades the working copy to the new subversion 1.7 layout of
the .svn directory. This should be done once after you have
upgraded your subversion to 1.7. Note that when you accidentally
run this twice you get an error, but nothing breaks. Since this
command is so rarely needed, it is not advertised in the command
line help.
info
Display the svn info for the remote url. This is useful when your
svn program has been updated and the security mechanisms on your OS
now require you to explictly allow access to the stored credentials.
The other commands either do not access the internet or are
non-interactive (like command up). In fact, the reason for adding
this command is that a non-interactive 'svn update' will fail when
you have not granted access to your credentials yet for this new svn
program. This has happened a bit too often for me (Maurits).
Output directory naming
-----------------------
If you don't specify an output directory name for your checkout url, it just
takes the last part. To make life easier, we do have some adjustments we
make:
- ``https://xxx/yyy/product/trunk`` becomes ``product`` instead of
``trunk``. (Handy for subversion).
- ``https://xxx/yyy/product/branches/experiment`` becomes
``product_experiment`` instead of ``experiment`` (Handy for subversion).
- ``https://xxx/customername/buildout/trunk`` becomes ``customername``
instead of "trunk" or "buildout". (Old convention we still support).
- Bzr checkouts that start with "lp:" (special launchpad urls) get their "lp:"
stripped.
- Git checkouts lose the ".git" at the end of the url.
- If you want to preserve the directory configuration of your version control
system, add the ``preserve_tree`` option to a group. It should contain one
or more base checkout urls (one per line). If the checkout url starts with
one of the ``preserve_tree`` urls, the folder structure after it is
preserved.
With a ``preserve_tree`` of ``https://github.com``,
``https://github.com/reinout/checkoutmanager`` becomes
``reinout/checkoutmanager`` instead of ``checkoutmanager``. Also handy for
subversion, which often has nested directories.
If the ``preserve_tree`` base url isn't found, the standard rules are used,
so you won't get an error.
If you want different behaviour from the defaults above, just specify a
directory name (separated by a space) in the configuration file after the
url. So ``https://github.com/reinout/checkoutmanager bla_bla`` becomes
``bla_bla`` instead of ``checkoutmanager``
Custom commands
---------------
You can write your own custom commands. To do that you need to create a Python
package and register an entry point in your ``setup.py`` for the
``checkoutmanager.custom_actions`` target.
A ``test`` command is included with ``checkoutmanager`` and can serve as an
example. It is registered like this in checkoutmanager's own ``setup.py``:
.. code:: python
entry_points={
'checkoutmanager.custom_actions': [
'test = checkoutmanager.tests.custom_actions:test_action'
]
}
The entry point function must take one positional argument which will be the
``checkoutmanager.dirinfo.DirInfo`` instance associated with the directroy
for which the command is being executed. The function can also take optional
keyword arguments. See ``checkoutmanager.tests.custom_actions.test_action`` for
an example.
Arguments are passed to the custom command using the following syntax:
.. code:: bash
checkoutmanager action:arg1=val1,arg2=val2
Config file
-----------
.. Comment: the config file is included into the long description by setup.py,
it is in checkoutmanager/sample.cfg!
Sample configuration file::
# Sample config file. Should be placed as .checkoutmanager.cfg in your home
# directory.
#
# There are different sections per base location and version control
# system.
#
# ``checkoutmanager co`` checks them all out (or clones them).
# ``checkoutmanager up`` updates them all.
# ``checkoutmanager st`` to see if there are uncommitted changes.
# ``checkoutmanager out`` to see if there are unpushed git/hg commits.
[git-example]
vcs = git
basedir = ~/example/git
checkouts =
https://github.com/reinout/checkoutmanager
git@github.com:django/django.git
[recipes]
# Buildout recipes I work on.
vcs = svn
basedir = ~/example/svn
checkouts =
http://svn.zope.org/repos/main/z3c.recipe.usercrontab/trunk
[hg-example]
vcs = hg
basedir = ~/example/utilities
checkouts =
https://bitbucket.org/reinout/eolfixer
https://bitbucket.org/reinout/createcoverage
# [dotfolders]
# # Advanced usage!
# # Folders that end up as dotted configfolders in my home dir.
# # Note that there's a directory name behind the repository
# # location, separated by a space.
# vcs = bzr
# basedir = ~
# checkouts =
# lp:emacsconfig/trunk .emacs.d
# sftp://somewhere/subversion/trunk .subversion
# # By ignoring everything, we do not find missing import files but also
# # don't get warnings for every subdirectory in our home dir
# ignore =
# *
# .*
TODO
====
- Wait for feedback so that I can improve the documentation.
- Perhaps make a better sample config (one that actually works instead of the
current one that's structured for benefit of the automated tests).
- Modernize test setup (pytest instead of zope's testrunner + doctests).
Credits
=======
- Created by `Reinout van Rees <http://reinout.vanrees.org>`_.
- "out" command by Dmitrii Miliaev.
- Git support by Robert Kern.
- Globbing support for the ignores by Patrick Gerken.
- Custom commands support by Rafael Oliveira.
- Parallelism code by Morten Lied Johansen.
Source code is on github at https://github.com/reinout/checkoutmanager .
Bugs and feature requests can be reported at
https://github.com/reinout/checkoutmanager/issues .
Changelog of checkoutmanager
============================
2.7 (2021-09-28)
----------------
- More robust error handling.
[mortenlj]
2.6.1 (2019-09-23)
------------------
- Fixed small, but essential, README error.
2.6 (2019-09-10)
----------------
- Updated the setup (mostly: buildout version pins) so that the project can be
developed/tested again.
- The ``exists`` and ``co`` command used to check only if a directory
existed. Now it also checks if the dot directory (``.git``, ``.svn``)
exists. This way an empty directory also will get filled with a checkout.
2.5 (2016-11-07)
----------------
- Fix #19: sometimes git remote changes were seen where there were none.
[reinout]
2.4.1 (2015-09-10)
------------------
- Bugfix for the 2.4-introduced ``run_one()`` function.
[chintal]
2.4 (2015-09-09)
----------------
- Added ``in`` command that reports incoming changes (so: the changes you'd
get by running ``checkoutmanager up``). Due to differences between versions
of git/svn/hg/bzr, the reporting might not be entirely accurate. It is
*very* hard to get right. So: please `report an issue
<https://github.com/reinout/checkoutmanager/issues>`_ if something is not
quite right.
[chintal]
- Added better support for using checkoutmanager as a library. Provided you
first load a config file, you can now programmatically run actions on
individual directories or urls. See the source code for the
``checkoutmanager.runner.run_one()`` function.
[chintal]
2.3 (2015-09-08)
----------------
- Added a preserve_tree option to config files to allow structured
checkouts mirroring the repository tree.
[chintal]
2.2 (2015-08-24)
----------------
- Checkoutmanager now also runs **on python 3**!
[reinout]
- Moved from bitbucket (https://bitbucket.org/reinout/checkoutmanager) to
github (https://github.com/reinout/checkoutmanager).
[reinout]
2.1 (2015-08-18)
----------------
- Fixed ``missing`` command: do not swallow the output when
looking for not yet checked out items. Fixes issue #24.
[maurits]
2.0 (2015-03-25)
----------------
- Huge speed increase because commands are now run in parallel instead of
sequentially. Great fix by Morten Lied Johansen. For me, "checkoutmanager
up" now takes 19 seconds instead of 105 seconds!
1.17 (2015-02-06)
-----------------
- Added support for custom commands: now you can write an extension for
checkoutmanager so that you can run ``checkoutmanager
your_custom_command``. See the README for documentation. Patch by Rafael
Oliveira.
1.16 (2015-01-02)
-----------------
- Added globbing support for ignores.
1.15 (2013-09-27)
-----------------
- Handle corner case in determining directory name for a git clone.
1.14 (2013-08-12)
-----------------
- Added ``--force-interactive`` to ``svn info`` for svn version 1.8
and higher. This is for the "hidden" ``instancemanager info``
command that is handy for updating your repositories when you've
switched svn versions. (See the changelog entry for 1.10). Patch by
Maurits.
1.13 (2012-07-20)
-----------------
- Not using the sample config file as the test config file anymore. This means
there's a much nicer and more useful sample config file now.
(Thanks Craig Blaszczyk for his pull request that was the basis for this!)
1.12 (2012-04-14)
-----------------
- For bzr, the "out" command uses the exit code instead of the command output
now. This is more reliable and comfortable. Fix by Jendrik Seipp, thanks!
1.11 (2012-03-20)
-----------------
- Allow more than one vcs in a directory. This was already possible
before, but now known you no longer need to list all the checkouts
of the competing vcs in the ignore option. Also, items that are
ignored in one section are now also ignored in other sections for
the same directory.
Fixes #11.
[maurits]
1.10 (2012-01-16)
-----------------
- Using --mine-only option to ``bzr missing`` to only show our outgoing
changesets when running checkoutmanager's "out" command for bzr.
- Copying sample .cfg file if it doesn't exist instead of only suggesting the
copy. Fixes #12.
- Added hidden info command. Should be only useful for subversion if
your svn program is updated and your OS requires you to give svn
access to your stored credentials again, for each repository.
[maurits]
1.9 (2011-11-08)
----------------
- Added ``upgrade`` command that upgrades your subversion checkouts to
the new 1.7 layout of the ``.svn`` directory.
[maurits]
1.8 (2011-10-13)
----------------
- Using ``git push --dry-run`` now to detect not-yet-pushed outgoing changes
with ``checkoutmanager out``. Fixes #9 (reported by Maurits van Rees).
1.7 (2011-10-06)
----------------
- Added --configfile option. Useful when you want to use checkoutmanager to
manage checkouts for something else than your regular development projects.
In practice: I want to use it for an 'sdistmaker' that works with git.
1.6 (2010-12-27)
----------------
- Full fix for #7: checkoutmanager doesn't stop on the first error, but
continues. And it reports all errors afterwards. This helps when just one
of your svn/hg/whatever servers is down: the rest will just keep working.
- Partial fix for #7: ``svn up`` runs with ``--non-interactive`` now, so
conflict errors errors are reported instead of pretty much silently waiting
for interactive input that will never come.
1.5 (2010-09-14)
----------------
- Using ``except CommandError, e`` instead of ``except CommandError as e`` for
python2.4 compatibility.
1.4 (2010-08-17)
----------------
- Added git support (patch by Robert Kern: thanks!) Fixes issue #6.
1.3 (2010-08-09)
----------------
- Added new "out" action that shows changesets not found in the default push
location of a repository for a dvcs (hg, bzr). The action doesn't make
sense for svn, so it is ignored for svn checkouts. Fixes issue #1. Thanks
Dmitrii Miliaev for this fix!
1.2.1 (2010-08-06)
------------------
- Bugfix: when reporting an error, the os.getcwd method itself would get
printed instead of the *output* of os.getcwd()...
1.2 (2010-08-04)
----------------
- If the config file doesn't exist, just print the config file hints instead
of the generic usage info.
- Fixed issue #4: the generic 'buildout' name is stripped from the path.
svn://somewhere/customername/buildout/trunk is a common pattern.
- Added -v option that prints the commands and the directory where you execute
them. Fixes issue #3.
- Reporting on not yet checked out items when running "checkoutmanager
missing". Fixes issue #2.
- Checking return code from executed commands. On error, the command and
working directory is printed and also the output. And the script stops
right away. Fixes #5.
- Updated the documentation, for instance by mentioning the config file name
and location.
1.1 (2010-08-02)
----------------
- Switched from "commands" module to "subprocesses" for windows
compatibility.
1.0 (2010-08-01)
----------------
- Small fixes. It works great in practice.
- Moved from bzr to hg and made it public on bitbucket.org.
- Big documentation update as I'm going to release it.
0.1 (2010-05-07)
----------------
- First reasonably working version.
- Initial library skeleton created by thaskel.