معرفی شرکت ها
confect-0.4.3
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
a Python configuration library loads Python configuration files
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | confect-0.4.3 |
| نام | confect |
| نسخه کتابخانه | 0.4.3 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | Yen, Tzu-Hsi |
| ایمیل نویسنده | joseph.yen@gmail.com |
| آدرس صفحه اصلی | https://github.com/d2207197/confect |
| آدرس اینترنتی | https://pypi.org/project/confect/ |
| مجوز | Apache-2.0 |
Confect - a Python configuration library loads Python configuration files
**************************************************************************
Introduction
==============
Why you need a configuration library?
-------------------------------------
- **For storing secrets**
You have a project that needs to access database or other services with password or some secret keys.
Storing secrets and passwords in your code is not smart.
You need a configuration file and a library for loading and using it in the runtime.
- **For different runtime environments**
For example, database IP addresses and passwords in development environment normally differs from production environment.
You need multiple configuration files for storing those information for different environment, and load one of them in the run time.
- **For better parameter management**
You're running some experiments, e.g. working on Machine Learning projects.
There're a bunch of parameters needs to be changed in the run time.
And you want to manage them in a smarter and more elegant way.
How confect differs from others?
-------------------------------------
- **Python configuration files**
This makes it possible to
+ have complex type objects as configuration values, like Decimal, timedelta
or any class instance
+ dynamically handle complicated logic, you can use conditional statements
like ``if`` in it.
+ read other TOML/YMAL/JSON/ini files or even environment variables in the
configuration file.
- **Loads configuration file through module importing**
Confect loads configuration file through a given file path, or through module importing.
It's easy to control the source of configuration file through ``PYTHONPATH``.
- **Loads configuration file multiple times**
Sometimes we need multiple configuration files — one for project,
one for team and one for personal use.
And we want that the personal configuration file has the highest priority.
If there's a configuration setting existing in that file, it would override values
from other files.
- **Loads configuration properties from environment variable**
This feature is convenient if you want to change a single or some properties values,
and don't want to modify the configuration file.
- **Attachs command line options to some click_ command**
You can change any configuration value through command line options, if your command is created by click_.
- **Better maintainability**
Confect forces users to define configuration properties and set a default value before using them.
And the ``conf`` object is immutable for reducing the possibility of making errors.
Install
========
``confect`` is a Python package hosted on PyPI and works only with Python 3.6 up.
Just like other Python packages, install it by pip_ into a virtualenv_
, or use poetry_ to manage project dependencies and virtualenv.
.. code:: console
$ pip install confect
Basics
===========
Conf Object
-----------
Calling ``conf = confect.Conf()`` creates a new configuration manager object.
Suppose ``projx`` is your top-level package name. Put the following lines into
``projx/core.py`` or ``projx/confspec.py``
.. code:: python
import confect
conf = confect.Conf()
It is possible to create multiple ``Conf`` objects, but normally it's not what
you want. In most cases, initialize and manage only one ``Conf`` object in your
application, then import and use it anywhere.
Configuration Properties Declaration
------------------------------------
**Configuration properties should be declared before using it.** This feature
makes your code more readable and maintainable.
Two ways to declare properties.
1. context manager:
.. code:: python
with conf.declare_group(group_name) as group_name:
group_name.prop1 = 'default value'
group_name.prop2 = 42
2. function call
.. code:: python
conf.declare_group(group_name, prop1='default value', prop2=42)
Group names and property names should be valid Python variable names, which
consist of letters (A-Z, a-z), digits (0-9), and the underscore character (_).
Normally, the group name is your class name, module name or subpackage name.
Default Value and Parser
^^^^^^^^^^^^^^^^^^^^^^^^^
Default values of all properties should be defined along with the declaration.
Use ``confect.prop(default, desc=None, prop_type=None)`` to specify details other than the
default value. ``desc`` is for commentary and the help message in CLI option.
Argument of ``prop_type`` is an instance of confect.PropertyType which is
responsable for CLI argument and environment variable parsing. ``prop_type`` of
popular Python types would be infered from default value automatically.
Default values don't have to be a workable value (e.g. fake secret keys or
passwords). The true workable value can be defined in the configuration file.
However, even if it's not a workable value, the mock default values still make
the declaration and the code more readable and maintainable. For instance:
.. code:: python
with conf.declare_group('aws') as aws:
aws.access_key_id = 'true-access-key'
aws.secret_access_key = 'fake-key-plz-set-it-in-local_conf.py'
Declaration Example
^^^^^^^^^^^^^^^^^^^^^
.. code:: python
import confect
conf = confect.Conf()
# declare properties with context manager
with conf.declare_group('api') as api:
# default value only. confect would infer property type automatically
api.cache_prefix = 'projx_cache'
api.cache_expire = confect.prop(
default=60 * 60 * 24,
desc="expire time in seconds")
# add description for CLI help message and commentary
api.url_base_path = confect.prop(
default='api/v2/',
desc='URL base path of API')
with conf.declare_group('db') as db:
db.host = '127.0.0.1'
db.db_name = 'projx'
db.username = 'projx_admin'
# if default value has to be None, it'd be better to assign property
# type manually for parsing
db.password = confect.prop(
default=None,
prop_type=confect.prop_type.String(),
desc='`None` for no password')
db.port = confect.prop(
default=None,
prop_type=confect.prop_type.Integer(),
desc='`None` for db engine default port')
# declare properties with function call
conf.declare_group(
'ctr_predict_model',
model_pickle_s3folder='s3://some-bucket/path/to/folder',
model_version=confect.prop(default='v3')
)
Declaration Location
^^^^^^^^^^^^^^^^^^^^^
Property declarations can be put into the module where the ``conf`` object is
located. Or, you can put them into those modules where you need these
configurations, like ``projx/db.py`` or ``projx/api.py``. Just make sure your
application import all these modules eagerly, not lazily.
Configuration Access
--------------------
After the group and properties are declared, they are accessable through
the ``conf`` object directly, like ``conf.group_name.prop_name``.
``projx/api.py``
.. code:: python
from projx.core import conf
@routes(conf.api.url_base_path + 'add')
@redis_cache(key=conf.api.cache_prefix, expire=conf.api.cache_expire)
def add(a, b)
return a + b
``projx/db.py``
.. code:: python
from projx.core import conf
engine = create_engine(
f'mysql://{conf.db.username}:{conf.db.password}'
f'@{conf.db.host}/{conf.db.db_name}')
Access Errors
^^^^^^^^^^^^^^^^^^^^^^^^^
Make sure that the configuration properties are **declared before access**. If not,
exceptions would be raised.
.. code:: python
>>> conf.unknown_group.unknown_prop
Traceback (most recent call last):
...
UnknownConfError: "Unknown configuration group 'unknown_group'"
.. code:: python
>>> conf.api.unknown_prop
Traceback (most recent call last):
...
UnknownConfError: "Unknown 'unknown_prop' property in configuration group 'api'"
**Configuration properties and groups are immutable.** They are meant to be
altered globally by loading configuration files, environment variables or CLI
argument.
.. code:: python
>>> conf.api.cache_expire = 60 * 60 * 3
Traceback (most recent call last):
...
confect.error.FrozenConfPropError: Configuration properties are frozen.
Setting Configuration Properties
---------------------------------
Configuration properties are immutable in the application runtime. This feature make sure
the runtime environment is stable without unexpected behavior.
The standard ways to change the configuration properties are:
1. Load from Python file ``conf.load_module(module_name)`` and ``conf.load_file(file_path)``. (Check `Loading Configuration File`_)
2. Load from environment variable ``conf.load_envvar(prefix)``. (Check `Loading Environment Variables`_)
3. Override by CLI options ``conf.click_options(click_command)``. (Check `Command Line Options`_)
Confect still provide a hacky way to change them in the runtime, but use them wisely.
1. Alter configuration in the runtime(Check `Runtime Configuration Altering`_)
Loading Configuration File
--------------------------
Confect loads Python configuration files. That makes your configuration file
programmable and unrestricted as we described in the section `How confect
differs from others?`_.
Two ways to load configuration file.
1. Through Python module importing: ``conf.load_module(module_name)``
2. Through Python file reading: ``conf.load_file(file_path)``
No matter the loading statement is located before or after properties
declaration, property values in configuration file always override default
values in the declarations. It's possible to load configuration file multiple times,
the latter one would replace values from former loading.
Be aware, *you should access your configuration properties after load
configuration files.* If not, you might get wrong/default value. Therefore, we
usually load configuration file right after the statement of creating the
``Conf`` object.
.. code:: python
import confect
conf = confect.Conf()
# load configuration files through importing
try:
conf.load_module('local_conf')
except ImportError:
pass
SYSTEM_CONF_PATH = Path('path/to/system_conf.py')
if SYSTEM_CONF_PATH.exists():
conf.load_file(SYSTEM_CONF_PATH)
Use ``PYTHONPATH`` environment varibale to control the source of configuration file.
.. code:: console
$ vi local_conf.py
$ export PYTHONPATH=.
$ python your_application.py
Write Configuration File
^^^^^^^^^^^^^^^^^^^^^^^^^
Configuration files are written in Python, but they are isolated from your application. The configuration declaration(check `Configuration Properties Declaration`_) use the ``conf`` object directly, and declare properties with default value with ``conf.declar_group(...)``. While configuration files use ``confect.c`` to override declared properties. Configuration files shouldn't be import directly, they can only be loaded with ``conf.load_module(module_name))`` or ``conf.load_file(file_path)``.
In configuration file, import ``confect.c`` object and set all properties on it
as if ``c`` is the conf object. Here's an example of configuration file.
``local_conf.py``
.. code:: python
from confect import c
import os
DEBUG = True
if DEBUG:
c.cache.expire = 1
c.cache.key = os.environ['CACHE_KEY']
# loading some secret file and set configuration
import json
with open('db_secret.json') as f:
db_secret = json.load(f)
c.db.username = db_secret['username']
c.db.password = db_secret['password']
It's not necessary and is unusual to have all configuration properties be defined in the
configuration file. *Put only those configuration properties that you want to override to the configuration file.*
You can set any property in any configuration group onto the ``c`` object.
However, **they are only accessable if you declared it in the source code with**
``Conf.declare_group(group_name)``. See `Configuration Properties Declaration`_ for details.
The ``c`` object only exits when loading a python configuration file, it's not
possible to import it in your source code.
Advanced Usage
===================
Loading Environment Variables
------------------------------
# overrides configuration with environment variables with the prefix `projx`
conf.load_envvars('projx')
``Conf.load_envvars(prefix: str)`` automatically searches environment variables
in ``<prefix>__<group>__<prop>`` format. All of these three identifier are case
sensitive. If you have a configuration property ``conf.cache.expire_time`` and
you call ``Conf.load_envvars('projx')``. It will set that ``expire_time``
property to the parsed value from ``projx__cache__expire_time`` environment
variable.
>>> import os
>>> os.environ['projx__cache__expire'] = '3600'
>>> conf = confect.Conf()
>>> conf.load_envvars('projx') # doctest: +SKIP
If ``cache.expire`` has been declared, then
>>> conf.cache.expire
3600
Command Line Options
-------------------------
``conf.click_options`` decorator attachs all declared configuration to a click_
command.
``projx/__main__.py``
.. code:: python
import click
from projx.core import conf
@click.command()
@conf.click_options
def cli():
click.echo(f'cache_expire: {conf.api.cache_expire}')
if __name__ == '__main__':
cli()
It automatically creates a comprehensive help message with all properties and default values.
.. code:: console
$ python -m projx.cli --help
Usage: cli.py [OPTIONS]
Options:
--api-cache_expire INTEGER [default: 86400]
--api-cache_prefix TEXT [default: projx_cache]
--api-url_base_path TEXT [default: api/v2/]
--db-db_name TEXT [default: proj_x]
--db-username TEXT [default: proj_x_admin]
--db-password TEXT [default: your_password]
--db-host TEXT [default: 127.0.0.1]
--help Show this message and exit.
The option do change the value of configuration property.
.. code:: console
$ python -m projx.cli
cache_expire: 86400
$ python -m projx.cli --api-cache_expire 33
cache_expire: 33
Parser
---------------
Confect includes predefined parsers of these primitive types.
- ``str``: ``s``
- ``int``: ``ast.literal_eval(s)``
- ``float``: ``ast.literal_eval(s)``
- ``bytes``: ``s.encode(encoding)``
- ``datetime.datetime`` : ``dt.datetime.strptime(s, fmt)``
- ``datetime.date`` : ``dt.datetime.strptime(s, fmt).date()``
- ``tuple`` : ``json.loads(s)``
- ``dict``: ``json.loads(s)``
- ``list``: ``json.loads(s)``
Complex Configuration Loading
-----------------------------
The code in the section `Conf Object`_ is a simple example that loads only through module importing.
Here's an much more complex example that demostrates how to dynamically select and load configurations.
.. code:: python
import sys
import confect
conf = confect.Conf()
# load configuration file
if len(sys.argv) == 2:
conf.load_file(sys.argv[1])
else:
try:
conf.load_file('path/to/team_conf.py')
FileNotFoundError:
logger.warning('Unable to find team configuration file')
try:
conf.load_file('path/to/personal_conf.py')
FileNotFoundError:
logger.info('Unable to find personal configuration file')
# load configuration file through importing
try:
conf.load_module('projx_conf')
except ImportError:
logger.warning('Unable to load find configuration module %r',
'proj_x_conf')
# overrides configuration with environment variables
conf.load_envvars('projx')
Runtime Configuration Altering
-------------------------------
``Conf.mutate_locally()`` context manager creates an environment that makes
``Conf`` object temporarily mutable. All changes would be restored when it
leaves the block. It is usaful on writing test case or testing configuration
properties in Python REPL.
>>> conf = Conf()
>>> conf.declare_group( # declare group through keyword arguments
... 'dummy',
... prop1=3,
... prop2='some string')
...
>>> with conf.mutate_locally():
... conf.dummy.prop1 = 5
... print(conf.dummy.prop1)
5
... call_some_function_use_this_property()
>>> print(conf.dummy.prop1) # all configuration restored
3
To-Dos
======
- A public interface for exporting a conf group into a dictionary
- A plugin for `argparse <https://docs.python.org/3/library/argparse.html>`_ that adds command line options for altering configuration properties.
- Copy-on-write mechenism in ``conf.mutate_locally()`` for better performance and memory usage.
- API reference page
.. _click: http://click.pocoo.org/
.. _pip: https://pip.pypa.io/en/stable/
.. _virtualenv: https://hynek.me/articles/virtualenv-lives/
.. _poetry: https://poetry.eustace.io/
نیازمندی
| مقدار | نام |
|---|---|
| >=2.0) | click |
| >=2.0.0,<3.0.0) | pendulum |
زبان مورد نیاز
| مقدار | نام |
|---|---|
| >=3.6 | Python |
نحوه نصب
نصب پکیج whl confect-0.4.3:
pip install confect-0.4.3.whl
نصب پکیج tar.gz confect-0.4.3:
pip install confect-0.4.3.tar.gz