aioredis_cluster
================
[](https://pypi.org/project/aioredis-cluster/) 
Redis Cluster support for [aioredis](https://github.com/aio-libs/aioredis) (support only v1.x.x).
Many implementation features were inspired by [go-redis](https://github.com/go-redis/redis) project.
Requirements
------------
* [Python](https://www.python.org) 3.6.5+
* [async_timeout](https://pypi.org/project/async_timeout/)
* [dataclasses](https://pypi.org/project/dataclasses/) (only for Python 3.6)
Features
--------
* commands execute failover (retry command on other node in cluster)
* support resharding replies ASK/MOVED
* restore cluster state from alive nodes
* one node is enough to know the topology and initialize client
* cluster state auto reload
Limitations
-----------
### Commands with limitations
* Keys in `mget`/`mset` must provide one key slot.
```python
# works
await redis.mget("key1:{foo}", "key2:{foo}")
# throw RedisClusterError
await redis.mget("key1", "key2")
```
### Commands are not supported
`Redis` methods below do not works and not supported in cluster mode implementation.
```
cluster_add_slots
cluster_count_failure_reports
cluster_count_key_in_slots
cluster_del_slots
cluster_failover
cluster_forget
cluster_get_keys_in_slots
cluster_meet
cluster_replicate
cluster_reset
cluster_save_config
cluster_set_config_epoch
cluster_setslot
cluster_readonly
cluster_readwrite
client_setname
shutdown
slaveof
script_kill
move
select
flushall
flushdb
script_load
script_flush
script_exists
scan
iscan
quit
swapdb
migrate
migrate_keys
wait
bgrewriteaof
bgsave
config_rewrite
config_set
config_resetstat
save
sync
pipeline
multi_exec
```
But you can always execute command you need on concrete node on cluster with usual `aioredis.RedisConnection`, `aioredis.ConnectionsPool` or high-level `aioredis.Redis` interfaces.
Installation
------------
```bash
pip install aioredis-cluster
```
Usage
-----
```python
import aioredis_cluster
redis = await aioredis_cluster.create_redis_cluster([
"redis://redis-cluster-node1",
])
# or
redis = await aioredis_cluster.create_redis_cluster([
"redis://redis-cluster-node1",
"redis://redis-cluster-node2",
"redis://redis-cluster-node3",
])
# or
redis = await aioredis_cluster.create_redis_cluster([
("redis-cluster-node1", 6379),
])
await redis.set("key", "value", expire=180)
redis.close()
await redis.wait_closed()
```
License
-------
The aioredis_cluster is offered under MIT license.
Changes
=======
2.3.1 (2022-07-29)
------------------
* fix bypass `username` argument for pool creation
2.3.0 (2022-07-26)
------------------
* add support Redis 6 `AUTH` command with username
* factories `create_cluster`, `create_redis_cluster`, `aioredis_cluster.aioredis.create_connection` now support `username` argument
* add `auth_with_username` method for `AbcConnection`, `AbcPool` and impementations
2.2.2 (2022-07-19)
------------------
* fix problem when RedisConnection was GC collected after unhandled `asyncio.CancelledError`
* fix default `db` argument for pool/connection in cluster mode
2.2.1 (2022-07-18)
------------------
* (revert) apply cluster state only if cluster metadata is changed
2.2.0 (2022-07-18)
------------------
* fetch several cluster state candidates from cluster for choose best metadata for final local state
* apply cluster state only if cluster metadata is changed
* FIX: handle closed pubsub connection before gc its collected that trigger `Task was destroyed but it is pending!` message in log
* improve logging in state loader
2.1.0 (2022-07-10)
------------------
* fix bug when `ConnectionsPool.acquire()` is stuck because closed PUB/SUB connection is not cleanup from `used` set
* fix `ConnectionsPool.acquire()` incorrect wakeup order for connection waiters when connection is released
* `ConnectionsPool.execute()` now acquire dedicate connection for execution if command is blocking, ex. `BLPOP`
* `ConnectionsPool.execute()` now raises `ValueError` exception for PUB/SUB family command
* In `ConnectionsPool` PUB/SUB dedicated connections now is closing on `close()` call
* add `aioredis_cluster.abc.AbcConnection` abstract class
* add property `readonly` and method `set_readonly()` for `aioredis_cluster.abc.AbcConnection` and `aioredis_cluster.abc.AbcPool`
* `aioredis_cluster.Cluster` now require `pool_cls` implementation from `aioredis_cluster.abc.AbcPool`
* add `ssl` argument for factories `create_cluster`, `create_redis_cluster` and `Cluster` constructor
* add 10% jitter for cluster state auto reload interval
* fix incorrect iterate free connections in `select()`, `auth()` methods for `ConnectionsPool`
2.0.0 (2022-06-20)
------------------
* include `aioredis==1.3.1` source code into `aioredis_cluster._aioredis` and introduce `aioredis_cluster.aioredis` but for compatible and migration period
* this release have not backward incompatible changes
* __DEPRECATION WARNING:__ you must migrate from `import aioredis` to `import aioredis_cluster.aioredis` because `aioredis_cluster` starts vendorize `aioredis` package and maintain it separately. Using `aioredis` package __will be removed in v3__
* fix reacquire connection in `aioredic.ConnectionsPool` after Redis node failure
1.8.0 (2022-05-20)
------------------
* Add `xadd_620` commands method for support `XADD` options for Redis 6.2+
1.7.1 (2021-12-15)
------------------
* add `ClusterState.slots_assigned`
* require reload cluster state for some cases with `UncoveredSlotError`
1.7.0 (2021-12-15)
------------------
* add `execute_timeout` for `Manager`
* improve cluster state reload logging
* reduce number of addresses to fetch cluster state
* acquire dedicate connection from pool to fetch cluster state
* extend `ClusterState` by new attributes: `state`, `state_from`, `current_epoch`
1.6.1 (2021-11-23)
------------------
* fix keys extraction for `XREAD` and `XREADGROUP` commands
1.6.0 (2021-11-20)
------------------
* make public `Address`, `ClusterNode` and `ClusterState` structs. Available by import `from aioredis_cluster import`
* `Cluster` provides some new helpful methods:
* `get_master_node_by_keys(*keys)` - return master `ClusterNode` which contains keys `keys`
* `create_pool_by_addr(addr, **kwargs)` - create connection pool by `addr` and return pool wrapped by `commands_factory` from `Cluster` constructor. By default is `aioredis_cluster.RedisCluster` instance.
* `get_cluster_state()` - return `ClusterState` instance with recent known cluster state received from Redis cluster
* `extract_keys(command_sequence)` - returns keys of command sequence
* drop `pytest-aiohttp` plugin for tests
* add `pytest-asyncio` dependency for tests
* switch `asynctest` -> `mock` library for aio tests
* drop `attrs` dependency. For Python 3.6 you need install `dataclasses`
* fix extract keys for `BLPOP`/`BRPOP` commands
* add support keys extraction for `ZUNION`, `ZINTER`, `ZDIFF`, `ZUNIONSTORE`, `ZINTERSTORE`, `ZDIFFSTORE` commands
* acquire dedicate connection from pool for potential blocking commands like `BLPOP`, `BRPOP`, `BRPOPLPUSH`, `BLMOVE`, `BLMPOP`, `BZPOPMIN`, `BZPOPMAX`, `XREAD`, `XREADGROUP`
1.5.2 (2020-12-14)
------------------
* README update
1.5.1 (2020-12-11)
------------------
* speedup crc16. Use implementation from python stdlib
1.5.0 (2020-12-10)
------------------
* remove `state_reload_frequency` from `ClusterManager`. `state_reload_interval` now is one relevant option for state auto reload
* default `state_reload_interval` increased and now is 300 seconds (5 minutes)
* commands registry loads only once, on cluster state initialize
* improve failover. First connection problem cause retry to random slot replica
* improve python3.9 support
* default `idle_connection_timeout` now is 10 minutes
1.4.0 (2020-09-08)
------------------
* fix `aioredis.locks.Lock` issue (https://github.com/aio-libs/aioredis/pull/802, [bpo32734](https://bugs.python.org/issue32734))
* now `aioredis_cluster.Cluster` do not acquire dedicate connection for every execute
* `aioredis_cluster` now requires `python>=3.6.5`
1.3.0 (2019-10-23)
------------------
* improve compatible with Python 3.8
* improve failover logic while command timed out
* read-only commands now retries if attempt_timeout is reached
* add required dependeny `async_timeout`
* `aioredis` dependency bound now is `aioredis >=1.1.0, <2.0.0`
1.2.0 (2019-09-10)
------------------
* add timeout for command execution (per execution try)
* add Cluster option `attempt_timeout` for configure command execution timeout, default timeout is 5 seconds
* Cluster.execute_pubsub() fixes
1.1.1 (2019-06-07)
------------------
* CHANGES fix
1.1.0 (2019-06-06)
------------------
* Cluster state auto reload
* new `state_reload_frequency` option to configure state reload frequency
* new `state_reload_interval` option to configure state auto reload interval
* `follow_cluster` option enable load cluster state from previous cluster state nodes
* establish connection only for master nodes after cluster state load
* change default commands_factory to aioredis_cluster.RedisCluster instead aioredis.Redis
* all cluster info commands always returns structs with str, not bytes
* `keys_master` and `all_masters` methods now try to ensure cluster state instead simply raise exception if connection lost to cluster node, for example
* `max_attempts` always defaults fix
1.0.0 (2019-05-29)
------------------
* Library full rewrite
* Cluster state auto reload
* Command failover if cluster node is down or key slot resharded
0.2.0 (2018-12-27)
------------------
* Pipeline and MULTI/EXEC cluster implementation with keys distribution limitation (because cluster)
0.1.1 (2018-12-26)
------------------
* Python 3.6+ only
0.1.0 (2018-12-24)
------------------
* Initial release based on aioredis PR (https://github.com/aio-libs/aioredis/pull/119)
Contributors
============
* [Anton Ilyushenkov](https://github.com/DriverX)
* [Vadim Pushtaev](https://github.com/VadimPushtaev)
* [erastov](https://github.com/erastov)
* [roman901](https://github.com/roman901)