معرفی شرکت ها
c8connector-0.0.9
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
C8 Connector Interface
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | c8connector-0.0.9 |
| نام | c8connector |
| نسخه کتابخانه | 0.0.9 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | Macrometa |
| ایمیل نویسنده | info@macrometa.com |
| آدرس صفحه اصلی | - |
| آدرس اینترنتی | https://pypi.org/project/c8connector/ |
| مجوز | Apache-2.0 |
# Implementing C8 Connectors.
Users can extend `C8Connector` interface and develop 3 types of connectors.
1. Source Connectors (Connectors that ingest data)
2. Target Connectors (Connectors that export data)
3. Integration Connectors (Generic integrations for other services)
When developing these connectors, developers must adhere to a few guidelines mentioned below.
## Naming the Connector
- Package name of the connector must be in the `macrometa-{type}-{connector}` format (i.e `macrometa-source-postgres`).
- Module name of the connector must be in the `macrometa_{type}_{connector}` format (i.e `macrometa_source_postgres`).
## Project structure (package names and structure)
- Project source code must follow the below structure.
```text
.
├── LICENSE
├── README.md
├── GETTING_STARTED.md
├── macrometa_{type}_{connector}
│ ├── __init__.py
│ └── main.py
│ └── {other source files or modules}
├── pyproject.toml
└── setup.cfg
```
- Within the `/macrometa_{type}_{connector}/__init__.py` there must be a class which implements `C8Connector` interface.
## Dependencies/Libraries and their versions to use.
- Connectors must only use following dependencies/libraries and mentioned versions' when developing.
```text
python = ">=3.7"
c8connector = "latest"
pipelinewise-singer-python = "1.2.0"
```
- Developers must not use `singer-sdk` or any other singer sdk variants other than `pipelinewise-singer-python`.
## Connector specific documentation
- Every connector project should have a GETTING_STARTED.md file, documenting the connector configuration and all other requirements for the connector.
It should be formatted like a User-Facing document and it should also provide the necessary instructions for the end user to be able to use the connector.
Developers can follow the Generic Template available [here](https://github.com/Macrometacorp/c8connector/blob/main/GETTING_STARTED.md) and apply any necessary changes required on top of it for the specific connector.
## Resolving reserved key conflicts between macrometa collection and external DB
- For Source connectors:
Macrometa collection(document) has the following reserved keys, `_key`, `_id` and `_rev`. `_key` is the primary key and hence, it will have the value of the primary key of source data and `_id` and `_rev` are always autogenerated. So if `_key`, `_id`, `_rev` also exists in source data (assuming _key is not the primary key of source data) then these values from source data would be lost.
Hence we need to append an additional `_` to these reserved keys if they are present in source data. If `_key` is the primary key of the source data itself then no need to append `_` to `_key`. We should also check that the new key generated doesn't exist in the source data, If it exists then keep appending `_`.
During the actual workflow run, this logic is implemented at the target level (macrometa-target-collection). But we also need the same to be implemented at source connector levels for `samples` and `schemas` API.
Refer [PR] (https://github.com/Macrometacorp/macrometa-source-postgres/pull/16)
- For Target connectors:
As seen for source connector with target as (macrometa-target-collection), the same reserved keys conflict can arise in case of target connectors too, where External database might have some fixed reserved keys which might be their primary key, autogenerated or internal key. So if such reserved keys also exist in source collection then these values from source collection will be lost in the target data.
In such cases we should first specify all the reserved keys as a list of string in the `reserved_keys` field of target connector. If there is a fixed primary key it should always be specified as the first element of the list, else if there isn't a fixed primary key but there are other reserved keys then the first element should be an empty string followed by the list of reserved keys, Example: ["", "reservedkey1", "reservedkey2"]. If no reserved keys exist return an empty list []. Refer [PR] (https://github.com/Macrometacorp/macrometa-target-collection/pull/9)
In addition to this, we also need to implement the logic of appending `_` to the reserved keys (only if they exist in source collection) before writing the data in the external DB at the target connector level. If `_key` is also the reserved primary key of the target external DB then no need to append `_` to this reserved primary key. We should also check that the new key generated doesn't exist in the source collection, If it exists then keep appending `_`. Refer [PR] (https://github.com/Macrometacorp/macrometa-target-collection/pull/10)
> **_NOTE:_** This is applicable only when there are certain keys reserved in the external database.
## Samples
- Postgres Source Connector: [Git Repository](https://github.com/Macrometacorp/macrometa-source-postgres)
- Oracle Source Connector: [Git Repository](https://github.com/Macrometacorp/macrometa-source-oracle)
- C8 Collections target Connector: [Git Repository](https://github.com/Macrometacorp/macrometa-target-collection)
نیازمندی
| مقدار | نام |
|---|---|
| ==2.9.3 | psycopg2-binary |
| ==3.0.0 | watchdog |
زبان مورد نیاز
| مقدار | نام |
|---|---|
| >=3.8.1,<3.11 | Python |
نحوه نصب
نصب پکیج whl c8connector-0.0.9:
pip install c8connector-0.0.9.whl
نصب پکیج tar.gz c8connector-0.0.9:
pip install c8connector-0.0.9.tar.gz