معرفی شرکت ها
consistent-0.0.9
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
Make sure AI model outputs are consistent
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | consistent-0.0.9 |
| نام | consistent |
| نسخه کتابخانه | 0.0.9 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | - |
| ایمیل نویسنده | - |
| آدرس صفحه اصلی | - |
| آدرس اینترنتی | https://pypi.org/project/consistent/ |
| مجوز | MIT License Copyright (c) 2023 drorIvry Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
<img align="center" src="./logo.png">
[](https://github.com/drorIvry/consisTent/actions/workflows/pre-commit.yml)
[](https://github.com/drorIvry/consisTent/actions/workflows/python-publish.yml)
consisTent is a full blows testing framework for prompts. The goal of consisTent is to create reproducible tests for LLM based applications regardless of the FM used.
## Installation
```sh
pip install consistent
```
## Concepts
consisTent comes with 2 types of validators (testers)
### syntactic validators
This type of validators is used to do static assertions of the LLM output. for example validating if the output is in a certain JSON format and assert the schema.
You can also use syntactic validators to assert something is a valid piece of code (JS or Python are supported at the moment)
**Syntactic validators are used to assert the FORM of the response**
### example
```python
consisTent.JsValidator().validate('console.log("Im a JS program!")')
consisTent.PyValidator().validate('print("Im a python program!")')
consisTent.JsonValidator().validate('"question": {"is this a valid JSON?"}')
```
### syntactic validators
This type of validators is used to assert the quality of the response with more "soft" parameters for example, check if something is factually correct, check for hallucinations, check for labels like "funny"/"interesting" etc... another type of semantic validator is the semantic consistency validator where you provide a seed of validated input and a threshold, and the test will assert the semantic distance of the new output from the seed cluster.
**Semantic validators are used to assert the CONTENT of the response**
### example
```python
import consisTent
seed = [
"the cat sat on the mat",
"the feline layed on the carpet",
]
consisTent.ConsistencyValidator(
seed_size=2,
consistency_threshold=0.5,
).validate(
seed=seed,
model_output="the dog sat on the mat",
)
```
### Label Test
```python
OPENAI_KEY = "XXXXXXXXXXXXXXX"
consisTent.LabelsValidator(openai_key=OPENAI_KEY).validate(
labels=[
"funny",
"short",
"about rabbits",
],
model_output="What do you call a rabbit that tells jokes? A funny bunny!",
)
```
### facts validation
```python
OPENAI_KEY = "XXXXXXXXXXXXXXX"
consisTent.FactsValidator(openai_key=OPENAI_KEY).validate(
facts=["this car weighs 1000KG"],
model_output="I can lift this car",
)
```
نیازمندی
| مقدار | نام |
|---|---|
| - | chromadb |
| - | openai |
| - | js2py |
| - | jsonschema |
| - | langchain |
| - | tomli |
| - | black |
| - | bumpver |
| - | isort |
| - | pip-tools |
| - | pytest |
| - | pre-commit |
| - | pyflakes |
زبان مورد نیاز
| مقدار | نام |
|---|---|
| >=3.8 | Python |
نحوه نصب
نصب پکیج whl consistent-0.0.9:
pip install consistent-0.0.9.whl
نصب پکیج tar.gz consistent-0.0.9:
pip install consistent-0.0.9.tar.gz