معرفی شرکت ها
ai-minimization-toolkit-0.0.1
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
A toolkit for minimizing the data required to apply machine learning
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | ai-minimization-toolkit-0.0.1 |
| نام | ai-minimization-toolkit |
| نسخه کتابخانه | 0.0.1 |
| نگهدارنده | ['A. Goldsteen'] |
| ایمیل نگهدارنده | ['abigailt@il.ibm.com'] |
| نویسنده | - |
| ایمیل نویسنده | - |
| آدرس صفحه اصلی | - |
| آدرس اینترنتی | https://pypi.org/project/ai-minimization-toolkit/ |
| مجوز | new BSD |
.. -*- mode: rst -*-
minimization-toolkit - A toolkit for performing data minimization for machine learning models
=============================================================================================
The EU General Data Protection Regulation (GDPR) mandates the principle of data minimization, which requires that only data necessary to fulfill a certain purpose be collected. However, it can often be difficult to determine the minimal amount of data required, especially in complex machine learning models such as neural networks.
This toolkit is a first-of-a-kind implementation to help reduce the amount of personal data needed to perform predictions with a machine learning model, by removing or generalizing some of the input features. The type of data minimization this toolkit focuses on is the reduction of the number and/or granularity of features collected for analysis.
The generalization process basically searches for several similar records and groups them together. Then, for each feature, the individual values for that feature within each group are replaced with a represenataive value that is common across the whole group. This process is done while using knowledge encoded within the model to produce a generalization that has little to no impact on its accuracy.
The minimization-toolkit is compatible with: ``Python 3.7``.
Official `ai-minimization-toolkit documentation <https://ai-minimization-toolkit.readthedocs.io/en/master/>`__
Using the minimization-toolkit
------------------------------
The main class, ``GeneralizeToRepresentative``, is a scikit-learn compatible ``Transformer``, that receives an existing estimator and labeled training data, and learns the generalizations that can be applied to any newly collected data for analysis by the original model. The ``fit()`` method learns the generalizations and the ``transform()`` method applies them to new data.
It is also possible to export the generalizations as feature ranges.
The current implementation supports only numeric features, so any categorical features must be transformed to a numeric representation before using this class.
Start by training your machine learning model. In this example, we will use a ``DecisionTreeClassifier``, but any scikit-learn model can be used. We will use the iris dataset in our example.
.. code:: python
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
dataset = datasets.load_iris()
X_train, X_test, y_train, y_test = train_test_split(dataset.data, dataset.target, test_size=0.2)
base_est = DecisionTreeClassifier()
base_est.fit(X_train, y_train)
Now create the ``GeneralizeToRepresentative`` transformer and train it. Supply it with the original model and the desired target accuracy. The training process may receive the original labeled training data or the model's predictions on the data.
.. code:: python
predictions = base_est.predict(X_train)
gen = GeneralizeToRepresentative(base_est, target_accuracy=0.9)
gen.fit(X_train, predictions)
Now use the transformer to transform new data, for example the test data.
.. code:: python
transformed = gen.transform(X_test)
The transformed data has the same columns and formats as the original data, so it can be used directly to derive predictions from the original model.
.. code:: python
new_predictions = base_est.predict(transformed)
To export the resulting generalizations, retrieve the ``Transformer``'s ``_generalize`` parameter.
.. code:: python
generalizations = base_est._generalize
The returned object has the following structure::
{
ranges:
{
list of (<feature name>: [<list of values>])
},
untouched: [<list of feature names>]
}
For example::
{
ranges:
{
age: [21.5, 39.0, 51.0, 70.5],
education-years: [8.0, 12.0, 14.5]
},
untouched: ["occupation", "marital-status"]
}
Where each value inside the range list represents a cutoff point. For example, for the ``age`` feature, the ranges in this example are: ``<21.5, 21.5-39.0, 39.0-51.0, 51.0-70.5, >70.5``. The ``untouched`` list represents features that were not generalized, i.e., their values should remain unchanged.
نیازمندی
| مقدار | نام |
|---|---|
| - | numpy |
| - | scipy |
| - | scikit-learn |
| - | pandas |
| - | sphinx |
| - | sphinx-rtd-theme |
| - | numpydoc |
| - | matplotlib |
| - | pytest |
| - | pytest-cov |
نحوه نصب
نصب پکیج whl ai-minimization-toolkit-0.0.1:
pip install ai-minimization-toolkit-0.0.1.whl
نصب پکیج tar.gz ai-minimization-toolkit-0.0.1:
pip install ai-minimization-toolkit-0.0.1.tar.gz