معرفی شرکت ها
Clusters-Features-1.0.3
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشتر
تبلیغات ما
مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.
مشاهده بیشترتوضیحات
The Clusters-Features package allows data science users to compute high-level linear algebra operations on any type of data set.
| ویژگی | مقدار |
|---|---|
| سیستم عامل | - |
| نام فایل | Clusters-Features-1.0.3 |
| نام | Clusters-Features |
| نسخه کتابخانه | 1.0.3 |
| نگهدارنده | [] |
| ایمیل نگهدارنده | [] |
| نویسنده | Simon Bertrand |
| ایمیل نویسنده | simonbertrand.contact@gmail.com |
| آدرس صفحه اصلی | https://github.com/Simon-Bertrand |
| آدرس اینترنتی | https://pypi.org/project/Clusters-Features/ |
| مجوز | - |
# Clusters-Features : a Python module to evaluate the quality of clustering #
<div align="center">
<img src="https://user-images.githubusercontent.com/84455908/128029707-0bf27b40-0570-4f13-8b04-cfaa6c2b560e.png" height="121" width="400" style="display: block; margin-left:auto; margin-right: auto; width: 50%;" /><br />
<i> This package is made for unsupervised learning. All criterias are used with internal validation and make no use of ground-truth labels. </i>
# Official documentation : <a href="https://simonbertrand.pages.unistra.fr/Clusters-Features/" alt="Official documentation">simonbertrand.pages.unistra.fr/Clusters-Features </a> #
</div>
# Table of contents
1. [Introduction](#introduction) <br />
2. [Command Line Interface](#CLI)<br />
3. [Data](#Data)<br />
4. [Score](#Score) <br />
5. [Confusion Hypersphere](#CH) <br />
6. [Info](#Info) <br />
7. [Density](#Info) <br />
8. [Utils](#Utils) <br />
9. [Graphs](#Graph) <br />
## Introduction <a name="introduction"></a>
Clusters-Features is a package that computes many operations using only the dataset and the target vector.
<br /><br /><b> Data </b><br />
The package provides all the usefull data such as pairwise distances or distances between every elements and the centroid of given cluster. You can also check for the maximum/minimum distances between two elements of different clusters or even each intercentroid distances. But you can also get different radius for each centroids and analyse them to firstly understand the shape of the clusters. The distribution of all radius for each cluster is also available. All informative data is contained inside the subclass Data.
<br /><br /><b> Score </b><br />
Approximatively 40 different internal indices have been implemented in Python :<i><b> Ball-Hall Index, Dunn Index, Generalized Dunn Indexes (18 indexes), C Index, Banfeld-Raftery Index, Davies-Bouldin Index, Calinski-Harabasz Index, Ray-Turi Index, Xie-Beni Index, Ratkowsky Lance Index, SD Index, Mclain Rao Index, Scott-Symons Index, PBM Index, Point biserial Index, Det Ratio Index, Log SumSquare Ratio Index, Wemmert-Gançarski Index.</b></i> We use two systems to generate these index, the first is caching each computed index and the other directly compute them.
All these score are defined in the main reference, check for the [Score](#Score) section to find the reference.
<br /><br /><b> Confusion Hypersphere </b><br />
Clusters-Features also provides a deep analysis of the multidimensionnal space. Confusion Hypersphere consists of counting the number of element contained in several hyperspheres centered on different positions and with different radius. This feature allows users to understand which clusters are confused (in the sense of the Euclidean norm) with other clusters. These indicators make it possible to determine which clusters are the most separated from the others and this is clearly adapted to convex clusters since the hypersphere is convex.
<br /><br /><b> Info </b><br />
Info gives two kind of boards such as clusters board which gives you information for each clusters. The general board gives informations at a general scale of the dataset .
<br /><br /><b> Density </b><br />
This section uses a meshgrid to estimate a density by summing n-dim (for n=2 or n=3) Gaussian distrubution centred on each dataset points. We put the minimum contour as a given percentile of the current density. If percentile is 99% then only 1% of the highest density values will be retained. We can make it for 2D grid or 3D grid but it is quickly limited due to the large number of combinations needed to generate an n-dim grid.
<br /><br /><b> Utils </b><br />
Implement external packages and utils to the current dataset.
<br /><br /><b> Graph (Falcutative) </b><br />
Graph allows users to plot few kind of data generated by Clusters-Features. As Plotly is used to plot, this section is facultative in the case where user only need to get the different data and matrix to plot with their own module. In order to disable this section, you will have to go to settings.py and put to False the variable "Activated_Graph" and then re-build the package using setuptools. All requirements.txt are going to be generated in consequences of these settings.
## Dependencies ##
Native dependencies :
- [Numpy](https://pypi.org/project/numpy/)
- [Pandas](https://pypi.org/project/pandas/)
- [Scikit-learn](https://pypi.org/project/scikit-learn/)
- [Scipy](https://pypi.org/project/scipy/)
Falcultative dependencies (may cause errors if the user forces the use of the method of these falcultative dependencies without having installed the correct libraries) :
- Graph : [Plotly](https://pypi.org/project/plotly/)
- Utils : [umap-learn](https://pypi.org/project/umap-learn/), [Numba](https://pypi.org/project/numba/), [statsmodels](https://pypi.org/project/statsmodels/)
Graph is dependent of Utils to correctly work but the reciprocal is not true.
## Command Line Interface <a name="CLI"></a>
This package provides a command line interface that is available by running this command
```bash
python3 ./clustersfeatures-cli.py -h
```
The documentation for the CLI is contained inside the script. Just use --help arguments to understand what it does.
## Import the module ##
```python
from ClustersFeatures import *
```
## Load a random data set ##
We choose here the scikit-learn digits data set because it is in high dimension (64) and has a large number of observations.
```python
from sklearn.datasets import load_digits
import pandas as pd
digits = load_digits()
pd_df=pd.DataFrame(digits.data)
pd_df['target'] = digits.target
pd_df
```
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
<th>9</th>
<th>...</th>
<th>55</th>
<th>56</th>
<th>57</th>
<th>58</th>
<th>59</th>
<th>60</th>
<th>61</th>
<th>62</th>
<th>63</th>
<th>target</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>0.0</td>
<td>0.0</td>
<td>5.0</td>
<td>13.0</td>
<td>9.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>6.0</td>
<td>13.0</td>
<td>10.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0</td>
</tr>
<tr>
<th>1</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>12.0</td>
<td>13.0</td>
<td>5.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>11.0</td>
<td>16.0</td>
<td>10.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1</td>
</tr>
<tr>
<th>2</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>4.0</td>
<td>15.0</td>
<td>12.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>3.0</td>
<td>11.0</td>
<td>16.0</td>
<td>9.0</td>
<td>0.0</td>
<td>2</td>
</tr>
<tr>
<th>3</th>
<td>0.0</td>
<td>0.0</td>
<td>7.0</td>
<td>15.0</td>
<td>13.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>8.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>7.0</td>
<td>13.0</td>
<td>13.0</td>
<td>9.0</td>
<td>0.0</td>
<td>0.0</td>
<td>3</td>
</tr>
<tr>
<th>4</th>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>11.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>2.0</td>
<td>16.0</td>
<td>4.0</td>
<td>0.0</td>
<td>0.0</td>
<td>4</td>
</tr>
<tr>
<th>...</th>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
</tr>
<tr>
<th>1792</th>
<td>0.0</td>
<td>0.0</td>
<td>4.0</td>
<td>10.0</td>
<td>13.0</td>
<td>6.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>2.0</td>
<td>14.0</td>
<td>15.0</td>
<td>9.0</td>
<td>0.0</td>
<td>0.0</td>
<td>9</td>
</tr>
<tr>
<th>1793</th>
<td>0.0</td>
<td>0.0</td>
<td>6.0</td>
<td>16.0</td>
<td>13.0</td>
<td>11.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>6.0</td>
<td>16.0</td>
<td>14.0</td>
<td>6.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0</td>
</tr>
<tr>
<th>1794</th>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>11.0</td>
<td>15.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>2.0</td>
<td>9.0</td>
<td>13.0</td>
<td>6.0</td>
<td>0.0</td>
<td>0.0</td>
<td>8</td>
</tr>
<tr>
<th>1795</th>
<td>0.0</td>
<td>0.0</td>
<td>2.0</td>
<td>10.0</td>
<td>7.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>5.0</td>
<td>12.0</td>
<td>16.0</td>
<td>12.0</td>
<td>0.0</td>
<td>0.0</td>
<td>9</td>
</tr>
<tr>
<th>1796</th>
<td>0.0</td>
<td>0.0</td>
<td>10.0</td>
<td>14.0</td>
<td>8.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>2.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>8.0</td>
<td>12.0</td>
<td>14.0</td>
<td>12.0</td>
<td>1.0</td>
<td>0.0</td>
<td>8</td>
</tr>
</tbody>
</table>
<p>1797 rows × 65 columns</p>
</div>
The important thing is that the given "pd_df" dataframe in the following argument has to be concatenated with the target vector.
Then, just specify as second argument which column name has the target. The program is making automatically the separation :
```python
CC=ClustersCharacteristics(pd_df,label_target="target")
```
## Data tools <a name="Data"></a>
The ClustersCharacteristics object creates attributes that define clusters. We can find for example the barycenter.
```python
CC.data_barycenter
```
0 0.000000
1 0.303840
2 5.204786
3 11.835838
4 11.848080
...
59 12.089037
60 11.809126
61 6.764051
62 2.067891
63 0.364496
Length: 64, dtype: float64
But also centroids, where the column j of the following matrix correspond to the coordinates of centroid of cluster j.
```python
CC.data_centroids
```
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th>target</th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
<th>9</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>0.000000</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.000000</td>
</tr>
<tr>
<th>1</th>
<td>0.022472</td>
<td>0.010989</td>
<td>0.932203</td>
<td>0.644809</td>
<td>0.000000</td>
<td>0.967033</td>
<td>0.000000</td>
<td>0.167598</td>
<td>0.143678</td>
<td>0.144444</td>
</tr>
<tr>
<th>2</th>
<td>4.185393</td>
<td>2.456044</td>
<td>9.666667</td>
<td>8.387978</td>
<td>0.453039</td>
<td>9.983516</td>
<td>1.138122</td>
<td>5.100559</td>
<td>5.022989</td>
<td>5.683333</td>
</tr>
<tr>
<th>3</th>
<td>13.095506</td>
<td>9.208791</td>
<td>14.186441</td>
<td>14.169399</td>
<td>7.055249</td>
<td>13.038462</td>
<td>11.165746</td>
<td>13.061453</td>
<td>11.603448</td>
<td>11.833333</td>
</tr>
<tr>
<th>4</th>
<td>11.297753</td>
<td>10.406593</td>
<td>9.627119</td>
<td>14.224044</td>
<td>11.497238</td>
<td>13.895604</td>
<td>9.585635</td>
<td>14.245810</td>
<td>12.402299</td>
<td>11.255556</td>
</tr>
<tr>
<th>...</th>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
</tr>
<tr>
<th>59</th>
<td>13.561798</td>
<td>9.137363</td>
<td>13.966102</td>
<td>14.650273</td>
<td>7.812155</td>
<td>14.736264</td>
<td>10.685083</td>
<td>11.659218</td>
<td>12.695402</td>
<td>12.044444</td>
</tr>
<tr>
<th>60</th>
<td>13.325843</td>
<td>13.027473</td>
<td>13.118644</td>
<td>13.972678</td>
<td>11.812155</td>
<td>9.362637</td>
<td>15.093923</td>
<td>2.206704</td>
<td>13.011494</td>
<td>13.144444</td>
</tr>
<tr>
<th>61</th>
<td>5.438202</td>
<td>8.576923</td>
<td>11.796610</td>
<td>8.672131</td>
<td>1.955801</td>
<td>2.532967</td>
<td>13.044199</td>
<td>0.011173</td>
<td>6.735632</td>
<td>8.894444</td>
</tr>
<tr>
<th>62</th>
<td>0.275281</td>
<td>3.049451</td>
<td>8.022599</td>
<td>1.409836</td>
<td>0.000000</td>
<td>0.197802</td>
<td>4.480663</td>
<td>0.000000</td>
<td>1.206897</td>
<td>2.094444</td>
</tr>
<tr>
<th>63</th>
<td>0.000000</td>
<td>1.494505</td>
<td>1.932203</td>
<td>0.065574</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.093923</td>
<td>0.000000</td>
<td>0.011494</td>
<td>0.055556</td>
</tr>
</tbody>
</table>
<p>64 rows × 10 columns</p>
</div>
We can show the list of clusters labels :
```python
CC.labels_clusters
```
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
And look for the data with the same label target. For example we take here the first cluster label of the above list.
```python
Cluster=CC.labels_clusters[0]
CC.data_clusters[Cluster]
```
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
<th>9</th>
<th>...</th>
<th>54</th>
<th>55</th>
<th>56</th>
<th>57</th>
<th>58</th>
<th>59</th>
<th>60</th>
<th>61</th>
<th>62</th>
<th>63</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>0.0</td>
<td>0.0</td>
<td>5.0</td>
<td>13.0</td>
<td>9.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>6.0</td>
<td>13.0</td>
<td>10.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>10</th>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>9.0</td>
<td>15.0</td>
<td>11.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>1.0</td>
<td>10.0</td>
<td>13.0</td>
<td>3.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>20</th>
<td>0.0</td>
<td>0.0</td>
<td>3.0</td>
<td>13.0</td>
<td>11.0</td>
<td>7.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>2.0</td>
<td>12.0</td>
<td>13.0</td>
<td>4.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>30</th>
<td>0.0</td>
<td>0.0</td>
<td>10.0</td>
<td>14.0</td>
<td>11.0</td>
<td>3.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>4.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>11.0</td>
<td>16.0</td>
<td>12.0</td>
<td>3.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>36</th>
<td>0.0</td>
<td>0.0</td>
<td>6.0</td>
<td>14.0</td>
<td>10.0</td>
<td>2.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>7.0</td>
<td>16.0</td>
<td>11.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>...</th>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
</tr>
<tr>
<th>1739</th>
<td>0.0</td>
<td>0.0</td>
<td>10.0</td>
<td>11.0</td>
<td>7.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>4.0</td>
<td>...</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>7.0</td>
<td>12.0</td>
<td>8.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>1745</th>
<td>0.0</td>
<td>0.0</td>
<td>7.0</td>
<td>14.0</td>
<td>8.0</td>
<td>4.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>6.0</td>
<td>13.0</td>
<td>7.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>1746</th>
<td>0.0</td>
<td>0.0</td>
<td>9.0</td>
<td>15.0</td>
<td>6.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>2.0</td>
<td>...</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>8.0</td>
<td>15.0</td>
<td>11.0</td>
<td>4.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
<tr>
<th>1768</th>
<td>0.0</td>
<td>0.0</td>
<td>5.0</td>
<td>16.0</td>
<td>10.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>5.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>4.0</td>
<td>15.0</td>
<td>16.0</td>
<td>8.0</td>
<td>1.0</td>
<td>0.0</td>
</tr>
<tr>
<th>1793</th>
<td>0.0</td>
<td>0.0</td>
<td>6.0</td>
<td>16.0</td>
<td>13.0</td>
<td>11.0</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>...</td>
<td>1.0</td>
<td>0.0</td>
<td>0.0</td>
<td>0.0</td>
<td>6.0</td>
<td>16.0</td>
<td>14.0</td>
<td>6.0</td>
<td>0.0</td>
<td>0.0</td>
</tr>
</tbody>
</table>
<p>178 rows × 64 columns</p>
</div>
Users are able to get a pairwise distance matrix generated by the Scipy library (fast).
If (xi,j)i,j is the returned matrix, then xi,j is the distance between element of index i and element of index j. The matrix is symetric as we use Euclidian norm to evaluate distances.
```python
CC.data_every_element_distance_to_every_element
```
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
<th>9</th>
<th>...</th>
<th>1787</th>
<th>1788</th>
<th>1789</th>
<th>1790</th>
<th>1791</th>
<th>1792</th>
<th>1793</th>
<th>1794</th>
<th>1795</th>
<th>1796</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>0.000000</td>
<td>59.556696</td>
<td>54.129474</td>
<td>47.571000</td>
<td>50.338852</td>
<td>43.908997</td>
<td>48.559242</td>
<td>56.000000</td>
<td>44.395946</td>
<td>40.804412</td>
<td>...</td>
<td>39.874804</td>
<td>49.749372</td>
<td>52.640289</td>
<td>51.458721</td>
<td>49.989999</td>
<td>36.249138</td>
<td>26.627054</td>
<td>50.378567</td>
<td>37.067506</td>
<td>47.031904</td>
</tr>
<tr>
<th>1</th>
<td>59.556696</td>
<td>0.000000</td>
<td>41.629317</td>
<td>45.475268</td>
<td>47.906158</td>
<td>47.127487</td>
<td>40.286474</td>
<td>50.960769</td>
<td>48.620983</td>
<td>52.820451</td>
<td>...</td>
<td>52.009614</td>
<td>48.969378</td>
<td>42.965102</td>
<td>32.572995</td>
<td>47.707442</td>
<td>51.390661</td>
<td>59.177699</td>
<td>38.587563</td>
<td>48.569538</td>
<td>50.328918</td>
</tr>
<tr>
<th>2</th>
<td>54.129474</td>
<td>41.629317</td>
<td>0.000000</td>
<td>53.953684</td>
<td>52.096065</td>
<td>55.443665</td>
<td>45.650849</td>
<td>49.335586</td>
<td>42.602817</td>
<td>54.836119</td>
<td>...</td>
<td>59.076222</td>
<td>47.927028</td>
<td>46.335731</td>
<td>39.191836</td>
<td>46.936127</td>
<td>51.826634</td>
<td>52.009614</td>
<td>38.340579</td>
<td>50.774009</td>
<td>43.954522</td>
</tr>
<tr>
<th>3</th>
<td>47.571000</td>
<td>45.475268</td>
<td>53.953684</td>
<td>0.000000</td>
<td>51.215232</td>
<td>33.660065</td>
<td>47.254629</td>
<td>56.824291</td>
<td>42.449971</td>
<td>45.166359</td>
<td>...</td>
<td>37.934153</td>
<td>55.569776</td>
<td>50.099900</td>
<td>43.988635</td>
<td>58.566202</td>
<td>40.286474</td>
<td>55.551778</td>
<td>49.527770</td>
<td>44.147480</td>
<td>41.267421</td>
</tr>
<tr>
<th>4</th>
<td>50.338852</td>
<td>47.906158</td>
<td>52.096065</td>
<td>51.215232</td>
<td>0.000000</td>
<td>54.147945</td>
<td>36.959437</td>
<td>59.481089</td>
<td>52.507142</td>
<td>55.054518</td>
<td>...</td>
<td>48.620983</td>
<td>26.172505</td>
<td>55.794265</td>
<td>48.723711</td>
<td>31.416556</td>
<td>53.981478</td>
<td>51.449004</td>
<td>46.882833</td>
<td>52.668776</td>
<td>50.970580</td>
</tr>
<tr>
<th>...</th>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
</tr>
<tr>
<th>1792</th>
<td>36.249138</td>
<td>51.390661</td>
<td>51.826634</td>
<td>40.286474</td>
<td>53.981478</td>
<td>29.325757</td>
<td>52.191953</td>
<td>55.605755</td>
<td>40.037482</td>
<td>36.262929</td>
<td>...</td>
<td>31.749016</td>
<td>54.543561</td>
<td>55.758407</td>
<td>48.083261</td>
<td>55.488738</td>
<td>0.000000</td>
<td>41.940434</td>
<td>46.151923</td>
<td>23.537205</td>
<td>40.963398</td>
</tr>
<tr>
<th>1793</th>
<td>26.627054</td>
<td>59.177699</td>
<td>52.009614</td>
<td>55.551778</td>
<td>51.449004</td>
<td>49.325450</td>
<td>45.354162</td>
<td>60.456596</td>
<td>48.041649</td>
<td>47.265209</td>
<td>...</td>
<td>43.416587</td>
<td>45.912961</td>
<td>53.272882</td>
<td>52.449976</td>
<td>46.324939</td>
<td>41.940434</td>
<td>0.000000</td>
<td>46.957428</td>
<td>42.438190</td>
<td>46.465041</td>
</tr>
<tr>
<th>1794</th>
<td>50.378567</td>
<td>38.587563</td>
<td>38.340579</td>
<td>49.527770</td>
<td>46.882833</td>
<td>46.904158</td>
<td>33.466401</td>
<td>54.516053</td>
<td>34.885527</td>
<td>49.929951</td>
<td>...</td>
<td>45.077711</td>
<td>46.421978</td>
<td>33.896903</td>
<td>29.189039</td>
<td>42.602817</td>
<td>46.151923</td>
<td>46.957428</td>
<td>0.000000</td>
<td>44.158804</td>
<td>28.879058</td>
</tr>
<tr>
<th>1795</th>
<td>37.067506</td>
<td>48.569538</td>
<td>50.774009</td>
<td>44.147480</td>
<td>52.668776</td>
<td>32.557641</td>
<td>48.207883</td>
<td>55.928526</td>
<td>37.000000</td>
<td>28.827071</td>
<td>...</td>
<td>38.183766</td>
<td>50.507425</td>
<td>54.359912</td>
<td>47.265209</td>
<td>48.754487</td>
<td>23.537205</td>
<td>42.438190</td>
<td>44.158804</td>
<td>0.000000</td>
<td>39.420807</td>
</tr>
<tr>
<th>1796</th>
<td>47.031904</td>
<td>50.328918</td>
<td>43.954522</td>
<td>41.267421</td>
<td>50.970580</td>
<td>38.496753</td>
<td>40.224371</td>
<td>56.267220</td>
<td>28.337255</td>
<td>40.926764</td>
<td>...</td>
<td>38.288379</td>
<td>50.941143</td>
<td>38.820098</td>
<td>38.600518</td>
<td>49.223978</td>
<td>40.963398</td>
<td>46.465041</td>
<td>28.879058</td>
<td>39.420807</td>
<td>0.000000</td>
</tr>
</tbody>
</table>
<p>1797 rows × 1797 columns</p>
</div>
While centroids are not elements of the dataset, we can also compute the distance between each element to each centroid.
```python
CC.data_every_element_distance_to_centroids
```
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
<th>9</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>14.013361</td>
<td>47.567376</td>
<td>43.896678</td>
<td>39.554151</td>
<td>40.407399</td>
<td>36.647929</td>
<td>41.599287</td>
<td>43.074401</td>
<td>37.369109</td>
<td>32.423583</td>
</tr>
<tr>
<th>1</th>
<td>54.059820</td>
<td>19.017525</td>
<td>38.701490</td>
<td>42.313696</td>
<td>38.269485</td>
<td>42.273369</td>
<td>44.388144</td>
<td>40.861554</td>
<td>33.800663</td>
<td>44.312148</td>
</tr>
<tr>
<th>2</th>
<td>47.757029</td>
<td>32.206345</td>
<td>37.375370</td>
<td>45.438311</td>
<td>43.187064</td>
<td>50.233787</td>
<td>43.272912</td>
<td>41.584089</td>
<td>33.846710</td>
<td>45.754658</td>
</tr>
<tr>
<th>3</th>
<td>44.250476</td>
<td>36.468356</td>
<td>33.283540</td>
<td>22.386098</td>
<td>48.069136</td>
<td>36.198086</td>
<td>41.894212</td>
<td>45.404349</td>
<td>36.063988</td>
<td>31.605201</td>
</tr>
<tr>
<th>4</th>
<td>45.592148</td>
<td>39.322928</td>
<td>52.408033</td>
<td>51.138040</td>
<td>28.340976</td>
<td>48.653228</td>
<td>39.984571</td>
<td>48.264247</td>
<td>44.208386</td>
<td>48.142841</td>
</tr>
<tr>
<th>...</th>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
<td>...</td>
</tr>
<tr>
<th>1792</th>
<td>34.293071</td>
<td>41.151239</td>
<td>43.677849</td>
<td>30.575459</td>
<td>45.769008</td>
<td>36.516206</td>
<td>46.428263</td>
<td>42.576472</td>
<td>33.327387</td>
<td>16.959423</td>
</tr>
<tr>
<th>1793</th>
<td>20.429465</td>
<td>48.646926</td>
<td>47.491512</td>
<td>45.995613</td>
<td>40.876460</td>
<td>40.516369</td>
<td>41.939685</td>
<td>46.740119</td>
<td>40.285358</td>
<td>40.804954</td>
</tr>
<tr>
<th>1794</th>
<td>44.631741</td>
<td>29.885611</td>
<td>38.886808</td>
<td>43.396579</td>
<td>36.316532</td>
<td>42.594489</td>
<td>37.825320</td>
<td>42.725794</td>
<td>25.598846</td>
<td>42.437926</td>
</tr>
<tr>
<th>1795</th>
<td>34.565247</td>
<td>39.389382</td>
<td>43.806621</td>
<td>35.557630</td>
<td>41.311856</td>
<td>38.202818</td>
<td>41.673728</td>
<td>42.664164</td>
<td>32.926630</td>
<td>25.207579</td>
</tr>
<tr>
<th>1796</th>
<td>41.031409</td>
<td>37.724803</td>
<td>37.444086</td>
<td>36.772758</td>
<td>42.390657</td>
<td>40.799218</td>
<td>33.921312</td>
<td>45.775584</td>
<td>28.071988</td>
<td>35.917805</td>
</tr>
</tbody>
</table>
<p>1797 rows × 10 columns</p>
</div>
It is possible to generate a matrix of intercentroid distance.
If (xi,j)i,j is the returned matrix, then xi,j is the distance between centroid of cluster i to centroid of cluster j.
These distances are not related to points of the dataset. We put NaN into the diagonal terms in order to facilitate the manipulation of min/max.
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
<th>9</th>
</tr>
</thead>
<tbody>
<tr>
<th>0</th>
<td>nan</td>
<td>42.026024</td>
<td>39.274919</td>
<td>37.062579</td>
<td>35.981220</td>
<td>34.078029</td>
<td>34.274506</td>
<td>41.772576</td>
<td>32.909593</td>
<td>29.617374</td>
</tr>
<tr>
<th>1</th>
<td>42.026024</td>
<td>nan</td>
<td>28.949723</td>
<td>31.742287</td>
<td>28.674700</td>
<td>32.469295</td>
<td>34.570287</td>
<td>31.187817</td>
<td>20.950348</td>
<td>32.126942</td>
</tr>
<tr>
<th>2</th>
<td>39.274919</td>
<td>28.949723</td>
<td>nan</td>
<td>26.489600</td>
<td>42.689686</td>
<td>32.375712</td>
<td>36.657425</td>
<td>35.570382</td>
<td>25.605848</td>
<td>32.960968</td>
</tr>
<tr>
<th>3</th>
<td>37.062579</td>
<td>31.742287</td>
<td>26.489600</td>
<td>nan</td>
<td>43.499594</td>
<td>29.822474</td>
<td>41.152654</td>
<td>33.369483</td>
<td>25.511462</td>
<td>21.103269</td>
</tr>
<tr>
<th>4</th>
<td>35.981220</td>
<td>28.674700</td>
<td>42.689686</td>
<td>43.499594</td>
<td>nan</td>
<td>35.577158</td>
<td>30.756650</td>
<td>33.444921</td>
<td>31.858925</td>
<td>38.689544</td>
</tr>
<tr>
<th>5</th>
<td>34.078029</td>
<td>32.469295</td>
<td>32.375712</td>
<td>29.822474</td>
<td>35.577158</td>
<td>nan</td>
<td>35.573804</td>
<td>32.098017</td>
<td>25.867262</td>
<td>28.060732</td>
</tr>
<tr>
<th>6</th>
<td>34.274506</td>
<td>34.570287</td>
<td>36.657425</td>
<td>41.152654</td>
<td>30.756650</td>
<td>35.573804</td>
<td>nan</td>
<td>43.514148</td>
<td>31.227114</td>
<td>39.306699</td>
</tr>
<tr>
<th>7</th>
<td>41.772576</td>
<td>31.187817</td>
<td>35.570382</td>
<td>33.369483</td>
<td>33.444921</td>
<td>32.098017</td>
<td>43.514148</td>
<td>nan</td>
<td>27.364089</td>
<td>33.513179</td>
</tr>
<tr>
<th>8</th>
<td>32.909593</td>
<td>20.950348</td>
<td>25.605848</td>
<td>25.511462</td>
<td>31.858925</td>
<td>25.867262</td>
<td>31.227114</td>
<td>27.364089</td>
<td>nan</td>
<td>24.630553</td>
</tr>
<tr>
<th>9</th>
<td>29.617374</td>
<td>32.126942</td>
<td>32.960968</td>
<td>21.103269</td>
<td>38.689544</td>
<td>28.060732</td>
<td>39.306699</td>
<td>33.513179</td>
<td>24.630553</td>
<td>nan</td>
</tr>
</tbody>
</table>
</div>
## Scores <a name="Score"></a>
There are many indices that allow users to evaluate the quality of clusters, such as internal cluster validation indices. In Python development, some libraries compute such scores, but it is not completely done. In this library, these scores have been implemented :
- Total dispersion matrix
- Within cluster dispersion matrixes
- Between group dispersion matrix
- Total sum square
- Pooled within cluster dispersion
The implemented indexes are :
- Ball-Hall Index
- Dunn Index
- Generalized Dunn Indexes (18 indexes)
- C Index
- Banfeld-Raftery Index
- Davies-Bouldin Index
- Calinski-Harabasz Index
- Ray-Turi Index
- Xie-Beni Index
- Ratkowsky Lance Index
- SD Index
- Mclain Rao Index
- Scott-Symons Index
- PBM Index
- Point biserial Index
- Det Ratio Index
- Log SumSquare Ratio Index
- Silhouette Index (computed with scikit-learn)
- Wemmert-Gançarski Index (Thanks to M.Gançarski for this intership)
Main reference for all these scores :
Clustering Indices
Bernard Desgraupes, University Paris Ouest - Lab Modal’X , November 2017
https://cran.r-project.org/web/packages/clusterCrit/vignettes/clusterCrit.pdf
#### In this library, there are two types of methods to calculate these scores: Using IndexCore which automatically caches the already calculated indexes or calculating directly using the "score_index_" methods. The second method can make the calculation of the same index repetitive, which can be very slow because we know that some of these indexes have a very high computational complexity. ####
### First method using IndexCore (faster) ###
```python
CC.compute_every_index()
```
{'general': {'max': {'Between-group total dispersion': 908297.1736053203,
'Mean quadratic error': 696.0267765360618,
'Silhouette Index': 0.16294320522575195,
'Dunn Index': 0.25897601382124175,
'Generalized Dunn Indexes': {'GDI (1, 1)': 0.25897601382124175,
'GDI (1, 2)': 0.9076143747196692,
'GDI (1, 3)': 0.3158503201955148,
'GDI (2, 1)': 0.25897601382124175,
'GDI (2, 2)': 0.9076143747196692,
'GDI (2, 3)': 0.3158503201955148,
'GDI (3, 1)': 0.5790691834873279,
'GDI (3, 2)': 2.0294215944379173,
'GDI (3, 3)': 0.7062398726473335,
'GDI (4, 1)': 0.2875582147985151,
'GDI (4, 2)': 1.0077843328765126,
'GDI (4, 3)': 0.35070952278095474,
'GDI (5, 1)': 0.28515682596025516,
'GDI (5, 2)': 0.9993683603053982,
'GDI (5, 3)': 0.34778075952490317,
'GDI (6, 1)': 0.6033066382644287,
'GDI (6, 2)': 2.1143648370097905,
'GDI (6, 3)': 0.735800169522378},
'Wemmert-Gancarski Index': 0.2502241827215019,
'Calinski-Harabasz Index': 144.1902786959258,
'Ratkowsky-Lance Index': nan,
'Point Biserial Index': -4.064966952313242,
'PBM Index': 34.22417733472788},
'max diff': {'Trace WiB Index': nan, 'Trace W Index': 1250760.117435303},
'min': {'Banfeld-Raftery Index': 11718.207536490032,
'Ball Hall Index': 695.801129352618,
'C Index': 0.1476415026698158,
'Ray-Turi Index': 1.5857819700225737,
'Xie-Beni Index': 1.9551313947642188,
'Davies Bouldin Index': 2.1517097380390937,
'SD Index': [array([0.627482, 0.070384])],
'Mclain-Rao Index': 0.7267985756237975,
'Scott-Symons Index': nan},
'min diff': {'Det Ratio Index': nan,
'Log BGSS/WGSS Index': -0.3199351306684197,
'S_Dbw Index': nan,
'Nlog Det Ratio Index': nan}},
'clusters': {'max': {'Centroid distance to barycenter': [26.422334274375757,
20.184062405495773,
22.958470492954795,
21.71559561353746,
25.717240507145213,
20.283308612864644,
26.419951469008378,
24.426658073844308,
13.44306158441342,
19.876908956223936],
'Between-group Dispersion': [124268.87523421964,
74146.1402843885,
93295.17202553005,
86296.77799167577,
119709.13913372548,
74877.09470781704,
126340.5042480812,
106802.4308135141,
31444.567428645732,
71116.47173772276],
'Average Silhouette': [0.3608993843537291,
0.05227459502398472,
0.14407593888502124,
0.15076708301431302,
0.16517001390130848,
0.1194825125348905,
0.28763816949713245,
0.19373598833558672,
0.08488231267929798,
0.07117051617968871],
'KernelDensity mean': [-87.26207798353086,
-102.79627948741418,
-118.2807433740146,
-102.80193279131969,
-102.79094365877583,
-102.79645332546204,
-87.27879146450985,
-102.77983243274437,
-118.2636521439672,
-118.29755528330563],
'Ball Hall Index': [396.35042923873254,
940.6359437266029,
751.2059752944557,
633.6276389262146,
736.2863160465186,
757.3853701243812,
512.8915478770488,
734.7467931712492,
741.1588717135685,
753.7224074074073]},
'min': {'Within-Cluster Dispersion': [70550.3764044944,
171195.74175824173,
132963.45762711865,
115953.85792349727,
133267.82320441987,
137844.13736263738,
92833.37016574583,
131519.67597765362,
128961.64367816092,
135670.03333333333],
'Largest element distance': [54.543560573178574,
72.85602240034794,
67.0,
62.3377895020348,
71.69379331573968,
66.53570470055908,
61.155539405682624,
67.93379129711516,
61.171888968708494,
63.773035054010094],
'Inter-element mean distance': [27.495251790928528,
41.577045912127325,
37.66525398978789,
34.81272464303223,
37.28558306007683,
38.08288651715454,
31.222158502521683,
37.241230341156786,
37.938810358062234,
37.830620986872184],
'Davies Bouldin Index': array([1.55628353, 2.70948787, 2.09498538, 2.43120015, 1.96455875,
2.09074874, 1.58102612, 1.94811882, 2.70948787, 2.43120015]),
'C Index': [0.15780619270180213,
0.4626045226116365,
0.37889533673771314,
0.31459485530776515,
0.3693066184157008,
0.38636193134197444,
0.23717385124578905,
0.36902306811086555,
0.3857833597084178,
0.3815092165505222]}},
'radius': {'min': {'Radius min': {0: 11.963104233270684,
1: 16.495963249417844,
2: 17.228366828448973,
3: 15.096075210359995,
4: 15.943646753449636,
5: 16.46455777853301,
6: 12.786523861254974,
7: 14.61523732739271,
8: 18.374826032773953,
9: 16.317673899226},
'Radius mean': {0: 19.364954,
1: 29.868519,
2: 26.747682,
3: 24.578193,
4: 26.464614,
5: 27.18575,
6: 22.162453,
7: 26.412302,
8: 26.896195,
9: 26.728077},
'Radius median': {0: 19.090152,
1: 27.705495,
2: 25.299287,
3: 23.495162,
4: 26.434238,
5: 27.194139,
6: 21.579562,
7: 25.358031,
8: 26.982504,
9: 25.201186},
'Radius 75th Percentile': {0: 22.142983,
1: 35.627396,
2: 30.263862,
3: 27.808539,
4: 29.727508,
5: 29.221274,
6: 24.736136,
7: 30.21675,
8: 30.137334,
9: 29.966933},
'Radius max': {0: 35.381597,
1: 48.76808,
2: 48.6619,
3: 40.02036,
4: 51.535976,
5: 40.584931,
6: 42.250871,
7: 44.424333,
8: 38.175815,
9: 45.985382}}}}
We can take the corresponding code in the indices.json file with this call
```python
CC._get_all_index
```
`
{'general': {'max': {'Between-group total dispersion': 'G-Max-01',
'Mean quadratic error': 'G-Max-02',
'Silhouette Index': 'G-Max-03',
'Dunn Index': 'G-Max-04',
'Generalized Dunn Indexes': 'G-Max-GDI',
'Wemmert-Gancarski Index': 'G-Max-05',
'Calinski-Harabasz Index': 'G-Max-06',
'Ratkowsky-Lance Index': 'G-Max-07',
'Point Biserial Index': 'G-Max-08',
'PBM Index': 'G-Max-09'},
'max diff': {'Trace WiB Index': 'G-MaxD-01', 'Trace W Index': 'G-MaxD-02'},
'min': {'Banfeld-Raftery Index': 'G-Min-01',
'Ball Hall Index': 'G-Min-02',
'C Index': 'G-Min-03',
'Ray-Turi Index': 'G-Min-04',
'Xie-Beni Index': 'G-Min-05',
'Davies Bouldin Index': 'G-Min-06',
'SD Index': 'G-Min-07',
'Mclain-Rao Index': 'G-Min-08',
'Scott-Symons Index': 'G-Min-09'},
'min diff': {'Det Ratio Index': 'G-MinD-01',
'Log BGSS/WGSS Index': 'G-MinD-02',
'S_Dbw Index': 'G-MinD-03',
'Nlog Det Ratio Index': 'G-MinD-04'}},
'clusters': {'max': {'Centroid distance to barycenter': 'C-Max-01',
'Between-group Dispersion': 'C-Max-02',
'Average Silhouette': 'C-Max-03',
'KernelDensity mean': 'C-Max-04',
'Ball Hall Index': 'C-Max-05'},
'min': {'Within-Cluster Dispersion': 'C-Min-01',
'Largest element distance': 'C-Min-02',
'Inter-element mean distance': 'C-Min-03',
'Davies Bouldin Index': 'C-Min-04',
'C Index': 'C-Min-05'}},
'radius': {'min': {'Radius min': 'R-Min-01',
'Radius mean': 'R-Min-02',
'Radius median': 'R-Min-03',
'Radius 75th Percentile': 'R-Min-04',
'Radius max': 'R-Min-05'}}}
`
These codes are usefull when you want to generate a single index using IndexCore :
```python
CC.generate_output_by_info_type("general", "max", "G-Max-01")
908297.1736053203
```
### Second method using "score_index_" methods ###
```python
CC.score_between_group_dispersion()
908297.1736053203
```
Make the same result as above but it computes a second time the same score.
### Speed test of different scores
```
pd_df :
shape - (1797, 65)
total elements=116805
Columns types:
pd_df.dtypes.value_counts() : 64 x float64 + 1 x Int32
score_index_ball_hall
5.06 ms ± 79.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_banfeld_Raftery
5.02 ms ± 39.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_c
104 ms ± 550 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_c_for_each_cluster
95.5 ms ± 720 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_calinski_harabasz
16.5 ms ± 257 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_davies_bouldin
12.3 ms ± 76.5 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_davies_bouldin_for_each_cluster
12.3 ms ± 300 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_det_ratio
181 ms ± 4.09 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
score_index_dunn
19.6 ms ± 1.06 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_generalized_dunn_matrix
994 ms ± 41.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
score_index_Log_Det_ratio
180 ms ± 2.86 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_log_ss_ratio
16.3 ms ± 249 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_mclain_rao
63.5 ms ± 6.55 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_PBM
23.5 ms ± 3.84 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_point_biserial
50.3 ms ± 434 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_ratkowsky_lance
12.3 ms ± 254 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_ray_turi
23.2 ms ± 889 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_scott_symons
153 ms ± 6.25 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_SD
211 ms ± 4.37 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
score_index_trace_WiB
138 ms ± 2.69 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
score_index_wemmert_gancarski
8.13 ms ± 93 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
score_index_xie_beni
85.8 ms ± 1.31 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
```
## Confusion Hypersphere <a name="CH"></a>
The confusion hypersphere subclass counts the number of element contained inside a n-dim sphere (hypersphere) of given radius and centered on each cluster centroid.
The given radius is the same for each hypersphere.
Args : "counting_type=" : ('including' or 'excluding') - If including, then the elements belonging cluster i and contained inside the hypersphere of centroid i are counted (for i=j). If excluding, then they're not counted.
"proportion=" : (bool) Return the proportion of element. Default option = False.
#### self.confusion_hypersphere_matrix
```python
CC.confusion_hypersphere_matrix(radius=35, counting_type="including", proportion=True)
```
<div>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>C:0</th>
<th>C:1</th>
<th>C:2</th>
<th>C:3</th>
<th>C:4</th>
<th>C:5</th>
<th>C:6</th>
<th>C:7</th>
<th>C:8</th>
<th>C:9</th>
</tr>
</thead>
<tbody>
<tr>
<th>H:0</th>
<td>0.994382</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.010929</td>
<td>0.000000</td>
<td>0.032967</td>
<td>0.060773</td>
<td>0.000000</td>
<td>0.005747</td>
<td>0.211111</td>
</tr>
<tr>
<th>H:1</th>
<td>0.000000</td>
<td>0.736264</td>
<td>0.090395</td>
<td>0.103825</td>
<td>0.187845</td>
<td>0.016484</td>
<td>0.110497</td>
<td>0.055866</td>
<td>0.574713</td>
<td>0.022222</td>
</tr>
<tr>
<th>H:2</th>
<td>0.000000</td>
<td>0.142857</td>
<td>0.881356</td>
<td>0.355191</td>
<td>0.000000</td>
<td>0.005495</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.310345</td>
<td>0.000000</td>
</tr>
<tr>
<th>H:3</th>
<td>0.000000</td>
<td>0.005495</td>
<td>0.225989</td>
<td>0.950820</td>
<td>0.000000</td>
<td>0.258242</td>
<td>0.000000</td>
<td>0.016760</td>
<td>0.327586</td>
<td>0.666667</td>
</tr>
<tr>
<th>H:4</th>
<td>0.050562</td>
<td>0.032967</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.928177</td>
<td>0.027473</td>
<td>0.154696</td>
<td>0.027933</td>
<td>0.028736</td>
<td>0.000000</td>
</tr>
<tr>
<th>H:5</th>
<td>0.095506</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.103825</td>
<td>0.005525</td>
<td>0.950549</td>
<td>0.022099</td>
<td>0.005587</td>
<td>0.293103</td>
<td>0.133333</td>
</tr>
<tr>
<th>H:6</th>
<td>0.089888</td>
<td>0.027473</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.033149</td>
<td>0.021978</td>
<td>0.983425</td>
<td>0.000000</td>
<td>0.068966</td>
<td>0.000000</td>
</tr>
<tr>
<th>H:7</th>
<td>0.000000</td>
<td>0.071429</td>
<td>0.028249</td>
<td>0.071038</td>
<td>0.060773</td>
<td>0.005495</td>
<td>0.000000</td>
<td>0.882682</td>
<td>0.201149</td>
<td>0.055556</td>
</tr>
<tr>
<th>H:8</th>
<td>0.044944</td>
<td>0.423077</td>
<td>0.293785</td>
<td>0.431694</td>
<td>0.011050</td>
<td>0.170330</td>
<td>0.110497</td>
<td>0.184358</td>
<td>0.977011</td>
<td>0.394444</td>
</tr>
<tr>
<th>H:9</th>
<td>0.421348</td>
<td>0.000000</td>
<td>0.005650</td>
<td>0.759563</td>
<td>0.000000</td>
<td>0.351648</td>
<td>0.000000</td>
<td>0.000000</td>
<td>0.356322</td>
<td>0.872222</td>
</tr>
</tbody>
</table>
</div>
To interpret this, if (xi,j)i,j is the returned matrix, then xi,j is the number of elements belonging cluster j that are contained inside the hypersphere with given radius centered on centroid of cluster i . If proportion is on True, then the number of elements becomes the proportion of elements belonging cluster j.
#### self.confusion_hypersphere_for_linspace_radius_each_element
This method returns the results of the above method for a linear radius space. "n_pts=" allows users to set the radius range.
```python
CC.confusion_hypersphere_for_linspace_radius_each_element(radius=35, counting_type="excluding", n_pts=10)
```
<div>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
<th>9</th>
</tr>
<tr>
<th>Radius</th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
</tr>
</thead>
<tbody>
<tr>
<th>0.0000</th>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>7.1578</th>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>14.3155</th>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>21.4733</th>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
<td>0</td>
</tr>
<tr>
<th>28.6311</th>
<td>3</td>
<td>10</td>
<td>3</td>
<td>36</td>
<td>1</td>
<td>24</td>
<td>1</td>
<td>0</td>
<td>34</td>
<td>30</td>
</tr>
<tr>
<th>35.7889</th>
<td>192</td>
<td>171</td>
<td>161</td>
<td>398</td>
<td>71</td>
<td>211</td>
<td>122</td>
<td>68</td>
<td>473</td>
<td>325</td>
</tr>
<tr>
<th>42.9466</th>
<td>1004</td>
<td>747</td>
<td>802</td>
<td>1023</td>
<td>641</td>
<td>940</td>
<td>837</td>
<td>765</td>
<td>1285</td>
<td>950</td>
</tr>
<tr>
<th>50.1044</th>
<td>1567</td>
<td>1346</td>
<td>1397</td>
<td>1470</td>
<td>1318</td>
<td>1536</td>
<td>1534</td>
<td>1369</td>
<td>1558</td>
<td>1479</td>
</tr>
<tr>
<th>57.2622</th>
<td>1602</td>
<td>1625</td>
<td>1589</td>
<td>1636</td>
<td>1624</td>
<td>1638</td>
<td>1629</td>
<td>1603</td>
<td>1566</td>
<td>1614</td>
</tr>
<tr>
<th>64.4200</th>
<td>1602</td>
<td>1638</td>
<td>1593</td>
<td>1647</td>
<td>1629</td>
<td>1638</td>
<td>1629</td>
<td>1611</td>
<td>1566</td>
<td>1620</td>
</tr>
</tbody>
</table>
</div>
#### confusion_hyperphere_around_specific_point_for_two_clusters
This method returns the number of elements belonging given Cluster1 or given Cluster2 that are contained inside the hypersphere of given radius and centered on given Point.
```python
Point= CC.data_features.iloc[0] #Choose an observation of the dataset
Cluster1= CC.labels_clusters[0] #Choose the cluster 1
Cluster2=CC.labels_clusters[1] #Choose the cluster 2
radius=110 #Large radius to capture the total of both clusters, the result should be the sum of data_clusters[Cluster1] and data_clusters[Cluster2] cardinals
CC.confusion_hyperphere_around_specific_point_for_two_clusters(Point,Cluster1,Cluster2, radius)
```
0 360
dtype: int64
360 elements belonging Cluster or Cluster2 are contained inside this hypersphere.
## Info <a name="Info"></a>
The Info subclass shows two different informative boards that gives many kinds of informations about the general dataset and the clusters.
The type column can be : "max", "min", "max diff", "min diff". If 'max' (respect. 'min'), then higher (respect. lower) is the score, the better is the clustering.
For "max diff" and "min diff", it is usefull to use them when you need to find the best number of clusters. Max diff will correspond to the maximum difference between clustering 1 with K clusters and clustering 2 with K' clusters (K!=K'). See the Bernard Desgraupes reference for more explanations.
```python
CC.general_info(hide_nan=False)
```
Current NaN Index :
Ratkowsky-Lance Index - G-Max-07
Trace WiB Index - G-MaxD-01
Scott-Symons Index - G-Min-09
Det Ratio Index - G-MinD-01
S_Dbw Index - G-MinD-03
Nlog Det Ratio Index - G-MinD-04
<div>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th></th>
<th>General Informations</th>
</tr>
</thead>
<tbody>
<tr>
<th>Between-group total dispersion</th>
<th>max</th>
<td>908297.173605</td>
</tr>
<tr>
<th>Mean quadratic error</th>
<th>max</th>
<td>696.026777</td>
</tr>
<tr>
<th>Silhouette Index</th>
<th>max</th>
<td>0.162943</td>
</tr>
<tr>
<th>Dunn Index</th>
<th>max</th>
<td>0.258976</td>
</tr>
<tr>
<th>Wemmert-Gancarski Index</th>
<th>max</th>
<td>0.250224</td>
</tr>
<tr>
<th>Calinski-Harabasz Index</th>
<th>max</th>
<td>144.190279</td>
</tr>
<tr>
<th>Point Biserial Index</th>
<th>max</th>
<td>-4.064967</td>
</tr>
<tr>
<th>PBM Index</th>
<th>max</th>
<td>34.224177</td>
</tr>
<tr>
<th>Trace W Index</th>
<th>max diff</th>
<td>1250760.117435</td>
</tr>
<tr>
<th>Banfeld-Raftery Index</th>
<th>min</th>
<td>11718.207536</td>
</tr>
<tr>
<th>Ball Hall Index</th>
<th>min</th>
<td>695.801129</td>
</tr>
<tr>
<th>C Index</th>
<th>min</th>
<td>0.147642</td>
</tr>
<tr>
<th>Ray-Turi Index</th>
<th>min</th>
<td>1.585782</td>
</tr>
<tr>
<th>Xie-Beni Index</th>
<th>min</th>
<td>1.955131</td>
</tr>
<tr>
<th>Davies Bouldin Index</th>
<th>min</th>
<td>2.15171</td>
</tr>
<tr>
<th>SD Index</th>
<th>min</th>
<td>[[0.627482, 0.070384]]</td>
</tr>
<tr>
<th>Mclain-Rao Index</th>
<th>min</th>
<td>0.726799</td>
</tr>
<tr>
<th>Log BGSS/WGSS Index</th>
<th>min diff</th>
<td>-0.319935</td>
</tr>
<tr>
<th>GDI (1, 1)</th>
<th>max</th>
<td>0.258976</td>
</tr>
<tr>
<th>GDI (1, 2)</th>
<th>max</th>
<td>0.907614</td>
</tr>
<tr>
<th>GDI (1, 3)</th>
<th>max</th>
<td>0.31585</td>
</tr>
<tr>
<th>GDI (2, 1)</th>
<th>max</th>
<td>0.258976</td>
</tr>
<tr>
<th>GDI (2, 2)</th>
<th>max</th>
<td>0.907614</td>
</tr>
<tr>
<th>GDI (2, 3)</th>
<th>max</th>
<td>0.31585</td>
</tr>
<tr>
<th>GDI (3, 1)</th>
<th>max</th>
<td>0.579069</td>
</tr>
<tr>
<th>GDI (3, 2)</th>
<th>max</th>
<td>2.029422</td>
</tr>
<tr>
<th>GDI (3, 3)</th>
<th>max</th>
<td>0.70624</td>
</tr>
<tr>
<th>GDI (4, 1)</th>
<th>max</th>
<td>0.287558</td>
</tr>
<tr>
<th>GDI (4, 2)</th>
<th>max</th>
<td>1.007784</td>
</tr>
<tr>
<th>GDI (4, 3)</th>
<th>max</th>
<td>0.35071</td>
</tr>
<tr>
<th>GDI (5, 1)</th>
<th>max</th>
<td>0.285157</td>
</tr>
<tr>
<th>GDI (5, 2)</th>
<th>max</th>
<td>0.999368</td>
</tr>
<tr>
<th>GDI (5, 3)</th>
<th>max</th>
<td>0.347781</td>
</tr>
<tr>
<th>GDI (6, 1)</th>
<th>max</th>
<td>0.603307</td>
</tr>
<tr>
<th>GDI (6, 2)</th>
<th>max</th>
<td>2.114365</td>
</tr>
<tr>
<th>GDI (6, 3)</th>
<th>max</th>
<td>0.7358</td>
</tr>
</tbody>
</table>
</div>
```python
CC.clusters_info
```
<div>
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th></th>
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
<th>6</th>
<th>7</th>
<th>8</th>
<th>9</th>
</tr>
<tr>
<th>index</th>
<th>Type</th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
<th></th>
</tr>
</thead>
<tbody>
<tr>
<th>Centroid distance to barycenter</th>
<th>max</th>
<td>26.42</td>
<td>20.18</td>
<td>22.95</td>
<td>21.71</td>
<td>25.71</td>
<td>20.28</td>
<td>26.41</td>
<td>24.42</td>
<td>13.44</td>
<td>19.87</td>
</tr>
<tr>
<th>Between-group Dispersion</th>
<th>max</th>
<td>124268</td>
<td>74146</td>
<td>93295</td>
<td>86296</td>
<td>119709</td>
<td>74877</td>
<td>126340</td>
<td>106802</td>
<td>31444</td>
<td>71116</td>
</tr>
<tr>
<th>Average Silhouette</th>
<th>max</th>
<td>0.36</td>
<td>0.05</td>
<td>0.14</td>
<td>0.15</td>
<td>0.16</td>
<td>0.11</td>
<td>0.28</td>
<td>0.19</td>
<td>0.08</td>
<td>0.07</td>
</tr>
<tr>
<th>KernelDensity mean</th>
<th>max</th>
<td>-87.26</td>
<td>-102.79</td>
<td>-118.28</td>
<td>-102.80</td>
<td>-102.79</td>
<td>-102.79</td>
<td>-87.27</td>
<td>-102.77</td>
<td>-118.26</td>
<td>-118.29</td>
</tr>
<tr>
<th>Ball Hall Index</th>
<th>max</th>
<td>396.35</td>
<td>940.63</td>
<td>751.20</td>
<td>633.62</td>
<td>736.28</td>
<td>757.38</td>
<td>512.89</td>
<td>734.74</td>
<td>741.15</td>
<td>753.72</td>
</tr>
<tr>
<th>Within-Cluster Dispersion</th>
<th>min</th>
<td>70550</td>
<td>171195</td>
<td>132963</td>
<td>115953</td>
<td>133267</td>
<td>137844</td>
<td>92833</td>
<td>131519</td>
<td>128961</td>
<td>135670</td>
</tr>
<tr>
<th>Largest element distance</th>
<th>min</th>
<td>54.54</td>
<td>72.85</td>
<td>67.00</td>
<td>62.33</td>
<td>71.69</td>
<td>66.53</td>
<td>61.15</td>
<td>67.93</td>
<td>61.17</td>
<td>63.77</td>
</tr>
<tr>
<th>Inter-element mean distance</th>
<th>min</th>
<td>27.49</td>
<td>41.57</td>
<td>37.66</td>
<td>34.81</td>
<td>37.28</td>
<td>38.08</td>
<td>31.22</td>
<td>37.24</td>
<td>37.93</td>
<td>37.83</td>
</tr>
<tr>
<th>Davies Bouldin Index</th>
<th>min</th>
<td>1.55</td>
<td>2.70</td>
<td>2.09</td>
<td>2.43</td>
<td>1.96</td>
<td>2.09</td>
<td>1.58</td>
<td>1.94</td>
<td>2.70</td>
<td>2.43</td>
</tr>
<tr>
<th>C Index</th>
<th>min</th>
<td>0.15</td>
<td>0.46</td>
<td>0.37</td>
<td>0.31</td>
<td>0.36</td>
<td>0.38</td>
<td>0.23</td>
<td>0.36</td>
<td>0.38</td>
<td>0.38</td>
</tr>
<tr>
<th>Radius min</th>
<th>min</th>
<td>11.96</td>
<td>16.49</td>
<td>17.22</td>
<td>15.09</td>
<td>15.94</td>
<td>16.46</td>
<td>12.78</td>
<td>14.61</td>
<td>18.37</td>
<td>16.31</td>
</tr>
<tr>
<th>Radius mean</th>
<th>min</th>
<td>19.36</td>
<td>29.86</td>
<td>26.74</td>
<td>24.57</td>
<td>26.46</td>
<td>27.18</td>
<td>22.16</td>
<td>26.41</td>
<td>26.89</td>
<td>26.72</td>
</tr>
<tr>
<th>Radius median</th>
<th>min</th>
<td>19.09</td>
<td>27.70</td>
<td>25.29</td>
<td>23.49</td>
<td>26.43</td>
<td>27.19</td>
<td>21.57</td>
<td>25.35</td>
<td>26.98</td>
<td>25.20</td>
</tr>
<tr>
<th>Radius 75th Percentile</th>
<th>min</th>
<td>22.14</td>
<td>35.62</td>
<td>30.26</td>
<td>27.80</td>
<td>29.72</td>
<td>29.22</td>
<td>24.73</td>
<td>30.21</td>
<td>30.13</td>
<td>29.96</td>
</tr>
<tr>
<th>Radius max</th>
<th>min</th>
<td>35.38</td>
<td>48.76</td>
<td>48.66</td>
<td>40.02</td>
<td>51.53</td>
<td>40.58</td>
<td>42.25</td>
<td>44.42</td>
<td>38.17</td>
<td>45.98</td>
</tr>
</tbody>
</table>
</div>
## Density <a name="Density"></a>
The Density subclass is based on projection 2D or 3D using dimensionnality reductors such as PCA or UMAP. As UMAP is only possible in 2D, we will only use PCA for 3D Density graphs. The main idea for approximating density is about summing Gaussian Distribution n-dim laws centered on each dataset point on a meshgrid corresponding to 2D or 3D.
This section returns a lot of data that are packed in a native Python dict. Each element returned (excluding the main return) inside the dict has to be activated by its own argument. See the following example:
#### self.density_projection_2D
Args:
- reduction_method : "PCA" or "UMAP"
- percentile : percentile of density that corresponds to the minimum value to show
- return_data :If True, return 2D PCA Data
- return_clusters_density : If True, return the 2D Grid with the Z values for each cluster
-
```python
CC.density_projection_2D("PCA", 95, return_data=True, return_clusters_density=True)
```
{'Z-Grid': -27.494448 -27.205068 -26.915688 -26.626307 -26.336927 \
-31.169904 0.000000 0.000000 0.000000 0.000000 0.000000
-30.853975 0.000000 0.000000 0.000000 0.000000 0.000000
-30.538045 0.000000 0.000000 0.000000 0.000000 0.000000
-30.222115 0.000000 0.000000 0.000000 0.000000 0.000000
-29.906185 0.000000 0.000000 0.000000 0.000000 0.000000
... ... ... ... ... ...
30.436407 0.000000 0.000000 0.000000 0.000000 0.000000
30.752337 0.000000 0.000000 0.000000 0.000000 0.000000
31.068267 0.000000 0.000000 0.000000 0.000000 0.000000
31.384197 0.000000 0.000000 0.000000 0.000000 0.000000
31.700126 0.000000 0.000000 0.000000 0.000000 0.000000
[200 rows x 200 columns],
'Clusters Density': {0: array([[0.00000000e+000, 0.00000000e+000, 0.00000000e+000, ...,
2.26543202e-251, 6.72200949e-253, 1.80509292e-254],
[0.00000000e+000, 0.00000000e+000, 0.00000000e+000, ...,
2.39644309e-090, 2.99562311e-092, 3.38890645e-094]]),
1: array([[1.22473352e-190, 9.95843640e-187, 7.32812819e-183, ...,
1.89152820e-043, 5.31903131e-045, 1.35364451e-046],
[6.13307159e-189, 4.98686467e-185, 3.66969092e-181, ...,
[2.18683154e-176, 3.42747973e-173, 4.86168499e-170, ...,
6.11416273e-291, 1.45371826e-293, 3.12806562e-296]]),
2: array([[9.82858841e-081, 1.91715088e-079, 4.01121949e-078, ...,
3.95263976e-164, 6.05314934e-167, 8.38934205e-170],
[1.15888672e-078, 2.15624607e-077, 3.99557766e-076, ...,
0.00000000e+000, 0.00000000e+000, 0.00000000e+000],
[2.14277916e-156, 6.99479702e-154, 2.07015420e-151, ...,
0.00000000e+000, 0.00000000e+000, 0.00000000e+000]]),
...
1.34702113e-296, 1.14119413e-299, 8.74977734e-303]]),
8: array([[3.68438507e-120, 1.91948293e-117, 9.05015769e-115, ...,
1.78634585e-151, 1.09968510e-153, 6.12666390e-156],
[1.19395354e-118, 6.22023065e-116, 2.93277042e-113, ...,
[2.30028280e-221, 1.12585950e-218, 4.98917831e-216, ...,
3.82113467e-265, 9.90939747e-269, 2.32570436e-272]]),
9: array([[5.32990773e-136, 8.38778107e-133, 1.19461196e-129, ...,
3.98260675e-172, 2.88650900e-175, 1.89334938e-178],
[5.65560268e-135, 8.90033385e-132, 1.26761124e-128, ...,
[3.33187669e-070, 4.02748126e-069, 4.64453319e-068, ...,
1.02174508e-253, 1.38205190e-256, 1.69183696e-259]])},
'2D PCA Data': PCA0 PCA1
0 -1.259467 21.274883
1 7.957610 -20.768700
... ...
1795 -4.872099 12.423954
1796 -0.344388 6.365550
[1797 rows x 2 columns]}
#### self.density_projection_3D
Use PCA 3D to project the dataset and make a 3D meshgrid to estimate the density on it with the 3D Gaussian distribution.
Args:
- percentile : percentile of density that corresponds to the minimum value to show
- return_grid :If True, return 3D Grid
- return_clusters_density : If True, return the 3D Grid with the A values for each cluster
```python
CC.density_projection_3D(99, return_grid=True, return_clusters_density=True)
```
{'A-Grid': array([[[3.48581797e-15, 1.62080230e-14, 6.90041904e-14, ...,
5.83374041e-13, 1.92066889e-13, 5.70629214e-14],
[6.60425258e-16, 6.17767595e-15, 5.01029852e-14, ...,
3.52400611e-12, 5.50927146e-13, 7.45284376e-14]]]),
'Clusters Density': {0: array([[[3.40385502e-48, 3.36307101e-47, 2.87017113e-46, ...,
6.07579853e-65, 1.97018960e-66, 5.51854095e-68],
[6.12214966e-32, 7.73521170e-31, 8.44230524e-30, ...,
6.56667612e-49, 8.52648274e-51, 9.59653245e-53]],
[[1.30566741e-19, 7.34944932e-19, 3.57377188e-18, ...,
3.11782459e-28, 1.14260012e-29, 3.68476733e-31],
[6.32366666e-22, 6.00865462e-21, 4.93929478e-20, ...,
4.28020934e-27, 2.75837798e-28, 1.53709049e-29]]]),
1: array([[[8.98310922e-30, 6.71213878e-29, 5.03213279e-28, ...,
2.87124806e-29, 8.73589680e-30, 2.29930435e-30]
[5.02339294e-21, 2.49290742e-20, 1.30264666e-19, ...,
2.55638337e-14, 4.29622018e-15, 6.43949192e-16]],
[2.83167266e-35, 8.65300585e-35, 2.28427319e-34, ...,
8.11025209e-33, 1.61201265e-33, 2.82711740e-34]]]),
2: array([[[1.36203887e-28, 9.67318866e-28, 5.96928017e-27, ...,
1.54322295e-14, 5.09928476e-15, 1.48590282e-15],
...
8: array([[[6.84981203e-33, 1.26631274e-31, 2.37110234e-30, ...,
9.64415278e-27, 5.73382414e-28, 3.22330467e-29],
[1.74336209e-34, 9.68068937e-34, 4.64782487e-33, ...,
1.05223775e-37, 1.79602795e-38, 2.64802829e-39]]]),
9: array([[[3.60638003e-23, 1.23523984e-22, 5.16583215e-22, ...,
5.75818352e-33, 1.73152931e-34, 4.49760341e-36],
2.86151947e-52, 3.19445757e-53, 3.40743714e-54]]])},
'3D Grid': {'X': array([[[-37.40388626, -37.40388626, -37.40388626, ..., -37.40388626,
-37.40388626, -37.40388626],
[ 38.04015058, 38.04015058, 38.04015058, ..., 38.04015058,
38.04015058, 38.04015058]]]),
'Y': array([[[-32.99333756, -32.99333756, -32.99333756, ..., -32.99333756,
-32.99333756, -32.99333756],
36.11064515, 36.11064515]]]),
'Z': array([[[-35.1620997 , -33.64347275, -32.12484579, ..., 36.21336725,
37.73199421, 39.25062116],
[-35.1620997 , -33.64347275, -32.12484579, ..., 36.21336725,
37.73199421, 39.25062116]]])}}
## Utils <a name="Utils"></a>
This section uses other modules to apply to the current self object. For example, PCA from scikit-learn is implemented. We also use UMAP from umap-learn. The list for utils methods :
- self.utils_KernelDensity - https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KernelDensity.html#sklearn.neighbors.KernelDensity
- self.utils_PCA - https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html?highlight=pca#sklearn.decomposition.PCA
- self.utils_ts_filtering_STL - https://www.statsmodels.org/devel/generated/statsmodels.tsa.seasonal.STL.html
- self.utils_UMAP - https://umap-learn.readthedocs.io/en/latest/
## Graphs <a name="Graph"></a>
This subsclass uses Plotly to plot to different data computed with the module.
```python
CC.graph_boxplots_distances_to_centroid(0)
```
```python
CC.graph_PCA_3D()
```
```python
CC.graph_reduction_2D("UMAP")
```
```python
CC.graph_reduction_2D("PCA")
```
```python
CC.graph_reduction_density_2D("PCA", 99, "contour")
```
```python
CC.graph_reduction_density_2D("UMAP", 99, "contour")
```
```python
CC.graph_reduction_density_2D("PCA", 99, "interactive")
```
```python
CC.graph_reduction_density_3D(99)
```
```python
CC.graph_reduction_density_3D(99,clusters=[0,1])
```
نیازمندی
| مقدار | نام |
|---|---|
| - | numpy |
| - | pandas |
| - | scipy |
| - | scikit-learn |
| - | plotly |
| - | numba |
| >=0.5.1 | umap-learn |
| - | statsmodels |
| - | Pillow |
زبان مورد نیاز
| مقدار | نام |
|---|---|
| >=3.8 | Python |
نحوه نصب
نصب پکیج whl Clusters-Features-1.0.3:
pip install Clusters-Features-1.0.3.whl
نصب پکیج tar.gz Clusters-Features-1.0.3:
pip install Clusters-Features-1.0.3.tar.gz