معرفی شرکت ها

تبلیغات ما

مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.

مشاهده بیشتر

تبلیغات ما

مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.

مشاهده بیشتر

تبلیغات ما

مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.

مشاهده بیشتر

تبلیغات ما

مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.

مشاهده بیشتر

تبلیغات ما

مشتریان به طور فزاینده ای آنلاین هستند. تبلیغات می تواند به آنها کمک کند تا کسب و کار شما را پیدا کنند.

مشاهده بیشتر

توضیحات

Aruana is a collection of methods that can be used for simple NLP tasks and for machine learning text preprocessing.

ویژگی	مقدار
سیستم عامل	-
نام فایل	Aruana-1.1.1
نام	Aruana
نسخه کتابخانه	1.1.1
نگهدارنده	[]
ایمیل نگهدارنده	[]
نویسنده	Nheeng
ایمیل نویسنده	contact@nheeng.com
آدرس صفحه اصلی	https://aruana.nheeng.com
آدرس اینترنتی	https://pypi.org/project/Aruana/
مجوز	Apache

# Aruana NLP for Machine Learning * [Aruana](https://aruana.nheeng.com/) * [Nhe'eng](https://nheeng.com/) Aruana is a collection of methods that can be used for simple NLP tasks. It can be used for tasks involving text preprocessing for machine learning. Aruana works mainly with text strings and lists of strings. The library is developed in Python 3. ## Installing Aruana ### pip $ pip3 install aruana If you want, you can also install Aruana in a virtual environment: $ python -m venv .env $ source .env/bin/activate $ pip3 install aruana ### Prerequisites Aruana uses following external Python libraries: - nltk (3.3) - tqdm (4.19.5) - pdoc (0.5.1) They are all documented in the [requirements.txt](requirements.txt) file. ## Usage examples To use Aruana, initialize it by choosing one of the three available languages ('en', 'fr', 'pt-br') aruana_en = Aruana('en') ### Quick preprocessing Aruana has the preprocess method, which applies commonly used preprocessed steps on you text. text = "At the end of the day, you're solely responsible for your success and your failure. And the sooner you realize that, you accept that, and integrate that into your work ethic, you will start being successful. As long as you blame others for the reason you aren't where you want to be, you will always be a failure." preprocessed_text = aruana_en.preprocess(text) print(preprocessed_text) >>> ['at', 'the', 'end', 'of', 'the', 'day', 'you', 'are', 'sole', 'respons', 'for', 'your', 'success', 'and', 'your', 'failur', 'and', 'the', 'sooner', 'you', 'realiz', 'that', 'you', 'accept', 'that', 'and', 'integr', 'that', 'into', 'your', 'work', 'ethic', 'you', 'will', 'start', 'be', 'success', 'as', 'long', 'as', 'you', 'blame', 'other', 'for', 'the', 'reason', 'you', 'are', 'not', 'where', 'you', 'want', 'to', 'be', 'you', 'will', 'alway', 'be', 'a', 'failur'] If you prefer, you can choose to: - tokenize the sentence - stem it - remove stop words - pos tag the portuguese sentences text = "At the end of the day, you're solely responsible for your success and your failure. And the sooner you realize that, you accept that, and integrate that into your work ethic, you will start being successful. As long as you blame others for the reason you aren't where you want to be, you will always be a failure." preprocessed_text = aruana_en.preprocess(text, stem=False, remove_stopwords=True) print(preprocessed_text) ['end', 'day', 'solely', 'responsible', 'success', 'failure', 'sooner', 'realize', 'that', 'accept', 'that', 'integrate', 'work', 'ethic', 'start', 'successful', 'long', 'blame', 'others', 'reason', 'want', 'be', 'always', 'failure'] ### List preprocessing If you have a list of sentences or you are using Pandas, you can pass the entire list for preprocessing by using the preprocess_list method. list_of_strings = ['I love you', 'Please, never leave me alone', 'If you go, I will die', 'I am watching a lot of romantic comedy lately', 'I have to eat icecream' ] list_processed = aruana_en.preprocess_list(list_of_strings, stem=False, remove_stopwords=True) print(list_processed) >>> [['love'], ['please', 'never', 'leave', 'alone'], ['go', 'die'], ['watching', 'lot', 'romantic', 'comedy', 'lately'], ['eat', 'icecream']] ### Defining your own pipeline Use the single available methods to create a custom pipeline instead of using the quick preprocessing function. text = "At the end of the day, @john you're solely responsible for your #success and your #failure. And the sooner you realize that, you accept that, and integrate that into your work ethic, you will start being #successful." text = aruana_en.lower_remove_white(text) text = aruana_en.expand_contractions(text) text = aruana_en.replace_handles(text, 'HANDLE') text = aruana_en.replace_hashtags(text, 'HASHTAG') text = aruana_en.remove_stopwords(text) text = aruana_en.replace_punctuation(text, placeholder='PUNCTUATION') text = aruana_en.tokenize(text) print(text) >>> ['end', 'day', 'PUNCTUATION', 'HANDLE', 'solely', 'responsible', 'HASHTAG', 'HASHTAG', 'PUNCTUATION', 'sooner', 'realize', 'that', 'PUNCTUATION', 'accept', 'that', 'PUNCTUATION', 'integrate', 'work', 'ethic', 'PUNCTUATION', 'start', 'HASHTAG', 'PUNCTUATION'] ## Development ### Testing 1. Create a clean test environment 2. Navigate to aruana project on your computer and generate a package using bdist_wheel $ python3 setup.py sdist bdist_wheel 3. Install the package $ python3 setup.py install ### Docs Navigate to aruana/aruana and type: $ pdoc --html aruana ### Release Follow the steps below before releasing a new version: 1. Update all necessary documents 2. Generate the package using bdist 3. Install the new version on a clean environment for testing 4. If everything is ok, generate the doc using pdoc ## Contributing Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct, and the process for submitting pull requests to us. ## Versioning Use [SemVer](http://semver.org/) for versioning. ## Authors * **Wilame Vallantin** - *Initial work* - [Nhe'eng](https://nheeng.com/) ## License This project is licensed under the Apache License - see the [LICENSE.md](LICENSE.md) file for details ## V. 1.1.1 ### New features - Adds the random_classification method, useful for random text classification for testing model accuracy - Adds the replace_with_blob method, useful for creating blobs from a corpus for testing method accuracy - adds the strings module, with a list of punctuation and diacritic strings - adds an internal tokenizer - adds a pos-tagger for portuguese (experimental, version 0.0.1) ### Improvements - expand_contractions recognizes now more words for portuguese - Preprocess text now converts emojis to text instead of completely removing them - Removes NLTK tokenizer and replaces it for an internal tokenizer

نحوه نصب

نصب پکیج whl Aruana-1.1.1:

pip install Aruana-1.1.1.whl

نصب پکیج tar.gz Aruana-1.1.1:

pip install Aruana-1.1.1.tar.gz