# fastutils
Collection of simple utils.
## Install
```shell
pip install fastutils
```
## Installed utils
- cipherutils
- AesCipher
- MysqlAesCipher
- RawKeyAesCipher
- RsaCipher
- IvCipher
- IvfCipher
- dictutils
- Object
- deep_merge
- select
- touch
- attrgetorset
- attrset
- update
- ignore_none_item
- change
- changes
- prefix_key
- diff
- fsutils
- mkdir
- rm
- filecopy
- treecopy
- copy
- pathjoin
- readfile
- write
- get_temp_workspace
- rename
- move
- file_content_replace
- touch
- expand
- expands
- first_exists_file
- info
- TemporaryFile
- funcutils
- get_default_values
- get_inject_params
- call_with_inject
- mcall_with_inject
- classproperty
- chain
- BunchCallable
- try_again_on_error
- hashutils
- get_file_hash
- get_file_md5
- get_file_sha
- get_file_sha1
- get_file_sha224
- get_file_sha256
- get_file_sha384
- get_file_sha512
- get_hash_hexdigest
- get_md5
- get_sha
- get_sha1
- get_sha224
- get_sha256
- get_sha384
- get_sha512
- get_hash_base64
- get_md5_base64
- get_sha_base64
- get_sha1_base64
- get_sha224_base64
- get_sha256_base64
- get_sha384_base64
- get_sha512_base64
- get_pbkdf2_hmac
- validate_pbkdf2_hmac
- get_pbkdf2_sha512
- validate_pbkdf2_sha512
- get_pbkdf2_sha384
- validate_pbkdf2_sha384
- get_pbkdf2_sha256
- validate_pbkdf2_sha256
- get_pbkdf2_sha224
- validate_pbkdf2_sha224
- get_pbkdf2_sha1
- validate_pbkdf2_sha1
- get_pbkdf2_sha
- validate_pbkdf2_sha
- get_pbkdf2_md5
- validate_pbkdf2_md5
- httputils
- get_urlinfo
- get_url_filename
- get_url_save_path
- get_sitename
- download
- imageutils
- get_image_bytes
- get_base64image
- parse_base64image
- resize
- resize_to_fixed_width
- resize_to_fixed_height
- jsonutils
- register_global_encoder
- simple_json_dumps
- listutils
- int_list_to_bytes
- pad
- chunk
- clean_none
- ignore_none_element # alias of clean_none
- unique
- replace
- append_new
- group
- compare
- first
- topological_sort
- is_ordered
- list2dict
- compare_execute
- logutils
- get_simple_config
- setup
- msgpackutils
- loads
- dumps
- Msg
- nameutils
- get_random_name
- get_last_names
- get_suggest_first_names
- guess_lastname
- guess_surname # alias of guess_lastname
- numericutils
- binary_decompose
- decimal_change_base
- get_float_part
- float_split
- pinyinutils
- to_pinyin
- randomutils
- Random
- UuidGenerator
- uuid4
- rsautils
- newkeys
- load_private_key
- load_public_key
- load_public_key_from_private_key
- encrypt
- decrypt
- export_key
- sign
- verify
- strutils
- random_string
- char_force_to_int
- force_bytes
- force_text
- force_int
- force_float
- force_numberic
- wholestrip
- split2
- split
- str_composed_by
- is_str_composed_by_the_choices # alias of str_composed_by
- is_hex_digits
- join_lines
- is_urlsafeb64_decodable
- is_base64_decodable
- is_unhexlifiable
- text_display_length
- text_display_shorten
- smart_get_binary_data
- is_chinese_character
- binarify
- unbinarify
- ints2bytes
- int2bytes
- substrings
- combinations2
- combinations
- captital_number
- clean
- do_clean # alias of clean
- camel
- no_mapping
- none_to_empty_string
- strip_string
- format_with_mapping
- unquote
- is_uuid
- stringlist_append
- html_element_css_append
- remove_prefix
- remove_suffix
- encodable
- decodable
- chunk
- get_all_substrings
- sysutils
- get_worker_id
- get_daemon_application_pid
- get_random_script_name
- execute_script
- get_worker_info
- timeutils
- TimeService
- treeutils
- build_tree
- tree_walk
- print_tree_callback
- print_tree
## Help
See pydoc for help.
```
python3 -m pydoc -b # start pydoc
```
## Usage Examples
### fastutils.strutils.chunk
```
from fastutils import strutils
In [33]: strutils.chunk('hello', 3)
Out[33]: ['hel', 'lo']
In [34]: strutils.chunk('hello', 6)
Out[34]: ['hello']
In [35]: strutils.chunk('hello', 5)
Out[35]: ['hello']
```
### fastutils.strutils.get_all_substrings
```
from fastutils import strutils
In [4]: strutils.get_all_substrings('a')
Out[4]: {'a'}
In [5]: strutils.get_all_substrings('ab')
Out[5]: {'a', 'ab', 'b'}
In [6]: strutils.get_all_substrings('abc')
Out[6]: {'a', 'ab', 'abc', 'b', 'bc', 'c'}
In [7]: strutils.get_all_substrings('abcd')
Out[7]: {'a', 'ab', 'abc', 'abcd', 'b', 'bc', 'bcd', 'c', 'cd', 'd'}
```
### fastutils.listutils.list2dict
```
In [1]: from fastutils import listutils
In [2]: listutils.list2dict([1,2,3], ['a', 'b'])
Out[2]: {'a': 1, 'b': 2}
In [3]: listutils.list2dict([1,2,3], ['a', 'b', 'c'])
Out[3]: {'a': 1, 'b': 2, 'c': 3}
In [4]: listutils.list2dict([1,2,3], ['a', 'b', 'c', 'd'])
Out[4]: {'a': 1, 'b': 2, 'c': 3, 'd': None}
```
### fastutils.listutils.compare_execute
*Example 1*
```
from fastutils import listutils
old_list = [1,2,3]
new_list = [2,3,4]
created, deleted, changed = listutils.compare_execute(
old_data=old_list,
new_data=new_list,
create_callback=lambda key, data: data,
delete_callback=lambda key, instance: instance,
change_callback=lambda key, instance, data: instance,
old_key=lambda x: x
)
assert created == [4]
assert deleted == [1]
assert changed == [2, 3]
```
*Example 2*
```
from fastutils import listutils
old_list = [1,2,3]
new_list = [3,4,5]
def do_create(key, data):
if data > 4:
return data
else:
return None
def do_delete(key, instance):
if instance > 1:
return instance
else:
return None
def do_change(key, instance, data):
return None
created, deleted, changed = listutils.compare_execute(
old_data=old_list,
new_data=new_list,
create_callback=do_create,
delete_callback=do_delete,
change_callback=do_change,
old_key=lambda x: x
)
assert created == [5]
assert deleted == [2]
assert changed == []
```
## Notice
- We suggest you use pycryptodome>=3.14.1, lower versions are not tested.
## Releases
### v0.42.14
- Add fastutils.listutils.compare_execute.
- Add fastutils.dictutils.Object.fix to fix the Object instance if you added a dict thing to the Objects' list/dict item.
### v0.42.13
- Add fastutils.strutils.get_all_substrings.
- Add fastutils.listutils.list2dict.
### v0.42.11
- Fix fastutils.cipherutils.CipherBase incompatibility problem.
### v0.43.9
- doc update.
### v0.42.8
- Fix threadutils.Counter incr/decr result value duplicate problem.
### v0.42.7
- Fix logutils formatter missing problem.
### v0.42.6
- Add filelike-methods for fsutils.TemporaryFile.
### v0.42.5
- Add TemporaryFile.open and TemporaryFile.close methods.
### v0.42.4
- Change rsautils.encrypt default envelope method to PKCS1_OAEP, and rsautils.decrypt support both PKCS1_OAEP and PKCS1_v1_5 methods.
- Add rsautils.sign and resautils.verify methods.
### v0.42.3
- Add console_handler_class and file_handler_class parameters for function logutils.get_simple_config.
- Add simple and simple_json formatter in logutils.get_simple_config.
### v0.42.2
- Add sysutils.get_worker_info.
- Fix logutils' implicit imports problem.
- Fix funcutils.classproperty.
### v0.42.1
- Fix logutils problem.
### v0.42.0
- Add msgpackutils.
- Fix problem in nameutils.
### v0.41.1
- Fix numericutils.get_float_part problem.
### v0.41.0
- cipherutils changes.
- Fix license_files missing problem.
### v0.40.2
- Add cipherutils.raw_aes_key.
- Add cipherutils.get_aes_mode.
### v0.40.0
- Add listutils.chunk.
- New threadutils.
- Fix dictutils.Object.
### v0.39.8
- fsutils.TemporaryFile add auto delete.
- dictutils.changes can return changed keys.
- dictutils.change can control do update or not.
- dictutils.change add ignore_empty_value support.
### v0.39.7
- Fix strutils.force_byte, strutils.force_text.
### v0.39.6
- Fix encode_datetime problem.
### v0.39.5
- Add field name while failed to do typing cast.
### v0.39.4
- Add more default jsonutils encoders.
### v0.39.1
- Fix logutils.setup default config for windows.
### v0.39.0
- Change logutils.setup parameter name.
- Fix bizerror.BizError encoding problem in jsonutils.
### v0.38.2
- Add timeutils.TimeService.
### v0.37.11
- Fix fsutils.write.
### v0.37.10
- fsutils.readfile add default params. The default value will be returned if the target file is not exists.
- Add dictutils.diff function.
- Force folder create in fsutils.write.
- Make sure log folder exists in logutils.setup.
- Add sysutils.execute_script.
### v0.37.3
- Add PyYAML deps in requirements.txt.
### v0.37.2
- SimpleProducerConsumerServerBase.on_consume_error function add new parameter: task.
- Fix threadutils problems. Many changes are made.
### v0.37.0
- Add fsutils.load_application_config.
### v0.36.2
- Add fsutils.TemporaryFile
- Add threadutils.SimpleProducerConsumerServer and fix problems in ServiceCore.
### v0.35.0
- strutils.is_uuid add allow_bad_characters parameter, default to False. For some old bad application using '815f5ecb-eg07-43af-8de2-87d3898093b5' as UUID.
- Add nameutils.guess_surname.
- Add fsutils.get_size_deviation, fsutils.get_unit_size and fsutils.get_size_display.
### v0.34.0
- Add sysutils.get_daemon_application_pid.
- Add httputils.
### v0.33.0
- Add fsutils.info
- Add strutils.encodable and strutils.decodable
### v0.32.0
- Add funcutils.try_again_on_error.
### v0.31.0
- Add fsutils.expands.
- Add fsutils.get_application_config_filepath.
### v0.30.1
- Add cipherutils result_encoders' typing tips.
### v0.30.0
- Add strutils.stringlist_append.
- Add strutils.html_element_css_append.
- Add strutils.split2.
- Add strutils.remove_prefix.
- Add strutils.remove_suffix.
- Change strutils.split("", [","]) returns [] instead of [""].
- Add cipherutils.MysqlAesCipher.
- Add listutils.topological_sort.
- Add nameutils.
### v0.29.0
- Use pypinyin module instead of dragonmapper.
- Replace the result lve to lue.
### v0.28.2
- Fix dictutils.update, so that it can works on anything.
### v0.28.1
- Add default parameter to fsutils.first_exists_file.
### v0.28.0
- Add fsutils.first_exists_file.
- Add fsutils.expand.
- Add listutils.first.
### v0.27.0
- Add lower_first parameter for strutils.camel.
- Add dictutils.prefix_key.
- Add sysutils.get_worker_id.
### v0.26.0
- Add treeutils.build_tree from list.
- Add treeutils.walk_tree.
- Add treeutils.print_tree.
- Add fsutils.touch.
### v0.25.0
- Add strutils.is_uuid.
- Add typingutils.cast_uuid.
### v0.24.2
- fsutils.file_content_replace add ignore_errors parameter.
### v0.24.1
- typingutils.cast_xxx treat empty string as None.
- randomutils.UuidGenerator.next add parameter n, so that it can generate n uuids. If n==1, returns a UUID instance, and if n>1, returns a list of UUIDs.
### v0.24.0
- Add strutils.unquote.
### v0.23.0
- Add funcutils.get_default_values.
- Add funcutils.chain.
### v0.22.0
- Add none-dict support in dictutils.change and dictutils.changes.
- Add strutils.format_with_mapping.
### v0.21.0
- Add listutils.compare(old_set, new_set).
- Add dictutils.change(object_instance, data_dict, object_key, dict_key=None) -> bool.
- Add dictutils.changes(object_instance, data_dict, keys) -> bool:
- Change listutils.replace, turn the default value of parameter inplace from True to False.
- Fix Object instance problems.
### v0.20.0
- Add strutils.clean and change strutils.camel.
- Add dictutils.to_object.
- Add pinyinutils.
### v0.19.0
- Add fsutils.filecopy and fsutils.treecopy. Tips: fsutils.copy combines the function of filecopy and treecopy, if the src is a file then use filecopy, and if the src is a folder then use treecopy.
- Fix problem in fsutils.copy.
### v0.18.0
- Add strutils.camel.
### v0.17.0
- Add fsutils.file_content_replace.
- Add fsutils.move.
- Change fsutils.rename's behavior.
- Change fsutils.copy's behavior.
### v0.16.0
- Add fsutils.
### v0.15.2
- Add hostname, seed1, seed4 in domain_template of randomutils.UuidGenerator.
### v0.15.1
- Add incr-lock for counter incr in randomutils.UuidGenerator.
### v0.15.0
- Add randomutils.UuidGenerator.
### v0.14.0
- Add strutils.substrings.
- Add strutils.combinations.
- Add cipherutils.Utf8Encoder.
- Add cipherutils.S1Cipher.
- Add cipherutils.S2Cipher.
- Add funcutils.classproperty
- WARN: Change a cipherutils.CipherBase's parameter name from encoder to result_encoder.
### v0.13.4
- Use new algorithm to improve randomutils.shuffle's performance.
### v0.13.3
- Add randomutils.shuffle.
- Change randomutils.randint parameters from (max, min=0) to (a, b=None).
### v0.13.2
- Add listutils.group.
- Add rsautils.export_key. And rsautils use Crypto.PublicKey.RSA for the base engine.
### v0.13.1
- Add strutils.force_float and strutils.force_numeric.
- Add typingutils.cast_numeric support.
### v0.13.0
- IvfCihper accept integer value when float_digits=0.
- Fix IvfCihper deviation problem which is caused by wrong module used in decrypt.
- IvfCihper use new algorithm in computing module and max_value, **so that result encrypted by version v0.12.0 can not decrypted by version v0.13.0**.
### v0.12.0
- Add IvfCihper for float number encrypt and decrypt. The output of IvfCipher.encrypt is string.
### v0.11.1
- Use class instead of raw api for s12 and iv ciphers. It's can avoid many times in generating seeds.
### v0.11.0
- Add cipherutils.
- Add strutils.binarify and strutils.unbinarify.
- Add randomutils.Random.
- Change aesutils functions' return type. Note: use cipherutils instead.
### v0.10.1
- Add bizerror dependency.
### v0.10.0
- Add strutils.is_chinese_character to test if the character is a chinese character.
- Add cacheutils.get_cached_value to get or set cached value.
### v0.9.0
- Add listutils.append_new to append new value and only new value to the list.
### v0.8.0
- Add strutils.smart_get_binary_data.
- Add rsautils.
### v0.7.0
- Add hashutils.get_file_hash.
- Add extra install requires for python 2.x.
- Add imageutils.parse_base64image and imageutils.get_image_bytes.
- Fix jsonutils.make_simple_json_encoder ignore bases problem.
### v0.6.0
- Add imageutils, add imageutils.get_base64image to make base64 image that can be rendered by web browser.
- Add imageutils.resize to scale image size.
- Add Image-Object-Encode support in jsonutils.
- Add threadutils, add threadutils.Service to simplify long-run-service programming.
- Raise bizerror.MissingParameter error in funcutils.get_inject_params while missing required parameter.
### v0.5.4
- Fix hashutils.get_hash_hexdigest and hashutils.get_hash_base64 problem.
### v0.5.3
- Using typingutils.smart_cast in funcutils.get_inject_params.
### v0.5.2
- Add unit test cases for typingutils.
- Fix cast_list, do strip for every element in comma-separated-list.
- Fix base64 import missing in typingutils.
### v0.5.1
- Add typingutils.cast_str.
### v0.5.0
- Set library property in get_encoder in jsonutils.
- Add typingutils.
### v0.4.0
- Add jsonutils, provides simple json encoder register system.
### v0.3.2
- Fix problems for python 2.7.
- Fix name error of funcutils.
### v0.3.1
- Fix problem casued by str.isascii() which is new from python 3.7.
### v0.3.0
- Add listutils.unique to remove duplicated elements from the list.
- Add listutils.replace to replace element value in thelist with new_value in collection of map.
### v0.2.0
- Add functuils.get_inject_params to smartly choose parameters from candidates by determine with the function's signature.
- Add functuils.call_with_inject to smartly call the function by smartly choose parameters.
### v0.1.1
- Add strutils.wholestrip function, use to remove all white spaces in text.
- Fix strutils.is_urlsafeb64_decodable, strutils.is_base64_decodable and strutils.is_unhexlifiable functions, that have problem to test text contains whitespaces.
### v0.1.0
- Add simple utils about operations of aes, dict, hash, list and str.