Metadata-Version: 2.1
Name: pphw-utils
Version: 1.0.0
Summary: Utilities for records pipeline processing
Home-page: https://github.com/mew-www/xy-pphw-utils
Author: Miy
Author-email: mew.proxy@hotmail.com
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown

Package for taking an array of records (utf-8 strings) and splitting it to batches of records, for a delivery system with following limits:
 - max size of a record is 1MB (otherwise discarded)
 - max size of a batch is 5MB (otherwise an additional batch created)
 - max number of records in a batch is 500 (otherwise an additional batch created)

Input is in format: [rec1, rec2, rec3, ..., recN]
Output is ini format: [batch1, batch2, batch3, ..., batchN] where each batch is array of records like input.

To run unittests:  
`python -m unittest -v test/test_records.py`
