Metadata-Version: 2.1
Name: fickle
Version: 0.2.0
Summary: load pickled data as safely as possible
Home-page: UNKNOWN
Author: Eduard Christian Dumitrescu
Author-email: eduard.c.dumitrescu@gmail.com
License: MIT
Description: # Fickle: Firewalled Pickle
        
        People abuse pickle. Especially researchers. Pickle is [not secure](https://docs.python.org/3/library/pickle.html). Published datasets and ML training weights are often distributed as pickle files (or formats which use pickle files, such as [PyTorch checkpoint.ckpt files](https://pytorch.org/tutorials/beginner/saving_loading_models.html)). Sometimes it is the only format that they are available in.
        
        # Examples
        
        Loading basic types is easy:
        
        ```python
        >>> from fickle import DefaultFirewall
        >>> import pickle
        >>>
        >>> my_picked_data = pickle.dumps({"list": [1, 2, "three", b"four"]})
        >>>
        >>> firewall = DefaultFirewall()
        >>> firewall.loads(my_picked_data)
        {'list': [1, 2, 'three', b'four']}
        ```
        
        Safely loading PyTorch checkpoint files into numpy arrays is just as easy:
        
        ```python
        >>> from fickle.ext.pytorch import fake_torch_load_zipped
        >>> from zipfile import ZipFile
        >>>
        >>> zf = ZipFile("/path/to/sd-v1-4.ckpt")
        >>> ckpt = fake_torch_load_zipped(zf)
        >>> tensor = ckpt["state_dict"]["model.diffusion_model.output_blocks.3.1.norm.weight"]
        >>> tensor.array
        array([0.39097363, 0.3898967 , 0.35191917, ..., 0.41924757, 0.4031702 ,
               0.37156993], dtype=float32)
        ```
        
        You can, optionally, even use `marshmallow` for validation!
        
        # Alternatives
        
        - [picklemagic](https://github.com/CensoredUsername/picklemagic)
        - [pikara](https://pypi.org/project/pikara)
        
        |                                                | fickle | picklemagic | pikara |
        |------------------------------------------------|--------|-------------|--------|
        | Does not rely on `pickle._Unpickler`?          | ✅      | ❌           | ✅      |
        | Uses `pickletools.genops`                      | yes    | no          | yes    |
        | Can load without executing?                    | ✅      | ✅           | ?      |
        | Forbid importing arbitrary objects?            | ✅      | ✅           | ?      |
        | Forbid calling `list.append`/`set.add`/etc?    | ✅      | ❌           | ?      |
        | Forbid calling all methods by default?         | ✅      | ❌           | ?      |
        | Can create dangerous circular structures?      | ✅      | ✅           | ?      |
        | Safe against billion laughs DoS attack?        | ?      | ?           | ?      |
        | Full support for all pickle opcodes?           | ❌      | ✅           | ?      |
        | Has unit tests?                                | ✅      | ❌           | ✅      |
        | Stable API?                                    | ❌      | ✅           | ✅      |
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
