Metadata-Version: 2.1
Name: shared-memory-dict
Version: 0.7.2
Summary: A very simple shared memory dict implementation
Home-page: https://github.com/luizalabs/shared-memory-dict
License: MIT
Keywords: django,aiocache,shared memory
Author: Arquitetura LuizaLabs
Author-email: arquitetura@luizalabs.com
Requires-Python: >=3.8,<4.0
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Provides-Extra: aiocache
Provides-Extra: all
Provides-Extra: django
Requires-Dist: aiocache (>=0.11.1,<0.12.0); extra == "aiocache" or extra == "all"
Requires-Dist: django (>=3.0.8,<4.0.0); extra == "django" or extra == "all"
Project-URL: Repository, https://github.com/luizalabs/shared-memory-dict
Description-Content-Type: text/markdown

# Shared Memory Dict

A very simple [shared memory](https://docs.python.org/3/library/multiprocessing.shared_memory.html) dict implementation.

**Requires**: Python >= 3.8

```python
>>> # In the first Python interactive shell
>> from shared_memory_dict import SharedMemoryDict
>> smd = SharedMemoryDict(name='tokens', size=1024)
>> smd['some-key'] = 'some-value-with-any-type'
>> smd['some-key']
'some-value-with-any-type'

>>> # In either the same shell or a new Python shell on the same machine
>> existing_smd = SharedMemoryDict(name='tokens', size=1024)
>>> existing_smd['some-key']
'some-value-with-any-type'
>>> existing_smd['new-key'] = 'some-value-with-any-type'


>>> # Back in the first Python interactive shell, smd reflects this change
>> smd['new-key']
'some-value-with-any-type'

>>> # Clean up from within the second Python shell
>>> existing_smd.shm.close()  # or "del existing_smd"

>>> # Clean up from within the first Python shell
>>> smd.shm.close()
>>> smd.shm.unlink()  # Free and release the shared memory block at the very end
>>> del smd  # use of smd after call unlink() is unsupported
```

> The arg `name` defines the location of the memory block, so if you want to share the memory between process use the same name.
> The size (in bytes) occupied by the contents of the dictionary depends on the serialization used in storage. By default pickle is used.

## Installation

Using `pip`:

```shell
pip install shared-memory-dict
```

## Locks

To use [multiprocessing.Lock](https://docs.python.org/3.8/library/multiprocessing.html#multiprocessing.Lock) on write operations of shared memory dict set environment variable `SHARED_MEMORY_USE_LOCK=1`.

## Serialization

We use [pickle](https://docs.python.org/3/library/pickle.html) as default to read and write the data into the shared memory block.

You can create a custom serializer by implementing the `dumps` and `loads` methods.

Custom serializers should raise `SerializationError` if the serialization fails and `DeserializationError` if the deserialization fails. Both are defined in the `shared_memory_dict.serializers` module.

An example of a JSON serializer extracted from serializers module:

```python
NULL_BYTE: Final = b"\x00"


class JSONSerializer:
    def dumps(self, obj: dict) -> bytes:
        try:
            return json.dumps(obj).encode() + NULL_BYTE
        except (ValueError, TypeError):
            raise SerializationError(obj)

    def loads(self, data: bytes) -> dict:
        data = data.split(NULL_BYTE, 1)[0]
        try:
            return json.loads(data)
        except json.JSONDecodeError:
            raise DeserializationError(data)

```

Note: A null byte is used to separate the dictionary contents from the bytes that are in memory.

To use the custom serializer you must set it when creating a new shared memory dict instance:

```python
>>> smd = SharedMemoryDict(name='tokens', size=1024, serializer=JSONSerializer())
```

### Caveat

The pickle module is not secure. Only unpickle data you trust.

See more [here](https://docs.python.org/3/library/pickle.html).

## Django Cache Implementation

There's a [Django Cache Implementation](https://docs.djangoproject.com/en/3.0/topics/cache/) with Shared Memory Dict:

```python
# settings/base.py
CACHES = {
    'default': {
        'BACKEND': 'shared_memory_dict.caches.django.SharedMemoryCache',
        'LOCATION': 'memory',
        'OPTIONS': {'MEMORY_BLOCK_SIZE': 1024}
    }
}
```

**Install with**: `pip install "shared-memory-dict[django]"`

### Caveat

With Django cache implementation the keys only expire when they're read. Be careful with memory usage

## AioCache Backend

There's also a [AioCache Backend Implementation](https://aiocache.readthedocs.io/en/latest/caches.html) with Shared Memory Dict:

```python
From aiocache import caches

caches.set_config({
    'default': {
        'cache': 'shared_memory_dict.caches.aiocache.SharedMemoryCache',
        'size': 1024,
    },
})
```

> This implementation is very based on aiocache [SimpleMemoryCache](https://aiocache.readthedocs.io/en/latest/caches.html#simplememorycache)

**Install with**: `pip install "shared-memory-dict[aiocache]"`

