Metadata-Version: 2.1
Name: faker-datasets
Version: 0.1.0
Summary: Faker provider that loads data from your datasets
Author-email: Domenico Andreoli <domenico.andreoli@elastic.co>
License: Copyright (c) 2022 Elasticsearch B.V.
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/elastic/faker-datasets
Project-URL: Bug Tracker, https://github.com/elastic/faker-datasets/issues
Keywords: faker,fixtures,data,test,mock,generator
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE

# Build [Faker](https://github.com/joke2k/faker#readme) providers based on datasets

`faker-datasets` offers a building block for seeding the data generation
with existing data.

You can create simple providers picking a random entry from a tailored dataset or
assemble complex ones where you generate new combinations from more datasets,
all this while keeping an eye on speed and memory consumption.

Let's see how to.

# Crash course

We'll use the wonderful [Countries State Cities DB](https://github.com/dr5hn/countries-states-cities-database)
maintained by [Darshan Gada](https://github.com/dr5hn). Download the
[cities](https://raw.githubusercontent.com/dr5hn/countries-states-cities-database/v1.9/cities.json) and the
[countries](https://raw.githubusercontent.com/dr5hn/countries-states-cities-database/v1.9/countries.json) datasets.

## Basic random picker

`Cities` generates a city by randomly picking an entry in the cities
dataset. Here the dataset is named `cities`, the dataset file is
`cities.json` (adjust to the actual path of the file saved earlier)
and the picker, the method to get a random city, is named `city`.

How we define it in file `cities_provider.py`:

```python
from faker_datasets import Provider, add_dataset

@add_dataset("cities", "cities.json", picker="city")
class Cities(Provider):
    pass
```

How we could use it to generate 10 cities:

```python
from faker import Faker
from cities_provider import Cities

fake = Faker()
fake.add_provider(Cities)

for _ in range(10):
    # Use of the picker named in @add_dateset
    city = fake.city()
    print("{name} is in {country_name}".format(**city))
```

One of the many possible outputs:

```
Poiana Cristei is in Romania
Codosera La is in Spain
Jeremoabo is in Brazil
Rodrígo M. Quevedo is in Mexico
Cary is in United States
Locking is in United Kingdom
Mezinovskiy is in Russia
Nesoddtangen is in Norway
Zalesnoye is in Ukraine
Cefa is in Romania
```

Because the data generation is a pseudo-random process, every execution outputs
different results. If you want reproducible outputs, you have to seed the Faker
generator as documented [here](https://faker.readthedocs.io/en/master/index.html#seeding-the-generator).

## Customize the random picker

`CitiesEx` is functionally identical to `Cities` but shows how to define
the picker by yourself. Here `picker=` is gone from the parameters of
`@add_dataset` but a new `city` method is defined.

```python
from faker_datasets import Provider, add_dataset, with_datasets

@add_dataset("cities", "cities.json")
class CitiesEx(Provider):

    @with_datasets("cities")
    def city(self, cities):
        return self.__pick__(cities)
```

Note how the `city` method is decorated with `@with_datasets("cities")`
and how, consequently, it receives the said dataset as parameter.
The call to `__pick__` just selects a random entry from `cities`.


## Matching a criterium

`CitiesFromCountry` exploits the custom picker to return only cities from a
given country. A first implementation could just discard cities from any
other country, getting slower with increasing bad luck.

```python
from faker_datasets import Provider, add_dataset, with_datasets

@add_dataset("cities", "cities.json")
class CitiesFromCountry(Provider):

    @with_datasets("cities")
    def city(self, cities, country_name):
        while True:
            city = self.__pick__(cities)
            if city["country_name"] == country_name:
                return city
```

It's better to limit to the number of attempts though otherwise if
`country_name` is misspelled the picker would enter in an infinite loop.

```python
from faker_datasets import Provider, add_dataset, with_datasets

@add_dataset("cities", "cities.json")
class CitiesFromCountry(Provider):

    @with_datasets("cities")
    def city(self, cities, country_name, max_attempts=10000):
        while max_attempts:
            city = self.__pick__(cities)
            if city["country_name"] == country_name:
                return city
            max_attempts -= 1
        raise ValueError("Run out of attempts")
```

Or, with same results, use the `match=` and `max_attempts=`
parameters of `__pick__`.

```python
from faker_datasets import Provider, add_dataset, with_datasets

@add_dataset("cities", "cities.json")
class CitiesFromCountry(Provider):

    @with_datasets("cities")
    def city(self, cities, country_name):
        # match tells to __picker__ whether the city is good or not
        match = lambda city: city["country_name"] == country_name
        return self.__pick__(cities, match=match, max_attempts=10000)
```

If you know ahead which country you are interested in, say Afghanistan,
you can use the `@with_match` picker decorator. It produces a new index
of only matching entries and the picking speed is again constant and
independent from bad luck.

```python
from faker_datasets import Provider, add_dataset, with_datasets, with_match

@add_dataset("cities", "cities.json")
class CitiesFromCountry(Provider):

    @with_datasets("cities")
    @with_match(lambda city: city["country_name"] == "Afghanistan")
    def afghan_city(self, cities):
        return self.__pick__(cities)
```

At such conditions though it's maybe better to massage your dataset and
leave only the entries matching your criteria.

## Using multiple datasets

`CitiesAndCountries` fuses two datasets for more advanced matches. Note
how `@add_dataset` makes multiple datasets available to the provider
and `@with_datasets` passes them to the given picker.

```python
from faker_datasets import Provider, add_dataset, with_datasets, with_match

@add_dataset("cities", "cities.json")
@add_dataset("countries", "countries.json")
class CitiesAndCountries(Provider):

    @with_datasets("cities", "countries")
    def city_by_region(self, cities, countries, region):
        def match(city):
            # Given a city, find its country info in the countries dataset
            country = next(country for country in countries if country["name"] == city["country_name"])
            # Check that the country is in the region of interest
            return country["region"] == region
        return self.__pick__(cities, match=match, max_attempts=10000)
```

The picker performs the data mix and match so that the region request
is satisfied or an error is signaled.

## Summary

You use `@add_dataset` to attach a dataset to your provider, if you specify
a `picker=` parameter you'll get for free a random picker of entries.
The more datasets you need, the more `@add_dataset` you can use.

If you have special needs you can define the pickers for yourself, each
using what datasets are most appropriate among those made available with
`@add_dataset`. You can add as many pickers as you need.

A picker can use `match=` and `max_attempts=` to make the generation respect
some useful criteria.
