Metadata-Version: 2.1
Name: datajunction-reflection
Version: 0.0.1a2
Summary: OSS Implementation of a DataJunction Reflection Service
Project-URL: repository, https://github.com/DataJunction/dj
Author-email: DataJunction Authors <roberto@dealmeida.net>
License: MIT
License-File: LICENSE
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: <4.0,>=3.8
Requires-Dist: celery[redis]>=5.2.3
Requires-Dist: importlib-metadata
Requires-Dist: pydantic<2.0
Requires-Dist: python-dotenv==0.19.2
Requires-Dist: requests>=2.26.0
Provides-Extra: uvicorn
Requires-Dist: uvicorn[standard]>=0.21.1; extra == 'uvicorn'
Description-Content-Type: text/markdown

# DJ Reflection Service

The reflection service polls the DJ core service for all nodes with associated tables, whether source
tables or materialized tables. For each node, it refreshes the node's schema based on the associated
table's schema that it retrieves from the query service. It also retrieves the available partitions and
the valid through timestamp of these tables and reflects them accordingly to DJ core.

This service uses a celery beat scheduler, with a configurable polling interval that defaults to once per
hour and async tasks for each node's reflection.
