Dependencies

Project-specific dependencies

When we introduced Kedro, we touched briefly on how to specify a project’s dependencies to make it easier for others to run your project and avoid version conflicts downstream.

You can add or remove dependencies. For a new project, edit src/requirements.txt and then run the following:

kedro build-reqs

The build-reqs command will:

  1. Generate src/requirements.in from the contents of src/requirements.txt

  2. pip compile the requirements listed in src/requirements.in

  3. Regenerate src/requirements.txt to specify a list of pinned project dependencies (those with a strict version)

Note

src/requirements.in contains “source” requirements, while src/requirements.txt contains the compiled version of those and requires no manual updates.

To further update the project requirements, you should modify src/requirements.in (not src/requirements.txt) and re-run kedro build-reqs.

kedro install

To install the project-specific dependencies, navigate to the root directory of the project and run:

kedro install

kedro install automatically compiles project dependencies by running kedro build-reqs behind the scenes if the src/requirements.in file doesn’t exist.

To skip the compilation step and install requirements as-is from src/requirements.txt, run the following:

kedro install --no-build-reqs

This takes the latest version of a dependency that is available within the range specified. It allows flexibility in the version of the dependency that pip installs. For example, if ipython>=7.0.0,<8.0 is specified, then the most up-to-date version available is installed.

To force the compilation, even if src/requirements.in already exists, run the following:

kedro install --build-reqs

In some cases, such as a production setting, this is useful to eliminate ambiguity and specify exactly the version of each dependency that is installed.

Workflow dependencies

To install all of the dependencies recorded in Kedro’s setup.py run:

pip install "kedro[all]"