Papermill Pipeline

The papermill pipeline executes Jupyter notebooks using Papermill. Plase read the official documentation on how to paramaterize the notebooks.

An example pipeline definition file is below (and can be obtained typing owl pdef get papermill – see Running Pipelines).

version: 1.2
name: papermill

input_dir: /home/user/myDir
output_dir: /home/user/myDir
notebook: notebook.ipynb

paramaters: {}


# use dask (optional, see docs)
# use_dask: false

resources:
  workers: 1
  memory: 8
  threads: 1

The pipeline requires the input and output directories and the name of the notebook to execute. The executed notebook will be saved in the output directory with the name notebook_out.ipynb in this example.

The parameters required to run the notebook are defined in parameters. See Papermill on information on how to parameterize notebooks.

The following shows an example output notebook that includes outputs and figures.

The notebook papermill_example.ipynb demonstrates a Papermill notebook example.