Track notebooks & scripts#
In addition to tracking Python scripts, LaminDB tracks interactive analyses performed in notebooks.
By calling track()
in a notebook or script, input data, and output data get automatically registered associated with the run.
Note
Provenance tracking of notebooks & scripts is analogous to tracking pipelines, scripts & UI data manipulation, see Project flow.
Setup#
!lamin init --storage ./test-track
Show code cell output
馃挕 connected lamindb: testuser1/test-track
import lamindb as ln
ln.settings.verbosity = "hint"
馃挕 connected lamindb: testuser1/test-track
Initiate tracking#
Call track()
to auto-generate IDs to track data lineage.
Copy these into your cell above track()
.
ln.settings.transform.stem_uid = "9priar0hoE5u"
ln.settings.transform.version = "0"
ln.track()
馃挕 Assuming editor is Jupyter Lab.
馃挕 notebook imports: lamindb==0.71.0
馃挕 saved: Transform(uid='9priar0hoE5u6K79', name='Track notebooks & scripts', key='track', version='0', type='notebook', updated_at=2024-05-01 18:49:03 UTC, created_by_id=1)
馃挕 saved: Run(uid='NyYni6a25EGrwZFJ0z1F', transform_id=1, created_by_id=1)
馃挕 tracked pip freeze > /home/runner/.cache/lamindb/run_env_pip_NyYni6a25EGrwZFJ0z1F.txt
LaminDB now automatically tracks all input and output data.
Save run reports and source artifact#
If you want to save a notebook including its run report & source artifact, run:
ln.finish()
See how transforms with execution reports looks in LaminHub:
Query for a notebook or script#
In the API, filter the Transform
registry to obtain a notebook record:
import lamindb as ln
transform = ln.Transform.filter(name="Track notebooks").one()
# Your notebook is linked with to its source artifact (stripped of its output cells) and execution report (with the notebook's output cells)
transform.source_artifact
transform.latest_report
On LaminHub, use the UI filter in the Transforms
view.
Show code cell content
# clean up test instance
!lamin delete --force test-track
!rm -r test-track
馃挕 deleting instance testuser1/test-track
馃挕 not deleting instance from hub as instance not found there
rm: cannot remove 'test-track': No such file or directory