Add support for multi-node for %load_node
or %load_pipeline
#4170
Labels
Issue: Feature Request
New feature or improvement to existing feature
Milestone
Description
This happens more commonly in Data Engineering pipeline, where you apply a series of transformation, aggregation on a set of table and pass it to next node. For example, you may get a "Column not found" error. loading the node that thrown out an error is only the first step to inspect the data, but there are still couple of manual steps to figure out where is the source of error.
The process roughly work as a binary search of the upstream nodes.
Context
This augment the existing debugging feature of Kedro and making this much easier for DS & DE
It's not a trivial task to figure out the correct execution order from a Kedro pipeline to a imperative manner (i.e. cells run sequentially in a notebook). The abstraction is a distraction mostly during debugging.
Possible Implementation
Limitation: Creating multiple cells is not easy, I tried in
%load_node
the first time but settle with the current solution because IPython do have limitations. We may be able to do this inJupyter Notebook
(not VSCode notebook) because there are better supportPossible Alternatives
The text was updated successfully, but these errors were encountered: