Describe how to use custom code with Kedro

[astrojuanlu]

Description

In general, i struggle to find any documentation on how to place/use/import custom libraries with kedro

https://linen-slack.kedro.org/t/16143547/hey-all-very-new-and-excited-user-of-kedro-here-hopefully-i-#600585f1-60b6-48e3-b0b6-6d1efe01480a

Context

This is natural for people who understand that Kedro projects are mostly Python libraries, but

1. This was not the case in Kedro<0.19, and
2. The fact that kedro run injects the project source into the path on startup hides the fact that users can, and should, install their own code.

https://github.com/kedro-org/kedro/blob/44a3d170fc15be2139f8fafcc647431c5f506aeb/kedro/framework/startup.py#L147-L148

[stichbury]

@astrojuanlu If possible, pleasecould you add a bit more detail of where this content should go in the docset and who could contribute it. It looks like something that engineering should write at first draft. What's your priority here -- high-ish?

[astrojuanlu]

I'd say this is Medium-Low priority, and something that definitely engineering should start.

I'd need a bit more time to propose an outline. There are some loose ends in our user journeys that I think would impact this content:

• Are we encouraging users to install their own code for development anywhere? The only place where pip install [-e] . is mentioned is in the packaging section of the tutorial https://docs.kedro.org/en/stable/tutorial/package_a_project.html which probably needs revisiting already:

In addition, it was always weird to have those instructions in the "Package an entire Kedro project" page - kedro package is about distributing the code (wheel + config), docs have nothing to do with it.

Originally posted by @astrojuanlu in https://github.com/kedro-org/kedro/issues/3376#issuecomment-1836354148

• Are we encouraging users to configure their IDEs assuming that they will install their own code? Looking at https://docs.kedro.org/en/stable/development/set_up_vscode.html and https://docs.kedro.org/en/stable/development/set_up_pycharm.html, I'd say the answer is "no".

We could tackle this task without revisiting these user journeys, but I think we risk introducing some inconsistent information, or making the page too long or complex.

I'll give this some more thought in the near future.

[yury-fedotov]

Starting on this per conversation with @noklam