numaflow
numaflow copied to clipboard
Add ability for examples in SDKs to be built without needing to clone enitre repo
Summary
Currently, in order to facilitate the CI of our SDKs efficiently, we use local references in the dependency management files of the examples. If a user just copies the examples folder without cloning the entire repo and tries to build a docker image, this would lead to an error. We should explore different ways that we can continue to have efficient CI while providing a good experience to new users that do not necessarily want to contribute to the SDK. Another point of discussion as mentioned by @kohlisid is:
Whether we want to expose the Dockerfiles and dependency management files to the user or not. If they care about the code files only and the Docker/Dependency is just for us, then they probably can be omitted from the structure altogether and we have those template in our script/internal folder to use whenever we need it for the CI.
This issue is meant to track the broader discussion of what we want the Numaflow user experience to be like, while also making it as easy as possible for developers to contribute, as Numaflow continues to grow and be adopted.
Use Cases
When a user is exploring the examples, or just onboarding to the Numaflow platform.
Message from the maintainers:
If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.
As per the discussion: https://intuit-teams.slack.com/archives/C02P1NKMBHQ/p1712945480906909
Another thought is that we should probably move the image-push
target outside the examples, as it is used by the CI and not necessarily needed by users that just want to build locally and that probably do not have access to quay.io.