Epic: Better onboarding experience
This issue encompasses:
- CLI command for
conduit initwhich creates a template pipeline configuration file based on the chosen source and destination (quick and dirty + patch release, to make it available) - Write a clear quickstart guide that brings a newcomer to the "aha" moment as fast as possible
- Video tutorial that follows the quickstart guide for users who want to see Conduit in action without installing it themselves
After this, we should be ready to reach out to users and verify that the onboarding experience makes sense, based on the feedback we can continuously improve it.
Tasks:
- [ ] https://github.com/ConduitIO/conduit/issues/1903
Draft idea for the quickstart guide:
- Install Conduit
- Should be as easy as running a single command (See https://github.com/ConduitIO/conduit/issues/1903).
- Run it
- Explain that running Conduit now results in an empty instance.
- Only useful if you want to interact with Conduit via the API.
- Build your first pipeline
- First pipeline should use connectors that don't require external resources.
generator->fileis a good candidate.- The
logdestination is another candidate, butfileis better, since the records land outside of Conduit and get persisted (as opposed to the log), so it's easier to understand what Conduit is doing.
- The
- Ideally, we should have a
conduit initcommand in this step. - Run Conduit and see data flowing at this stage.
- Add your first processor
- Mask a field or similar. We should identify a simple use case, but one that actually reflects some common processing use case.
- Restart Conduit and see the updated data flow again.
- Refer to next steps
- Discover connectors
- Discover advanced features (e.g. DLQ, batching, multiple collections, WASM processors)
FTR, Lovro's suggestion has been split into two pages: one for the getting started guide and one for how to add a processor to a pipeline.
The item that was originally added to this EPIC (https://github.com/ConduitIO/conduit-site/issues/187) has been moved to https://github.com/ConduitIO/conduit/issues/1919 since that's more appropriate. With that, this EPIC can be considered completed. The main issues we wanted to tackle was an easy way to getting started via conduit init and conduit pipelines init. That's already part of our main branch (and part of 0.12.3) and will be shipped also as part of our Conduit CLI.