conduit icon indicating copy to clipboard operation
conduit copied to clipboard

Epic: Better onboarding experience

Open raulb opened this issue 1 year ago • 1 comments

This issue encompasses:

  • CLI command for conduit init which creates a template pipeline configuration file based on the chosen source and destination (quick and dirty + patch release, to make it available)
  • Write a clear quickstart guide that brings a newcomer to the "aha" moment as fast as possible
  • Video tutorial that follows the quickstart guide for users who want to see Conduit in action without installing it themselves

After this, we should be ready to reach out to users and verify that the onboarding experience makes sense, based on the feedback we can continuously improve it.

Tasks:

  • [ ] https://github.com/ConduitIO/conduit/issues/1903

raulb avatar Oct 16 '24 18:10 raulb

Draft idea for the quickstart guide:

  1. Install Conduit
    • Should be as easy as running a single command (See https://github.com/ConduitIO/conduit/issues/1903).
  2. Run it
    • Explain that running Conduit now results in an empty instance.
    • Only useful if you want to interact with Conduit via the API.
  3. Build your first pipeline
    • First pipeline should use connectors that don't require external resources.
    • generator -> file is a good candidate.
      • The log destination is another candidate, but file is better, since the records land outside of Conduit and get persisted (as opposed to the log), so it's easier to understand what Conduit is doing.
    • Ideally, we should have a conduit init command in this step.
    • Run Conduit and see data flowing at this stage.
  4. Add your first processor
    • Mask a field or similar. We should identify a simple use case, but one that actually reflects some common processing use case.
    • Restart Conduit and see the updated data flow again.
  5. Refer to next steps
    • Discover connectors
    • Discover advanced features (e.g. DLQ, batching, multiple collections, WASM processors)

lovromazgon avatar Oct 17 '24 16:10 lovromazgon

FTR, Lovro's suggestion has been split into two pages: one for the getting started guide and one for how to add a processor to a pipeline.

hariso avatar Nov 08 '24 08:11 hariso

The item that was originally added to this EPIC (https://github.com/ConduitIO/conduit-site/issues/187) has been moved to https://github.com/ConduitIO/conduit/issues/1919 since that's more appropriate. With that, this EPIC can be considered completed. The main issues we wanted to tackle was an easy way to getting started via conduit init and conduit pipelines init. That's already part of our main branch (and part of 0.12.3) and will be shipped also as part of our Conduit CLI.

raulb avatar Jan 08 '25 17:01 raulb