lhotse icon indicating copy to clipboard operation
lhotse copied to clipboard
verified publisher icon lhotse-speech

Lhotse tutorials

Open pzelasko opened this issue 4 years ago • 4 comments

I'd like to compile a list of topics that would be good tutorial material, I can contribute some of those, and ideally we'd also get some community contributions. I've refreshed the examples directory in PR #617 so that each tutorial notebook has its own "theme" rather than dataset. That should make it easier/cleaner to add new tutorials.

Topics:

  • [x] a minimal, realistic, one-click "getting started" example (#617)
  • [x] Cut/CutSet Python API overview (#617)
  • [x] WebDataset integration (#619)
  • [x] How to combine multiple datasets (#624)
  • [ ] Determining the optimal batch_size / max_duration for dynamic batching samplers
  • [ ] Writing your own PyTorch Dataset for custom/specialized training tasks
  • [ ] Using custom fields in Cut for multiple feature types, alignments, etc.
  • [ ] Different kinds of feature extractors (and which one to select for what use-case)
  • [ ] Creating manifests for large datasets
  • [ ] Using Lhotse with PytorchLightning
  • [ ] Single notebook training recipes (similar to Lhotse + ESPNet minimal example) for
    • [ ] speaker ID
    • [ ] speech separation
    • [ ] TS-VAD
  • [ ] Leveraging existing Kaldi data preparation in Lhotse

pzelasko avatar Mar 14 '22 17:03 pzelasko

Some more ideas, based on projects that I have used Lhotse for.

  • Different kinds of feature extractors (and which one to select for what use-case)
  • Creating manifests for large datasets
  • Using Lhotse with PytorchLightning
  • Single notebook training recipes for speaker ID, speech separation, TS-VAD (similar to Lhotse + ESPNet minimal example)

desh2608 avatar Mar 14 '22 18:03 desh2608

For PyTorch Lightning, one of the maintainers re-wrote the ESPnet example to use PL when we were debugging some issues with fault tolerant training in PL, maybe we can use that snippet, ofc with a proper attribution (IIRC he agreed to that in a private conversation).

https://github.com/PyTorchLightning/pytorch-lightning/issues/10358#issuecomment-962131626

pzelasko avatar Mar 14 '22 18:03 pzelasko

Hi @pzelasko — I was wondering whether community contributions would still be welcome for some of the following (listed above), e.g.:

  • Using Lhotse with PytorchLightning
  • Determining the optimal batch_size / max_duration for dynamic batching samplers
  • Writing your own PyTorch Dataset for custom/specialized training tasks

I'm new to using Lhotse and have just started exploring how to use it, in particular with PyTorch Lightning (e.g. https://github.com/fauxneticien/lightning-speech-sampling). I'm documenting the things as I go anyway but would be happy to prepare those materials as eventual contributed tutorials for Lhotse in mind, but no worries if repo priorities have changed since these posts last year (March '22).

fauxneticien avatar May 26 '23 16:05 fauxneticien

Absolutely! I'd be happy to accept contributions with tutorials for the topics you listed.

pzelasko avatar May 26 '23 18:05 pzelasko