data-prep-kit
data-prep-kit copied to clipboard
[Feature] Create new transform to ingest markdown (.md) files and convert to parquet format.
Search before asking
- [X] I searched the issues and found no similar issues.
Component
Transforms/Other
Feature
Convert .md files to parquet files so that they can be processed by data prep pipeline This is the preferred input for InstructLab
Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
Hi, I would like to work on this. Can I be assigned to this issue?
Sure, I have assigned the issue to you. When you are ready, pls raise a PR and assign @daw3rd as a reviewer. Thanks!