Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

Published 20 hours ago • verified publisher icon LAION-AI

Create ivypanda essays dataset

Open Vechtomov opened this issue 2 years ago • 2 comments

Create dataset from https://ivypanda.com/essays/

Vechtomov avatar Jan 21 '23 18:01 Vechtomov

I'm working on it

Vechtomov avatar Jan 21 '23 19:01 Vechtomov

Here is the result: https://huggingface.co/datasets/qwedsacf/ivypanda-essays I parsed the essay title and content, removed ad blocks, and then used insciptis library to convert html to txt, so even tables and lists look good.

Vechtomov avatar Jan 23 '23 01:01 Vechtomov