🚀 Feature: Fetch documentation from URLs

[blacksmithop]

🔖 Feature description

Not every technical platform offers offers an easy way to export their documentation into a format DocsGPT can extract data from (FastAPI for example). It would be great if one could pass a URL and have the application index and scrap pages from it.

🎤 Why is this feature needed ?

Additional method of passing data

✌️ How do you aim to achieve this?

I would like to see URLs being supported alongside pdf's and docx files

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

• [x] I checked and didn't find similar issue

Are you willing to submit PR?

None

[vikiluk]

Could you please assign this issue to me? I would like to work on it.

[dartpain]

I think @pabik is working on this from one of the older issues. I think we can break this task down into few parts.

1. @tenokami can you prep a UI for uploader. That will sort of look like tabs, where on a second tab (remote) user will have a form where they select type (github, website, smth else) and provide necessary keys or urls.
2. Then @pabik will independently compete backend needed and @VikiLuk can pick up on the frontend implementation

[tRAPlIFE187]

Url are htp hhttp and .gro.htp/ or jpeg it

[blacksmithop]

Thank you!