[Question]: Does ragflow support retrieving data from a specified URL? Or extracting data from a specified URL as a knowledge base

[myplxdm]

Self Checks

• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (Language Policy).
• [x] Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• [ ] Please do not modify this template :) and fill in all the required fields.

Describe your problem

Does ragflow support retrieving data from a specified URL? Or extracting data from a specified URL as a knowledge base

[Yaping-Wang]

I would like this feature, too. Retrieving contents from academic websites might be better than parsing PDFs.

[writinwaters]

You might want to look at Tavily-based web search.

[myplxdm]

@writinwaters Thank you for your reply. Indeed, Tavily is a great supplement, but for highly specialized and authoritative information, it is best to obtain it from professional websites, such as those related to law, government affairs, etc. Therefore, I hope RAGFlow can build a knowledge base through URLs.

[writinwaters]

Ugh. Apologies. This feature is not on the roadmap. As a workaround, you can save the HTML files. They can be previewed during an AI chat.