open-semantic-search icon indicating copy to clipboard operation
open-semantic-search copied to clipboard

Does OSS follow links on pages and documents?

Open ZeroCool940711 opened this issue 4 years ago • 0 comments

I have a lot of links I've imported but it seems like OSS is not following any links it finds on web pages and documents, for web pages it only follow links on the same domain but not outside it, I even imported some CVS files with only urls on it and none of the urls were indexed, it only index the document and is not following the links it finds inside. Is it because I need to configure something specific for that to work or its just the way OSS works in general? If it's the later I think it would be nice to have an option to force it follow links but that should be the default behavior in my opinion as that's what usually any search engine do.

ZeroCool940711 avatar Apr 22 '20 13:04 ZeroCool940711