domain_discovery_tool_deprecated icon indicating copy to clipboard operation
domain_discovery_tool_deprecated copied to clipboard

Display the search results in DDT as soon as Google/Bing returns them

Open kienpt opened this issue 7 years ago • 1 comments

In the current version, when an user enters a search query, DDT does the following steps:

  1. DDT sends the queries to Google/Bing.
  2. DDT extracts URLs from query results.
  3. DDT fetches the extracted URLs.
  4. DDT parses the fetched pages then display the extracted information including title, short description of the pages.

These steps usually take a long time to complete, leading to unnecessary wait from users to see the search results. Although, we can't avoid these (since we need to fetch all the URLs eventually), it can be improved by running the steps 2, 3, 4 in a separated thread and display the query results as soon as DDT receives them from Google/Bing. We only need title and snippets which are already included in the search results. I am not sure that whether the Google Custom Search or Bing API return these or not, but If we send queries to Google and Bing the same way that Browser did we probably can get this information.

kienpt avatar Mar 23 '17 20:03 kienpt