tasking-manager icon indicating copy to clipboard operation
tasking-manager copied to clipboard

Add Project Data Quality view

Open emi420 opened this issue 1 year ago • 5 comments

Is your feature request related to a problem? Please describe. There's no way to way to monitor mapping in near real-time, highlighting quality issues at the same time.

Describe the solution you'd like Integrate Underpass Data Quality Tools and using Underpass UI components.

Describe alternatives you've considered Another alternative could be to develop this feature for Tasking Manager only, but there's no way currently of getting data and analyze it every minute without using Underpass. Also this could be done without using Underpass DB/REST API and/or Underpass UI, and develop new endpoints or a new UI, but this components already exists because they are intended to be used on other projects.

Additional context

There were several discussions for the design of this idea:

  • https://github.com/hotosm/tasking-manager/discussions/5617
  • https://github.com/hotosm/tasking-manager/discussions/5685

Finally, we arrived to the conclusion that creating a live monitoring tool, while displaying automatically detected data quality issues, have higher value than other options.

This is the draft design for the view:

image

It could be modified when doing the final implementation.

Technical notes

  • It will be available for 4 countries (1 per Hub) in the first version, in order to have time for testing and improvement
  • Should be located in: components/projectDataQuality.js
  • It will be implemented using Underpass React UI components, including the connection with an Underpass API for getting data
  • It will need need access to some data for the TM project: id, areaOfInterest and mappingTypes for filtering results.

emi420 avatar Oct 26 '23 12:10 emi420

I added the view in this branch:

https://github.com/hotosm/tasking-manager/tree/feature/liveMonitoring

Screenshot 2023-12-06 at 14 48 58

emi420 avatar Dec 06 '23 17:12 emi420

Hi @emi420 are we fetching data in certain intervals? Cause we don't have web socket integration right now to get the realtime data right ?

varun2948 avatar Dec 08 '23 04:12 varun2948

Hey @varun2948 , yes, when the "live" function is checked. I'll add websockets soon to the API and the UI (https://github.com/hotosm/underpass-ui)

emi420 avatar Dec 09 '23 12:12 emi420

Sounds great with the websocket implementation and realtime data.

varun2948 avatar Dec 09 '23 13:12 varun2948

We're building an approach to use gRPC instead of websockets. https://hotosm.github.io/tm-admin/communication/

emi420 avatar Dec 12 '23 13:12 emi420

From discussion today, we will include this feature only in expert mode. Once a new PR is up from @emi420, @royallsilwallz will implement the changes to support the view for expert mode features.

https://github.com/hotosm/tasking-manager/pull/6151 will be closed soon due to unmanageable conflicts.

cc @dakotabenjamin @manjitapandey

ramyaragupathy avatar Mar 19 '24 15:03 ramyaragupathy

Related PR: https://github.com/hotosm/tasking-manager/pull/6299 - @royallsilwallz you're good to pick on the next steps for expert mode integration here.

ramyaragupathy avatar Mar 21 '24 16:03 ramyaragupathy

Related ticket on expert mode integration to launch this feature in experimental mode https://github.com/hotosm/tasking-manager/issues/6303

ramyaragupathy avatar Mar 26 '24 04:03 ramyaragupathy

part of v4.7.2

ramyaragupathy avatar Apr 17 '24 10:04 ramyaragupathy