mushroom-observer icon indicating copy to clipboard operation
mushroom-observer copied to clipboard

iNat import

Open mo-nathan opened this issue 4 months ago • 11 comments

Import observation from iNat into MO. See MO/iNat Sync Discussion #1902, including @AlanRockefeller's idea summarized at https://github.com/MushroomObserver/mushroom-observer/discussions/1902#discussioncomment-8452672

  1. Provide link from an MO observation to an iNat observation.
  2. Create an MO observation based on an iNat observation.
  3. Enable MO to review a user's observations on iNat and see which have corresponding MO observation.
  4. Enable background job to sync iNat observations into MO.

Tasks

  • [ ] Barebones import of single iNat Observation (in foreground, unauthenticated, minimal Obs: just user, date)
    • [x] Link from observations form to Import iNat Obs
    • [x] Revise #new for just 1 Obs (title, size of field)
    • [ ] More robust test of InatImportsController
      • [x] Fixture without images
      • [ ] Test fixture with images
        • [ ] Stub APIKey
        • [ ] Stub calls to aws
    • [x] Revise iNat controller to import just one obs (muliple obss will be bulk_import)
      • [x] Check for multiple obss in param
      • [x] Create some fixtures:
        • [x] only_id: true
        • [x] only_id: false, single Obs
        • [x] Need different fixtures: Observations that aren't already in MO.
      • [x] Date
      • [x] Where
      • [x] Geoloc
      • [x] Geoloc hidden
      • [ ] Images
        • [x] Import Images
        • [x] associate imported images with MO Obs
          • [x] Get correct image IDs: get it from the API, rather than Image.last
        • [x] copyright holder observation_photo[:attribution]
        • [ ] license observation_photo[:license_code]
        • [ ] date. May need to get this from metadata
        • [ ] notes. iNat photos don't have notes. Maybe put original dimensions here.
        • [ ] original_name. Use iNat UUID
        • [ ] Image owner should be current user instead of owner of the API Key
        • [x] Better way to deal with Errno::ENOENT when using API to import images. See InatImportsController#add_inat_images
      • [ ] Name
        • [ ] Import only Fungi
        • [ ] Deal with iNat IDs s.l.
        • [ ] Deal with iNat IDs provisional
        • [ ] Try to resolve homonyms
          • [ ] Ignore MO Names sensu whatever
          • [ ] Can I map using both author from iNat taxon_id (requires another GET) and MO Name.author
        • [ ] Dictionary to translate iNat Names to MO Names?
      • [x] Add Notes
        • [ ] structure notes
      • [ ] Sequences
      • [ ] Location
      • [ ] Projects
      • [ ] Conversation? (perhaps as a comment?) How did Pulk handle conversion of MO Comments => iNat
      • [ ] Add Other iNat Observation fields
  • [ ] New User/API key for all iNat imports
  • [ ] Review iNat licenses and compatibility with MO licenses
    • [ ] iNat Photos with ND licenses - either don't import or require explicit consent to a broader license
    • [ ] Probably expand MO licenses to include all CC licenses without derivate restrictions, update to latest CC version.
  • [x] Observation source
  • [x] add iNat id column to Observations table
  • [ ] Don't import observations that are already in MO. (Ideas: Check MO notes, iNat MO link, what Pulk does, oldest MO Observations duplicated in iNat.)
  • [ ] avoid duplication between Observation#create and InatImport#create
  • [ ] auto link to iNat id
  • [ ] Authenticate
  • [ ] Link from Observation index to iNat bulk Import
    • [ ] import all a user's iNat observations
  • [ ] Move to a background job
  • [ ] Rate limit for the totality of requests (all users) handled by the background job.
  • [ ] Add iNat id to Observations copied/"mirrored" to iNat. Probably a script; Not a separate PR. Do I need to check iNat to find appropriate Obss that were copied via iNat's obsolete import MO obs feature?
  • [ ] Questions:
    • [ ] General organization
    • [ ] Better names for classes, methods, variables, ex: ImportedInatObs
    • [ ] Is there a better (including smaller) way to do iNat fixtures
    • [ ] Mapping iNat obs locations: ignore? use smallest existing MO loc? map some to MO exact equivalents, ex: "Lane County, OR, USA"?; adopt iNat scheme (separate PR).

mo-nathan avatar Feb 22 '24 23:02 mo-nathan