(easy!) Feature Request - search page help text update and request new preset for Relationships

[mkoo]

For TPT users (and any others looking for records with relationships) we need a few updates to make the search page better for searching relationships.

1. change the help text in the Related Item Identification box from "Related Identification" to "Any related taxonomy" see:

2. request a new preset which includes Related Items search option, which can be called "preset_relationship" (in my profile it's called relationship_search); screenshot below. But Mariel please weigh in @campmlc

this would help address the fxn requests in https://github.com/ArctosDB/arctos/issues/7675 and is related to the TPT grant that is wrapping up in the next month

[dustymc]

1. Hu? I need specifics, I don't have "any" available and don't even KINDA have the oomph to pull that off dynamically. I suspect I'm not understanding something....
2. Create a profile, let me know the name of it, let me know the "preset name" you'd like.

[mkoo]

1. Yeah this is super simple-- just swap out the help text to "Any related taxonomy" The current text in gray is "Related Identification" which is not helpful because it's in the title of the field; public users dont immediately think, ah here's where I can put my parasite family name! (oh I see the confusion-- I had notes in red text in the screenshot! sorry for the loster moment)

which can be called "preset_relationship" (in my profile it's called relationship_search)

I dont know if @campmlc saw this yet but what I have in my profile "relationship_search" is what we talked about for a new preset" called relationship (preset_relationship) Let me nudge her for a thumbs up!

[Jegelewicz]

here's where I can put my parasite family name!

Because you can't. That will only find related records IDENTIFIED to that family....

[mkoo]

I only know parasite families!

That will only find related records IDENTIFIED to that family....

exactly! that's ok for now but it's still not clear you can add any taxonomic identification to search on. Maybe it's an info [i] we need to add to explain this field....

[campmlc]

That was my question for @dustymc - to what extent is it possible to search across related record taxonomy? Can we cache certain fields, e.g. Class or Family or Genus? Or is this form only capably of searching on the exact value of the related identification? We used to have the ability to search a limited set of related higher taxonomy, I think using caching. If that isn't possibly, we probably need to keep the "Related Item Identification" field and note that this will only search "=" values.

[dustymc]

to what extent is it possible

Things that are in FLAT can be searched.

Class or Family or Genus?

Yep, those, in "as specified by the collection" form.

(And more than one of them is probably OK, but I'll need help with the form if this becomes multiple options.)

info [i] we need to add to explain this field

Yes, clearly! It's matching case-insensitive substring of flat.scientific_name, which is drawn from an arbitrary best-ranked identification.scientific_name. Some help packaging that (or whatever we end up with) up for a help popup would be most appreciated.

[campmlc]

So to clarify, can we search for a Family of hosts with relationship "host of" to a Family of parasites? Not just exact ID = Family, but anything with an identification that nests within that Family if the name has a classification in Arctos? In the case of the host record, I understand that requesting a Family search will pull everything that has an ID with a classification that includes that family. But would we be able to tweak the tool so the relationship search would be able to do the same - e.g. search on all records that have an identification that is nested within that family classification of parasites? Not those records with identification = family name?

[campmlc]

I strongly support the general idea here but would like to tweak a couple preset fields- we don't need media type and accession, for example. But I am on the road and will send a screenshot as soon as I get back to my computer.

[campmlc]

I've created a customized search and results profile with some additional fields and modifications to the one initially posted by @mkoo - see below.

However, I'm getting really strange results. Simple searches like "show me all MSB:Host records with relationship "host of" and related identification = "cestoda" show no results, even though I checked beforehand and there are over 2500 MSB:Para cestodes related to MSB:Host records.

Another search where I only looked on a highly specific related identification =Oochoristica bivitellolobata, an MSB:Herp parasite, and put in no other fields, yielded the following results, none of which have any relationship I can tell to the requested taxon or to any other parasite. ? https://arctos.database.museum/search.cfm?customoidoper=LIST&related_item_identification=Oochoristica%20bivitellolobata

[campmlc]

Here is the herp parasite record I was trying to find using the search form: https://arctos.database.museum/guid/MSB:Para:45864

[mkoo]

Update! Dusty has tightened up some joins so it's more reliable and we have an proposal to make searching on Relationships more streamlined/ easy (I assume)

The current search bar for Related Items looks like this:

However the first three fields are redundant wtih the Identifier search bar and not used in 80% of related item searches and thus just clutters and distracts. So we propose this:

1. keep the last two fields to search:

2. Add a 3rd field to search on Related Taxonomy which would allow users to search on the higher taxonomy of related items with constraints. Specifically, if the related record is identified to species (eg Zygocotyle lunata) then the search terms may include its accepted taxonomy (eg Animalia, Platyhelminthes, Trematoda, Digenea, Echinostomida, Paramphistomata, Zygocotylidae, Zygocotyle, Zygocotyle lunata) to find it. It will not find any alternative even if accepted (eg Global Names ).

The new search box could be called Related Taxonomy with the following documentation with that constraint/ assumption.

Thus is the above example, this search would only find related parasites identified as "Trematoda" and miss out any identified to species. Using the new proposed field Any Taxonomy would allow finding species within Trematoda

I'm going to respond to your comment next, Mariel

[mkoo]

I've created a customized search and results profile with some additional fields and modifications to the one initially posted by @mkoo - see below.

However, I'm getting really strange results. Simple searches like "show me all MSB:Host records with relationship "host of" and related identification = "cestoda" show no results, even though I checked beforehand and there are over 2500 MSB:Para cestodes related to MSB:Host records.

I think there's some assumptions of the collection and the search. Basically all of MSB:Host are of hosts of something (right?!) so very few records (only 8) have explicit relationship Host Of in this collection (https://arctos.database.museum/search.cfm?guid_prefix=MSB%3AHost&related_id_references=host%20of) and none of them are identified as Cestoda. I poked around the Cestoda records in MSB:Para and none of them have Host of relations but Same Lot As. But still doing searches with Identification = Cestoda gets me lots of records.

Another search where I only looked on a highly specific related identification =Oochoristica bivitellolobata, an MSB:Herp parasite, and put in no other fields, yielded the following results, none of which have any relationship I can tell to the requested taxon or to any other parasite. ? https://arctos.database.museum/search.cfm?customoidoper=LIST&related_item_identification=Oochoristica%20bivitellolobata

Similarly, doing a search on identification=Oochoristica bivitellolobata in the MSB:Para (https://arctos.database.museum/search.cfm?guid_prefix=MSB%3APara&scientific_name=Oochoristica%20bivitellolobata) gets me to the record you were looking for. However the Herp record is not reciprocally linked and has no Host of relationship thus why the way you showed does not get the results you were expecting.

So doing searches in the simplest way possible works and if you want to have relationship searches enabled then maybe you want to get some help to do some bulkloading of relationships. let me know!

[campmlc]

There should be over 20,000 MSB host records at least that have " host of" relationships, and even more MSB:Para records that should have "parasite of" relationships.

[mkoo]

There should be over 20,000 MSB host records at least that have " host of" relationships, and even more MSB:Para records that should have "parasite of" relationships.

You should show me then. a Search of Host of within MSB:Host results in 8: (https://arctos.database.museum/search.cfm?guid_prefix=MSB%3AHost&related_id_references=host%20of) Many have 'same lot of' or 'collected with' (eg https://arctos.database.museum/search.cfm?guid_prefix=MSB%3AHost&related_id_references=collected%20with) but these dont have 'host of'

If you want MSB:Host of to have 'host of' relationships (which makes sense of course) then I think that's a separate issue for some backend help.

At any rate, what do you think of improving the search fields for Relationships (https://github.com/ArctosDB/arctos/issues/7725#issuecomment-2093818922)? That would be a useful step, yes? then we can tackle collection organization...

[mkoo]

Ooo, testing it out now... works nicely!

[campmlc]

So the confusion is that in the original search I was trying to search the MSB:Host collection; and assumed I would be searching for "host of" in order to find related parasites. But this only works in the Identifier References field. For the "Related Items Identifier References" field, you have to search on hosts with related item reference of "parasite of". See below. This has always been super confusing in Arctos, especially in data entry, and needs explicit clarification.

When I search MSB:Host for related "parasite of" relationships, I get 10,591 results using the advanced bio geo search preset as a public user:

Here are the results of that search: Note the record limit drops to 250,000 - so I'm not sure if this is all of the results? We had discussed trying to fix this in our initial UI discussions on the new interface last year.

If I log in and do the same search using my personal Biotic Interactions custom search profile, this is what I see and get:

Note the explicit relationship here is that these records are "host of" related records, but when we do the search on related items, we have to choose "parasite of" to get these records. Super confusing.

Note the record limit this time stayed at 500,000. Why? No public user or even operator should be expected to why this is different or what this means in terms of potential search results limitation. We had a long discussion of how to fix this a year ago, but nothing was finalized.

[campmlc]

If I search MSB:Para for all related items, I have to search on related item relationship "host of" - confusing. But then I get 40,000+ parasite results. Note these are related as "parasite of" in the search results not just to MSB:Host (which was developed primarily as a host observation catalog) but also to MSB:Mamm, MSB:Bird, UAM:Mamm etc cataloged voucher records.

[mkoo]

Well see my comments here: https://github.com/ArctosDB/arctos/issues/7725#issuecomment-2093857711 Simple searches on the taxon were working without the relationship specified was my point

As to your example above, I get it. it returns 10,591 for me too (it's showing all results) and it's showing exactly which records have reciprocal relationships. Your first example was searching Host of relationships which dont exist.

Is that how you want to search then? MSB:Host Host of relationship records? If so becomes a data issue which we can file.

[campmlc]

So in answer to what do you think of improving the search fields for Relationships (#7725 (comment))? That would be a useful step, yes? then we can tackle collection organization...

I agree, we are close, let's prioritize and move forward. I would suggest the following fields for the preset. This is my personal Biotic Interactions Query and it contains the most important fields I can think of for finding interactions and relationships. I think we could fix the confusion in the "references" described previously by just renaming and swapping the position of the "Related Items Identifier References" field as shown so both the ID references and "Related Item References" fields are aligned, so it is clear one relates to record IDs and one to the related items?

And the preset results fields should minimally include the following:

[campmlc]

Here are the results to that search I just showed, which I believe addresses #7675 with the added benefit of allowing for examined/detected searches for host records that may not yet have cataloged related items to link to, and which would allow for a basic determination of parasite prevalence.

https://arctos.database.museum/search.cfm?guid_prefix=MSB%3AMamm&customoidoper=LIST&id_references=host%20of&related_id_references=parasite%20of&related_item_taxonomy=Acari&family=Sciuridae&attribute_type_1=examined%20for&attribute_value_1=ectoparasite&attribute_type_2=detected&attribute_value_2=ectoparasite&sp=Biotic%20IInteractions%20Query%20MSB

Sciuridae with Arthropod Acari parasites examined detected.zip

[campmlc]

Works for a parasite query as well:

and results

It would be even better if we could add the parasite attribute "verbatim host ID" to the potential flat attributes for download as this allows recording of host data when there is no host voucher. Alternatively or in addition, adding "related item ID" as a downloadable field would capture related item identifications across any relationship. Sugar on top?

[campmlc]

Even without the sugar, this would get us a huge way down the road towards discoverability of relationships and interactions. Anyone else?

[campmlc]

Discussed with @mkoo today. I would like to explicitly request the following for this preset, in order of actionable priority - with at least the simple request for 1 and 2 completed as soon as is possible.

1. Create the preset "Biotic Interactions" since this preset explicitly includes taxonomy and biotic relationships and corresponds to this search in GloBI.

2. Use the following existing customized search and results settings as shown in this Shareable URL: https://arctos.database.museum/search.cfm?customoidoper=LIST&oidtype=Arctos%20record%20GUID&id_references=host%20of&related_item_taxonomy=Arthropoda&taxon_name=Sciuridae&attribute_type_1=examined%20for&attribute_value_1=endoparasite&sp=Biotic_Interactions_Preset

3. For number 2 above, can the Related Record Summary be modified so that we can instead add a simplified "Related Item ID" to the Results field customize options and to this preset? This should be a simple comma separated list of identifications of the related items in the adjacent column. If there is more than one identification per related record, pick the ID order =1 or whatever is currently displayed on the record.

4. Add the existing parasite attribute "verbatim host ID" to both the customize results table and to the preset results next to the other attributes sex and individual count. This allows host information to be downloaded if there is no associated cataloged host voucher or observation to download.

[mkoo]

This issue was labeled as "easy" so we could focus on the easy to implement steps for fast turnaround-- 1 and 2 are ready for next release (tomorrow?!) but the others should be new issues

new: preset_biotic_interactions

[Jegelewicz]

description for the new preset:

Contains important fields for finding interactions and relationships.

[campmlc]

description for the new preset:

Contains important fields for finding interactions and relationships.

I would change that to " biotic interactions" if possible as I specifically tailored the fields for that, and it corresponds to the preset name.