Add thumbnail support

[isZumpo]

"A picture is worth a thousand words"

I though it could be nice to have some images of the detected birds since I am really bad at bird names... I saw that the BirdWeather GraphQL API offers endpoints to receive thumbnail for a given bird. In this pull request, I have started on a very crude implementation for fetching these thumbnails and adding them in some of the html views.

Daily summary view:

Recent detections view:

Of course the images could be placed better inside the html view, I just did not spend too much time on it.

In addition to only fetching the urls, a in memory caching mechanism is utilized. The cache saves the result for each bird after each request, such that the next request for that bird will be received from the cache instead. In addition, during startup, the cache will be initialized with every detected bird in the database.

I am a bit unsure of where to place these functions. Maybe it would be better to put these inside the birdweather internal/birdweather folder instead? Please give me some advice, and I will do my best to update the pull request accordingly.

[tphakala]

I like idea of having images for each species, that has been on my mind to implement, but I don't want it to be dependent on any online service. Using external resources is often against Terms of Service, and with BirdWeather there is also availability issues.

So I am all in to have thumbnails for each species but they must be locally served, and for that we need to find a royalty free source for photos or have them done using generative AI like Midjourney or Dall-E.

[isZumpo]

Yes those are some really good points! I am also quite curious about them. Especially the Terms of Service aspect... I suppose using the birdweather API to post things is different from utilizing their images. I can see that birdnet-pi is grabbing their images from flickr so that might be a more viable solution?

However, does everything really need to be served locally? That mostly increases the size of the binary, as well as the bandwidth requirements. Sure you get better availability since there is less external dependencies, and have an easier time in air-gapped environments, but is it always necessary? Perhaps some sort of hybrid approach is also a viable option?

[rytilahti]

Regarding ToS issues, perhaps wikidata could be a potential source?

[MueJosh]

This!

However, does everything really need to be served locally? That mostly increases the size of the binary, as well as the bandwidth requirements. Sure you get better availability since there is less external dependencies, and have an easier time in air-gapped environments, but is it always necessary? Perhaps some sort of hybrid approach is also a viable option?

To be honest, I really enjoy the "simple" view right now. This birdnet-Program is really lightweight and draws much less energy then for example the birdnet-pi. Which makes it suitable for battery powered applications (I know there are better solutions, like the AudioMoth or other diy ARU)

Me personally would prefer pictures from an online source, so it's possible to turn this feature off and to save storage. I know how most birds in my area look like- (I get it why pictures are great tho!) -

If needed I can provide some Bird-Pictures (photography brought me to birdnet ;) ) As long as it's not commercially used. (And this feature can be turned off before it downloads XX-hundreds of pictures)

Obviously I don't have ALL birds on my camera (If you're interested let me know, even when you don't want to implement it right now)

[matthew73210]

Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."

[isZumpo]

Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."

That is nice! In theory we could then create our own set of pictures from the same source and either bundle in the git repository itself or host somewhere. Though, I would prefer to use birdweather directly since their api providers some really cool features. Not sure how we should move forward on this topic...

[matthew73210]

Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."

That is nice! In theory we could then create our own set of pictures from the same source and either bundle in the git repository itself or host somewhere. Though, I would prefer to use birdweather directly since their api providers some really cool features. Not sure how we should move forward on this topic...

I created an issue as a feature tracker for stuff from birdweather. Probs discuss stuff there and see what other people think too.

[isZumpo]

Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."

That is nice! In theory we could then create our own set of pictures from the same source and either bundle in the git repository itself or host somewhere. Though, I would prefer to use birdweather directly since their api providers some really cool features. Not sure how we should move forward on this topic...

I created an issue as a feature tracker for stuff from birdweather. Probs discuss stuff there and see what other people think too.

Awesome :) Do you mind linking to the issue?

[matthew73210]

Emailed tim about terms of service etc.. here's what he has to say "We're using images from Wiki Commons - which we credit as provided in their API."

That is nice! In theory we could then create our own set of pictures from the same source and either bundle in the git repository itself or host somewhere. Though, I would prefer to use birdweather directly since their api providers some really cool features. Not sure how we should move forward on this topic...

I created an issue as a feature tracker for stuff from birdweather. Probs discuss stuff there and see what other people think too.

Awesome :) Do you mind linking to the issue?

#130

[coderabbitai[bot]]

Walkthrough

The recent updates encompass significant enhancements to the system's image handling and caching functionalities. Key additions include methods to fetch and cache bird thumbnails, improved database retrieval functions, and new image API integrations. These updates enhance the user interface by displaying bird images and attribution details in tables, leveraging asynchronous cache initialization for efficient image handling.

Changes

Files/Groups	Summary
internal/datastore/interfaces.go, internal/datastore/...	Added new methods for retrieving detected species and expanding bird data retrieval.
internal/httpcontroller/routes.go, internal/httpcontroller/utils.go	Introduced image caching, Wikimedia querying, and thumbnail display functions, along with route and utility updates for these features.
views/fragments/birdsTableHTML.html, views/fragments/recentDetections.html	Added thumbnail images and attribution details to bird species tables.
go.mod	Updated dependencies to support new features.
internal/httpcontroller/imageprovider/...	Implemented image fetching, caching mechanisms, and Wikipedia interactions for bird images.
internal/httpcontroller/init.go	Introduced bird image cache initialization in the server setup.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Server
    participant Datastore
    participant WikimediaAPI

    User->>Server: Request bird species data
    Server->>Datastore: Query for detected species
    Datastore-->>Server: Return detected species list
    Server->>WikimediaAPI: Request bird thumbnail for species
    WikimediaAPI-->>Server: Return bird thumbnail and attribution
    Server->>User: Display bird species with thumbnails

Poem

Whiskers twitch in coder's delight,
Bird thumbnails now take their flight,
Cached and loaded, swift as breeze,
With attribution notes, sure to please.
Datastore hums, steady and true,
Images fetched, a splendid view,
In rabbit's realm, code dreams anew.

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[PovilasID]

Regarding of internet dependency I think it would be OK to have as an option. whoBird has a toggle that pulls down images from Macaulay Library https://www.birds.cornell.edu/home/ebird-api-terms-of-use/ Looks to permit use but way I am reading it could not be packaged inside of the container, could have same setup as birdweather.

[tphakala]

I am sorry but I am not going to accept this PR, I am not okay with consuming any resources from BirdWeather.com without prior agreement with Tim & Co., and even then I don't see any point of accessing resources from them if original source is publicly available at somewhere else.

[isZumpo]

I am sorry but I am not going to accept this PR, I am not okay with consuming any resources from BirdWeather.com without prior agreement with Tim & Co., and even then I don't see any point of accessing resources from them if original source is publicly available at somewhere else.

I am completely on board with this thought too.

Was rebasing the thumbnail branch yesterday and planning on soon replacing the birdweather api calls for wikimedia ones. Though it might make more sense to close this PR and make a new for that?

[isZumpo]

Ok finally got around to creating some sort of PoC of the wikimedia api. Thanks to @PovilasID for making me aware of the query feature in the wikipedia api in the discussion thread yesterday. Was quite lost on how to get the correct image for a given bird in some automatic way otherwise :)

Image proof that it still works as expected:

However, I am still not quite sure how we are actually supposed to give the correct credit for the images. I can see that the birdweather graphql api returns these fields along with the image:

imageCredit - Credited author of image
imageLicense - Name of image license
imageLicenseUrl - URL to image license page

I have never used birdweather directly myself, so I don't know how these fields are displayed inside the application itself? Do you have any idea @matthew73210

[PovilasID]

@isZumpo np. I do not think we can really get good way to get author info I think for now you can probably just split the string of the image url on the dash after 300px and replace everything before it with https://en.m.wikipedia.org/wiki/File: aka https://en.m.wikipedia.org/wiki/File:Blackcap_(Sylvia_atricapilla)_male.jpg and stick in hover over or meta data or in full page with Source Wikipedia

macaulaylibrary.org has en embed view https://macaulaylibrary.org/asset/211672961/embed to get pics from them you need to know image ID... whobird just made 3 60k arrays... to be as lookup table lol. So Sylvia atricapilla_Eurasian Blackcap in https://github.com/woheller69/whoBIRD/blob/master/app/src/main/assets/labels_en.txt corresponds to blackc1 in https://github.com/woheller69/whoBIRD/blob/master/app/src/main/assets/taxo_code.txt and 211672961 in https://github.com/woheller69/whoBIRD/blob/master/app/src/main/assets/assets.txt this permits them build a quick image URL: https://cdn.download.ams.birds.cornell.edu/api/v1/asset/211672961/900 This is a very fragile way of doing things but there is probe a way to update it and this way is very fragile but I have to admit I do prefer their pictures.

[matthew73210]

Ok finally got around to creating some sort of PoC of the wikimedia api. Thanks to @PovilasID for making me aware of the query feature in the wikipedia api in the discussion thread yesterday. Was quite lost on how to get the correct image for a given bird in some automatic way otherwise :)

Image proof that it still works as expected:

However, I am still not quite sure how we are actually supposed to give the correct credit for the images. I can see that the birdweather graphql api returns these fields along with the image:

imageCredit - Credited author of image
imageLicense - Name of image license
imageLicenseUrl - URL to image license page

I have never used birdweather directly myself, so I don't know how these fields are displayed inside the application itself? Do you have any idea @matthew73210

Re credit, probs the best way is to have it mentioned on the github page and under options (if that's where you put the toggle). If memory serves they can be part of the metadata of the image itself. Not sure if this helps.

[tomlut]

I have never used birdweather directly myself, so I don't know how these fields are displayed inside the application itself?

On the birdweather website clicking on the 🛈 icon next to the bird thumbnail takes you to a page with information about the species, a larger image and image credit link at the bottom of the page. e.g. https://app.birdweather.com/species/golden-whistler

In the birdnet iOS app clicking on the thumbnail takes you to the Wikipedia article.

[isZumpo]

I have never used birdweather directly myself, so I don't know how these fields are displayed inside the application itself?

On the birdweather website clicking on the 🛈 icon next to the bird thumbnail takes you to a page with information about the species, a larger image and image credit link at the bottom of the page. e.g. https://app.birdweather.com/species/golden-whistler

In the birdnet iOS app clicking on the thumbnail takes you to the Wikipedia article.

Ok that looks great! Did some work today and managed to parse out the owner of the image and license itself:

will have to do some more code improvement and management of edge cases before I update the pr :)

[tphakala]

looks good

[isZumpo]

Finally managed to get something kind of ready. Let me tell you, getting the author information out of Wikipedia is very tricky since everyone is using different formats for the author block. Some just have their names, others have URLs, and then there are even some crazy people with tables and lots of links... I believe what I have now covers most of the different scenarios quite well!

Recent detections view

Daily summary view

I am a bit unsure about the daily summary view since the credits do not really fit into the table itself. Any suggestions on how we could format it in a nicer way there?

Anyways for those of you who want to try it out, feel free to either build this PR yourself. Or grab the image itself from my fork:

docker run ghcr.io/iszumpo/birdnet-go:f68715d97dddf310e05c525d4eba281f52f06dec ...

[tphakala]

Is this ready for merge? We can tweak things further later

[isZumpo]

Is this ready for merge? We can tweak things further later

Yeah, you are right let's not try to get everything perfect on the first try. There are observed issues for certain birds where Wikipedia does not return any "machine-readable" data. I believe that should be fixable without too much trouble in a future PR. Furthermore, it does not cause much more trouble than a missing image.

Since there are mentions of birdweather etc in the previous commits of this PR. Perhaps it is best to merge it by squashing?