autobrr
feat(indexers): TorrentName templating
Hi! Appreciate the PR, but this breaks the parsing for music since it's missing a category. Me and @varoOP have discussed some solutions to it before and unless AB themselves fixes it, then we need to do some extra parsing.
Regex playground: https://regex101.com/r/Xet2b7/1
Another issue is that the bare title isn't really good for sending to *arr apps either so then the question came up if we should fetch release title from their api, or rather scrape because that's what's needed.
Unfortunately even the API will give you the same thing the IRC announce does. To really get the release name, the site will have to be scraped - will need a cookie and the IP will have to be approved for site browsing.
My solution has been to add the release to the torrent client and have Sonarr pick it up.
I think Sonarr will have a better chance of matching something if Category is not included in the title. Looking at Prowlarr implementation for AB they are using the scrape api. Essentially searching and examining the results. They then parse release info and rebuild the release name. When Sonarr compatibility is turned on they assume everything is S01 and parse out the Episode number when available. We should be able to parse out the Episode number since its in the releaseTags. One difference will be that we wont have access to synonyms that the api provides. Still this might get *arr to match releases.
I think Sonarr will have a better chance of matching something if Category is not included in the title. Looking at Prowlarr implementation for AB they are using the scrape api. Essentially searching and examining the results. They then parse release info and rebuild the release name. When Sonarr compatibility is turned on they assume everything is S01 and parse out the Episode number when available. We should be able to parse out the Episode number since its in the releaseTags. One difference will be that we wont have access to synonyms that the api provides. Still this might get *arr to match releases.
Your latest commit do indeed fix the category thing, nice!
The autobrr indexer definitions are heavily inspired by Cardigann definitions but it's own thing. Prowlarr and I think Jackett went on to support code based definitions as well and I've been playing with that idea in autobrr to allow for more custom behavior on the indexer level without the need to do generic solutions.
AB is a good candidate to be the first with that solution, but it needs more work.
What if we allow rebuilding the torrentName using templates just like we do for torrentUrl?
That could be an option yeah, and could be usable for others. Is the needed data in the announce? I can build out the template etc if you tell me how you'd like it structured.
Yeah I think we have all the data needed in the announce. The following template essentially mirrors how Prowlarr formats resutls
[{{ .releaseGroup }}] {{ .torrentName }} {{ printf "%02d" .releaseEpisode }} {{ cat "[" .releaseTags "]" | replace " / " "][" }}
Just need to trim extra spaces in the releaseTags and make the episode bit conditional only for TV Series.
We should also improve the releaseTags matching group so it only grabs up to / Subtype (releaseGroup) / Episode X / Freeleech as those tags are relatively constant and everything beforehand is varies based on release.
This is looking solid. Will love to test it once the docker PR release is out.
Do I just map the releaseEpisode here
https://github.com/autobrr/autobrr/blob/5d9a8099dab6d16d168b7d179548ce405a82352e/internal/domain/release.go#L482
And then generate torrentName here (also maybe make this function more generic to allow regenerating any attribute with a template)
https://github.com/autobrr/autobrr/blob/33e3691737e233b597015fc7ff85a8c392c686e7/internal/domain/indexer.go#L150
Do I just map the
releaseEpisodeherehttps://github.com/autobrr/autobrr/blob/5d9a8099dab6d16d168b7d179548ce405a82352e/internal/domain/release.go#L482
And then generate
torrentNamehere (also maybe make this function more generic to allow regenerating any attribute with a template)https://github.com/autobrr/autobrr/blob/33e3691737e233b597015fc7ff85a8c392c686e7/internal/domain/indexer.go#L150
Yep something like that, but with parse.match.torrentname it needs to be added on the IndexerDefinition as well. And be sure to make a non empty check so it only runs if it's there. And then overwrite TorrentName with that new parsed value.
I think that's a decent solution. And if you could add some tests would be great as well 👏
Looks good but made some improvements 😄
- empty checks before trying to parse
- added a couple tests
- added year to the torrentName
@3744111 @varoOP I added Year to the torrentName since it can be useful in my mind, but it's up to you to decide if it is of use or not.
With year the releases are parsed into this:
And as for all the brackets, is it a good idea? 😄 also, the releaseTags do contain some things that might not be good to have in the release name but again, up to you.
That format along with the brackets is just what Prowlarr/Jackett use when generating release names.
Which items would you like to remove from tags? Also do we want to instead parse out all the relevant bits from tags and then merge things like {{ .resolution }?
@zze0s I will test this with cross-seed. We had issues with torrentName not being good enough but I think this will solve that. Let you know about the year then!
That format along with the brackets is just what Prowlarr/Jackett use when generating release names. Which items would you like to remove from tags? Also do we want to instead parse out all the relevant bits from tags and then merge things like
{{ .resolution }?
I think just doing the relevant bits might be a bit safer at least for tv and movies which probably is the main goal right.
The releaseTags has been parsed when it hits the parse.Match.TorrentName, so you don't have to do any more parsing of the tags. You should be able do just do {{ .Resolution }}and so on.
If it needs more special handling I think we really need to step away from the YAML definition here and do it's special handling in code instead. But like I mentioned before I don't have a great solution for that yet. Now that Go have Generics this might be a good use for it but it will need quite a bit of refactoring in other parts of the code like initialization of the definitions etc.
Hey, could we get the build fixed. Seems its failing with a forbidden error. Also anything else you would like done before we can merge this?
Error: buildx failed with: error: failed to solve: unexpected status: 403 Forbidden
Hey, could we get the build fixed. Seems its failing with a forbidden error. Also anything else you would like done before we can merge this?
Error: buildx failed with: error: failed to solve: unexpected status: 403 Forbidden
It fails because it's a fork and outside of the org, so a security thing.
If you're happy with the result I think I am too. Here's what the release name will look like
https://github.com/autobrr/autobrr/pull/381/files#diff-118db07a5b8d78f71bb704b23d5987b5002dcb95d35579089dff0afebc423d14R145