Jackett
[enhancement]: support Torznab o=JSON
Description
Aims to add support for the o parameter as described here, the examples in the standard are only listed for XML, the JSON structure in this PR resembles the XML as closely as possible.
If the parameter is omitted or is not equal to json, XML is assumed.
This is my first PR here, if anything looks wrong please let me know and I'll fix it.
Issues Fixed or Closed by this PR
- Closes #15018
compare the t=caps for example
Yup that indeed looks messed up. After taking a closer look the XML structure is custom built, this would also need to be done for json. The information is the same (except for the missing categories), for example the [0, 1, ...] is the integer representation of the enum values q, season, etc.
Overall, I think the json response should be prettified so its a little more readable than the default condensed output.
I would keep it minified by default, almost all of the time the response would be read by a machine and keeping it minified saves size. And if you're making the request through Postman or Bruno the output will be prettified in the response window.
also think there should be a header block containing data similar to the xml, things like the jackett server, and which version of the json specs this is (in this case json version 1.0 ;-), and the indexer details
I agree
finally I think any keyword that has an output of null or a empty set [] should not be reported, as is the case in the xml output
Also makes sense
Hello @BorisGerretzen, thank you for your contribution.
Sadly now it's late here so I can't provide any concrete feedback, but I'll give it a try.
Looking at your examples this doesn't really respects the Tornzab spec per se but we can improve on that. I was thinking at a resource mapper here with ToJson() and ToXml(), and with a model for the JSON resource with custom property names.
This is what I had in mind:
{
"@attributes": {
"version": "2.0"
},
"channel": {
"item": [
{
"title": "...",
"guid": "...",
"link": "...",
"comments": "...",
"pubDate": "Mon, 1 Feb 2024 12:34:56 +0000",
"size": "1",
"files": "1",
"grabs": "1",
"category": [
"5000",
"5040"
],
"description": "...",
"enclosure": {
"@attributes": {
"url": "....",
"length": "1",
"type": "application/x-bittorrent"
}
},
"torznab:size": "1",
"torznab:poster": "",
"torznab:seeders": "1",
"torznab:leechers": "1",
"torznab:peers": "1",
"torznab:files": "1",
"torznab:grabs": "1",
"torznab:infohash": "...",
"torznab:downloadvolumefactor": "1",
"torznab:uploadvolumefactor": "1",
"torznab:category": [
"5000",
"5040"
],
"torznab:imdbid": 0,
"torznab:tvdbid": 0,
"torznab:tvmazeid": 1
}
]
}
}
@mynameisbogdan @garfield69
I made some changes for the serialization of caps, it looks much more like the original XML now. It uses the GetXDocument function already present and converts this to json, that way we don't need to make this structure twice.
We can do the same for the regular output. Let me know what you think.
Also, is there a way to run this linter before committing?
https://github.com/Jackett/Jackett/blob/96eeff504f07cf1ed94600ddb326dae96b0715fe/azure-pipelines.yml#L314-L349
@mynameisbogdan, I know, but otherwise you have to maintain two versions of the same structure.
Sadly we must, so this is a nay from me so far.
Sorry about the merge rebase.
That's fine, I will rewrite it. The idea is then to have the same as below but instead of XDocument it's JObject? Or make an actual class out of this so it can be used for both XML and JSON? The same question for the regular results.
https://github.com/Jackett/Jackett/blob/96eeff504f07cf1ed94600ddb326dae96b0715fe/src/Jackett.Common/Models/TorznabCapabilities.cs#L288-L350
t=caps
{
"server": {
"@attributes": {
"version": "1.3",
"title": "...",
"strapline": "...",
"url": "...",
"image": "..."
}
},
"limits": {
"@attributes": {
"max": "100",
"default": "25"
}
},
"searching": {
"search": {
"@attributes": {
"available": "yes",
"supportedParams": "q,imdbid,tvdbid,tvmazeid,tag"
}
},
"tv-search": {
"@attributes": {
"available": "yes",
"supportedParams": "q,tvdbid,season,ep,imdbid,tmdbid,tvmazeid,tag"
}
},
"movie-search": {
"@attributes": {
"available": "yes",
"supportedParams": "q,imdbid,tmdbid,tag"
}
}
},
"categories": {
"category": [
{
"@attributes": {
"id": "2000",
"name": "Movies"
},
"subcat": [
{
"@attributes": {
"id": "2030",
"name": "Movies\/SD"
}
},
{
"@attributes": {
"id": "2040",
"name": "Movies\/HD"
}
},
{
"@attributes": {
"id": "2045",
"name": "Movies\/UHD"
}
},
{
"@attributes": {
"id": "2050",
"name": "Movies\/BluRay"
}
}
]
},
{
"@attributes": {
"id": "5000",
"name": "TV"
},
"subcat": [
{
"@attributes": {
"id": "5030",
"name": "TV\/SD"
}
},
{
"@attributes": {
"id": "5040",
"name": "TV\/HD"
}
},
{
"@attributes": {
"id": "5045",
"name": "TV\/UHD"
}
},
{
"@attributes": {
"id": "5060",
"name": "TV\/Sport"
}
}
]
}
]
},
"tags": {
"tag": [
{
"@attributes": {
"name": "scene",
"description": "Torrent release is from a scene group"
}
},
{
"@attributes": {
"name": "internal",
"description": "Torrent release is internal"
}
}
]
}
}
Right so why wrap all the things in @attributes, is that just so it can be converted to/from xml? I don't see why that is necessary. The current implementation is the same but without the @attributes.
Because this is the next best thing to emulate the Torznab XML in JSON.
But when using the Json output it doesn't matter if the property I access is an XML element or an attribute. If someone wants that distinction they can use the xml directly right?
If you really want that distinction I propose using what Newtonsoft does by default, which is prefixing the attributes with an @. That way we can just convert the already existing XML output.
I didn't find anything to show us exactly how the JSON response should be. But do as you seem fit, but that's just a simple API endpoint that supports search.
First off all my apologies for the long delay on this, I was busy with other things.
@mynameisbogdan I did some further investigation on how exactly the format should look like, I couldn't find examples in the Torznab standard either so I looked at one of my Newznab indexers to see how they formatted it. It turns out they use the format you described earlier, with the @attributes. I have implemented a converter from XML to JSON following this format with some accompanying tests to demonstrate it's use.
I can send you some examples of Newznab JSON output to compare if you'd like.
Some examples of the output currently: Caps:
{
"server": {
"@attributes": {
"title": "Jackett"
}
},
"limits": {
"@attributes": {
"default": "1000",
"max": "1000"
}
},
"searching": {
"search": {
"@attributes": {
"available": "yes",
"supportedParams": "q"
}
},
"tv-search": {
"@attributes": {
"available": "yes",
"supportedParams": "q,season,ep"
}
},
"movie-search": {
"@attributes": {
"available": "yes",
"supportedParams": "q"
}
}
},
"categories": {
"category": [
{
"@attributes": {
"id": "2000",
"name": "Movies"
}
},
{
"@attributes": {
"id": "3000",
"name": "Audio"
},
"subcat": [
{
"@attributes": {
"id": "3010",
"name": "Audio/MP3"
}
},
{
"@attributes": {
"id": "3040",
"name": "Audio/Lossless"
}
}
]
}
]
}
}
search:
Not sure what to set atom:link.type to, application/rss+xml seems wrong cause it's json now, but I'm not sure if application/rss+json is a thing. The Newznab indexer I looked at does not have this field.
{
"@attributes": {
"version": "2.0",
"xmlns:atom": "http://www.w3.org/2005/Atom",
"xmlns:torznab": "http://torznab.com/schemas/2015/feed"
},
"channel": {
"atom:link": {
"@attributes": {
"href": "http://127.0.0.1:9117/",
"rel": "self",
"type": "application/rss+xml"
}
},
"title": "AggregateSearch",
"description": "This feed includes all configured trackers",
"link": "http://127.0.0.1/",
"language": "en-US",
"category": "search",
"item": [
{
"title": "...",
"guid": "...",
"jackettindexer": "BitSearch",
"type": "public",
"comments": "...",
"pubDate": "Sat, 09 Dec 2023 00:00:00 +0100",
"size": "1610612736",
"grabs": "1200",
"description": {},
"link": "...",
"category": "5000",
"enclosure": {
"@attributes": {
"url": "...",
"length": "1610612736",
"type": "application/x-bittorrent"
}
},
"torznab:attr": [
{
"@attributes": {
"name": "category",
"value": "5000"
}
},
{
"@attributes": {
"name": "genre",
"value": ""
}
},
{
"@attributes": {
"name": "seeders",
"value": "84"
}
},
{
"@attributes": {
"name": "peers",
"value": "165"
}
},
{
"@attributes": {
"name": "infohash",
"value": "..."
}
},
{
"@attributes": {
"name": "magneturl",
"value": "..."
}
},
{
"@attributes": {
"name": "downloadvolumefactor",
"value": "0"
}
},
{
"@attributes": {
"name": "uploadvolumefactor",
"value": "1"
}
}
]
}
]
}
}