Add additional context to shows

[jef]

Description

In the original zap2xml, users were able to get extra metadata about a show. Artists, first aired, etc.

Based on #41.

[majortom9]

@ferrellsl "My PERL based zap2xml.pl is also still working and I prefer it because it supports the extended program data " I too use the perl script, crontab running for the last 15 years or so. Give it time. The data is there... I think jef's refactor was something that was long overdue. I like how he broke it up modular like. Should make it easier to maintain. I'm just not a javascript / node-js guy. Has the potential to "grow legs" now.

[GuruSR]

Does the new script work with the tvtv site or just gracenote which is dead? I didn't see it in there, guess the chaos and such hasn't caught up with the Wiki here, as the instructions aren't all there on how to convert things over.

[jef]

@GuruSR It works with GraceNote. I believe zap2it is dead and GraceNote replaced it.

https://tvtv.us/ just helps users get their Lineup ID; which I don't have a clever way to do so yet 😁

[GuruSR]

@GuruSR It works with GraceNote. I believe zap2it is dead and GraceNote replaced it.

https://tvtv.us/ just helps users get their Lineup ID; which I don't have a clever way to do so yet 😁

I'm using the Docker with my ASUSTOR, though the 180 minute repeat is a bit much, was hoping to either crontab it to a day of week/time and do it once (with a repeat on error up to x tries) or being able to set the second between pulls.

As for GraceNote, no longer works for my location, empty, so I'd recommend discontinuing it. Also, for Canada (CAN country), it is https://tvtv.ca, not https://tvtv.us, but it seems you have made changes for that, since I am getting a schedule as normal (but had to relink all the channels manually), due to GraceNote's bizarre setup.

[glenb77]

fwiw gracenote has been around forever and they license to schedules direct from the beginnning of schedules direct when gracenote terminated free personal use only accounts due to abuse. zap2it was an affiliate which it looks lite tvtv is also since gracenote's lineup id's come from it.

[jesmannstl]

Show images as well. Still works in the perl version usually a url with zap2it.tmsimg.com

I'm very impressed by the speed of this version.

Side note history. Tribune Media Services (TMS) owned Gracenote. Tribune was bought by Nexstar (owners of Newsnation) and eventually sold Gracenote to Nielsen Data. The old website zap2it.com was still owned by Nexstar so they were still licensing the data from Gracenote and discontinued the site because of that being a mirror of the Gracenote tv listings site. I am guessing the LA Times discontinued as well thats what the aid=lat stood for.

[jef]

Wow... That's some great context guys. I'm impressed by the history 🤯

[jef]

I've updated the code to get some more detailed information. Although, I don't have a great way to test the new format. This "EPG Viewer" plugin from Chrome is pretty barebones.

Can someone test this for me until I figure out a better way to test?

Thanks! You'll have to remove .txt from it since I can't upload .xml files.

xmltv.xml.txt

cc @ferrellsl

[ferrellsl]

The EPG Chrome extension is the only way I have to test as well. I can't load your xml file into NextPVR because of the different channel mappings that NextPVR gets from my tuner.

The EPG Extension does load the file and it looks good except for the missing icons. Some of the extended data also appears to be missing such as the season and episode numbers when I click on the program to see if it has extended info. See image:

When I open an XML file pulled from my HDHomeRun tuner, this is what I seen when I click the program. It does show the season and episode numbers:

Neither views show the actors, producers, etc, but after using XML Notepad it doesn't appear that you've gotten that far with the extra data yet:

[luckyjay]

Will the "additional context" also include "New" information? Today I noticed that my Plex guide is showing all entries as "New" and I think it's because that element is missing from listings now.

BTW thanks for this new version, it seems quite a bit faster than the Perl script. Looking forward to more enhancements.

[jef]

The EPG Chrome extension is the only way I have to test as well.

Oh! Good to know. It seems after further testing, the extension works pretty good, I just had my XML tags messed up 😅

I updated the code to include thumbnails. I got a few more updates and should have some more context. It does take about 5ish seconds longer to get all that info for 15 days. If I did it any faster, I got rate-limited.

[ferrellsl]

Will the "additional context" also include "New" information? Today I noticed that my Plex guide is showing all entries as "New" and I think it's because that element is missing from listings now.

BTW thanks for this new version, it seems quite a bit faster than the Perl script. Looking forward to more enhancements.

It's faster than the PERL script because the PERL script is also downloading images, icons, and the extended metadata such as the producers, actors names, guest stars, etc. as well as taking the time to compress/gzip and cache that data.

[ferrellsl]

Here's an example of the extended metadata displayed in Kodi while using NextPVR as my backend and guide grabber. This data is being pulled via the PERL script. Hoping Jef can get the npm version to this same level of functionality. He's done an amazing amount of work in just a few days!

[ferrellsl]

Here's the same program with the metadata as shown via NextPVR's guide. NextPVR doesn't show as much of the metadata as Kodi does. Note the missing cast members.

[jef]

Definitely will be able to get this extra metadata! I'm working on it now and so far promising. Making some other updates 😁

[ferrellsl]

Definitely will be able to get this extra metadata! I'm working on it now and so far promising. Making some other updates 😁

I have no doubt that you'll be successful. You've done an amazing job in just a few short days thus far.

[jesmannstl]

So basically All shows are showing as "New" I believe the perl script would add <previously-shown /> to programs that were not New

EDIT: This may be a Jellyfin issue as I loaded the perl version instead and got the same

[jesmannstl]

Emby this version in Emby did not show the NEW or repeat on the grid also not showing the Season/Episode <programme start="20250722010000 +0000" stop="20250722020000 +0000" channel="21300"> <title>The Quiz With Balls</title> <sub-title>The Bad Guys 2 in the Quiz With Balls</sub-title> <desc>The Sinclair-Bryant and Behrakis families compete.</desc> <rating><value>TV-PG</value></rating> <category>New</category> <category>Stereo</category> <category>CC</category> <episode-num system="season">2</episode-num> <episode-num system="episode">7</episode-num> <episode-num system="series">SH05039723</episode-num> <icon src="https://zap2it.tmsimg.com/assets/p26784239_b_v13_ad.jpg" /> </programme>

Perl version shows the NEW and Repeat on the grid and the NEW on the program details <programme start="20250721200000 -0500" stop="20250721210000 -0500" channel="I2.1.21300.gracenote.com"> <title lang="en">The Quiz With Balls</title> <sub-title lang="en">The Bad Guys 2 in the Quiz With Balls</sub-title> <desc lang="en">The Sinclair-Bryant and Behrakis families compete.</desc> <date>20250722</date> <length units="minutes">60</length> <icon src="https://zap2it.tmsimg.com/assets/p26784239_b_v13_ad.jpg" /> <url>https://tvlistings.gracenote.com//overview.html?programSeriesId=SH05039723&amp;tmsId=EP050397230017</url> <episode-num system="common">S02E07</episode-num> <episode-num system="dd_progid">EP05039723.0017</episode-num> <episode-num system="xmltv_ns">1.6.</episode-num> <new /> <subtitles type="teletext" /> <rating> <value>TV-PG</value> </rating> </programme>

[jef]

I've got 'new' figured out. It's all the crew and cast that is kind of getting rate limited right now.

I'm going to scratch my current apporach and look to see if I can find some other APIs to use.

I'll make a push for the 'new' and see if that works well for you!

[ferrellsl]

I've got 'new figured out. It's all the crew and cast that is kind of getting rate limited right now.

I'm going to scratch my current apporach and look to see if I can find some other APIs to use.

I'll make a push for the new and see if that works well for you!

The old PERL script used some sleep() statements to avoid rate limiting/server overload. Specifically line 981 of zap2xml.pl.

[jesmannstl]

I pulled a new ota and refreshing my test Emby and Jellyfin

lineupId is still being ignored

[jesmannstl]

Looks better in Emby. Jellyfin still lacking but it's probably a Jellyfin limitation.

[jesmannstl]

I'm not sure what programs actually use the cast crew etc. The old perl version usually included a url to the info which I'm not sure if the players use that either. Jellyfin and Emby do not possibly Plex? <url>https://tvlistings.gracenote.com//overview.html?programSeriesId=SH05115548&amp;tmsId=EP051155480304</url>

[ferrellsl]

I'm not sure what programs actually use the cast crew etc. The old perl version usually included a url to the info which I'm not sure if the players use that either. Jellyfin and Emby do not possibly Plex? <url>https://tvlistings.gracenote.com//overview.html?programSeriesId=SH05115548&amp;tmsId=EP051155480304</url>

Kodi uses the cast/crew data.

[jesmannstl]

I'm not sure what programs actually use the cast crew etc. The old perl version usually included a url to the info which I'm not sure if the players use that either. Jellyfin and Emby do not possibly Plex? <url>https://tvlistings.gracenote.com//overview.html?programSeriesId=SH05115548&amp;tmsId=EP051155480304</url>

Kodi uses the cast/crew data.

Will Kodi pull cast/crew from the URL I have no way to test

[jesmannstl]

Sidenote Jellyfin still sucks not putting Season/Episode

[ferrellsl]

I'm not sure what programs actually use the cast crew etc. The old perl version usually included a url to the info which I'm not sure if the players use that either. Jellyfin and Emby do not possibly Plex? <url>https://tvlistings.gracenote.com//overview.html?programSeriesId=SH05115548&amp;tmsId=EP051155480304</url>

Kodi uses the cast/crew data.

Will Kodi pull cast/crew from the URL I have no way to test

When using an xml guide created by the PERL script I'm pretty sure that Kodi is pulling the info from the gzipped data that gets cached from Gracenote. When I open one of the gzip archives there's a javascript file containing the cast/crew and other metadata. Here's a clip from one of those files.

{"seriesTitle":"CSI: Miami","seriesDescription":"A Florida team of forensics investigators uses cutting-edge scientific methods and old-fashioned police work to solve crimes. Horatio Caine, a former homicide detective, heads a group of investigators who work crimes amid the steamy tropical surroundings and cultural crossroads of Miami.","seriesImage":"p184820_b_h10_ag","backgroundImage":"p184820_i_h9_ad","releaseYear":"0","seriesGenres":"Action|Crime drama|Mystery|Thriller","totalTMSIDs":"312","overviewTab":{"cast":[{"priority":"1","type":"Cast","role":"Actor","characterName":"Lt. Horatio Caine","assetId":"15256_v9_ba","name":"David Caruso"},{"priority":"2","type":"Cast","role":"Actor","characterName":"Det. Calleigh Duquesne","assetId":"76185_v9_ba","name":"Emily Procter"},{"priority":"3","type":"Cast","role":"Actor","characterName":"Det. Eric Delko","assetId":"218146_v9_bc","name":"Adam Rodriguez"},{"priority":"4","type":"Cast","role":"Actor","characterName":"Det. Ryan Wolfe","assetId":"235666_v9_ba","name":"Jonathan Togo"},{"priority":"5","type":"Cast","role":"Actor","characterName":"Det. Frank Tripp","assetId":"75235_v9_bc","name":"Rex Linn"},{"priority":"6","type":"Cast","role":"Actor","characterName":"Natalia Boa Vista","assetId":"20387_v9_bb","name":"Eva LaRue"},{"priority":"7","type":"Cast","role":"Actor","characterName":"Walter Simmons","assetId":"264833_v9_bb","name":"Omar Miller"},{"priority":"8","type":"Cast","role":"Actor","characterName":"Jesse Cardoza","assetId":"67473_v9_bc","name":"Eddie Cibrian"},{"priority":"9","type":"Cast","role":"Actor","characterName":"Dr. Tara Price","assetId":"153438_v9_bc","name":"Megalyn Echikunwoke"},{"priority":"10","type":"Cast","role":"Actor","characterName":"Kyle Harmon","assetId":"254461_v9_ba","name":"Evan Ellingson"}],"crew":[{"priority":"11","type":"Crew","role":"Executive Producer","characterName":"","assetId":"71005_v9_bb","name":"Jerry Bruckheimer"},{"priority":"12","type":"Crew","role":"Executive Producer","characterName":"","assetId":"207492_v9_bb","name":"Carol Mendelsohn"},{"priority":"13","type":"Crew","role":"Executive Producer","characterName":"","assetId":"201594_v9_ba","name":"Ann Donahue"},{"priority":"14","type":"Crew","role":"Executive Producer","characterName":"","assetId":"199208_v9_bb","name":"Anthony Zuiker"}],"upcomingEpisode":{"episodeTitle":"Mayday","seasonNumber":"9","episodeNumber":"22","releaseYear":"","synopsis":"When the transport plane carrying a fugitive crash-lands, the CSIs learn the criminal is planning more than just an escape.","programGenres":"Action|Crime drama|Mystery|Thriller","isNew":false,"isLive":false,"isPremier":false,"isFinale":false,"tags":"STEREO|CC","originalAirDate":"2011-05-08T00:00Z","tmsID":"ep005240610286","displayRating":"TV-14","timings":[{"time":"2025-07-22T01:00Z","channels":[{"time":"2025-07-22T01:00Z","prgSvcID":"105263","callSign":"KAZTCD4","lcn":"7.4","channelAssetId":"s102148_ll_h15_ab"}]},{"time":"2025-07-29T18:00Z","channels":[{"time":"2025-07-29T18:00Z","prgSvcID":"101236","callSign":"KASWDT3","lcn":"61.3","channelAssetId":"s89923_

[luckyjay]

So basically All shows are showing as "New" I believe the perl script would add <previously-shown /> to programs that were not New

EDIT: This may be a Jellyfin issue as I loaded the perl version instead and got the same

I think NextPVR is also expecting <previously-shown />, it's guide is currently showing everything as "new". I believe the perl script used to include that.

[jesmannstl]

I had the same on Jellyfin and I think it's the previously shown flag missing as well

[jesmannstl]

So basically All shows are showing as "New" I believe the perl script would add <previously-shown /> to programs that were not New EDIT: This may be a Jellyfin issue as I loaded the perl version instead and got the same

I think NextPVR is also expecting <previously-shown />, it's guide is currently showing everything as "new". I believe the perl script used to include that.

I posted a fix in Issue 58 for the <previously shown /> as well as displaying the correct Season/Episode numbers