Automatic Updates Via Calibre Plugin

[ammasakshay]

Can you add automatic updates and perhaps a plugin for Calibre. The reason I am asking this, is that your plugin is very well suited for downloading Webnovels and Lightnovels. Webnovels update once and sometimes even twice a day. This means that you will have to redownload the entire book over and over for the new chapters (Or download just the extra chapters, but then it will be seperate from your main volume). One solution to this is to make a calibre plugin similar to fanficfare which can allow automatic updates. Whilte (https://github.com/dteviot/WebToEpub/issues/145) is similar, the FanFicFare app in Calibre does this very well for fanfiction, where you simply click on the book and click update and it does through the link, and I wanted something like that for LN/WN's. Though the principle itself is similar.

[dteviot]

@ammasakshay Converting WebToEpub into a Calibre plug-in is not practical, for two main reasons:

1. WebToEpub is written in JavaScript. Calibre is written in Python.
2. WebTOEpub is written assuming it's executing inside a Web Browser, and so has access to resources that are very different to Calibre.

You mention FanFicFare (which I'm familiar with) and imply it's not satisfactory for LN/WN. Can you be more specific about how it's unsatisfactory? Perhaps you could raise an issue against it suggesting how it could be improved?

As regards adding an "automatic update" to WebToEpub. I'm thinking I could do something along the following lines.

Step 1. Add a "Watch list" to WebToEpub. When a ToC is added to the Watch List, the URLs on the page are recorded against the page. (Along with which URLs were collected and added to an EPUB.) When WebToEpub looks at the page at a later time, only select the new URLs on the page.

Step 2. Add a page that allows user to view all the ToCs on the Watch List. And launch the ToC page (manually)

Step 3. (Maybe) Add button to automatically check each of the items on Watch List for new chapters (URLs). Note, it would probably be better for people to use something like a Reading List on https://www.novelupdates.com for that.

Step 4. (probably impossible) Automatically update EPUB file on PC with newly discovered chapters. Note becuase WebToEpub runs inside a browser, due to security restrictions, there are very some serious limits on what WebToEpub can do

1. No write/modify access to existing files.
2. Very limitied read access to files. (Basically, user has to grant permission by dropping the file onto WebToEpub's tab)

Probably the most I can do is something that allows you to drop two EPUB files onto a web page and then combines them. (I've started work on this, but it's still in the very early stages.)

[ammasakshay]

FanFicFare is unsatisfactory purely because it doesnt have enough LN/WN sources. It uses wuxuaworld.co which messes up chapters/doesn't have spacing properly on many chapters and most other sources it uses are poor. Every other aspect is exactly what I want. Secondly, can you make it possible to paste a list for the table of contents with each line being a chapter? Web2Epub generates funny results for the table of contents for some websites and, the current version requires you to edit each chapter name individually which is not practical for novels with 1k or more chapters. I like your plan for a self updater. From what i understand, you are basically creating a new epub with the new chapters and then using something like epubmerge.

[dteviot]

@ammasakshay

It uses wuxuaworld.co which messes up chapters/doesn't have spacing properly on many chapters and most other sources

I don't understand. Can you provide an example? e.g. Story at URL xxx does yyyy, it should do zzz

Secondly, can you make it possible to paste a list for the table of contents with each line being a chapter? Web2Epub generates funny results for the table of contents for some websites and, the current version requires you to edit each chapter name individually which is not practical for novels with 1k or more chapters.

Please raise a new issue for this. Include URLs of at least two series that have the problem. Also, provide several DIFFERENT examples of "Bad Title"s and what you would change them to. Note, I've got a couple of ideas on how the problem might be solved, with less effort on your part, but I need more information. The more (and different) examples you give, the more likely I'll be able to create a general solution.

[ammasakshay]

It uses wuxuaworld.co which messes up chapters/doesn't have spacing properly on many chapters and most other sources

I don't understand. Can you provide an example? e.g. Story at URL xxx does yyyy, it should do zzz

Example of wuxiaworld.co messing up spacing - https://www.wuxiaworld.co/God-of-Slaughter/1241123.html The same chapter on novelspread - https://www.novelspread.com/chapter/god-of-slaughter/c-133-the-countercharge

It is really irritating when you download a 1k chapter book and 20 - 30 chapters (many times more) have 0 spacing.

[Dimava]

Converting WebToEpub into a Calibre plug-in is not practical, for two main reasons:

1. WebToEpub is written in JavaScript. Calibre is written in Python.
2. WebTOEpub is written assuming it's executing inside a Web Browser, and so has access to resources that are very different to Calibre.

Well, techically you can All needed is to use playwright or puppeteer (i.e. browser automation) to launch a browser with extenstion, open website and click extension button If browser profile is kept, ReadingList may be used to only load updates (tho for this you have to know a bit of python to white calibre ext and know a bit of automation lib to automate downloading)

Maybe I'll try to make it some day