calibre-web icon indicating copy to clipboard operation
calibre-web copied to clipboard

Published 20 hours ago • verified publisher icon janeczku

Sync and view annotations from the Kobo E-reader

Open JVT038 opened this issue 2 years ago • 12 comments

Hi, I was wondering if it would be possible to sync the annotations / notes on the E-Reader to Calibre-Web. I know the Kobo Desktop app supports viewing the annotations made on a Kobo E-Reader, so I think it should be possible to implement somehow. Though I don't know too much about the syncing process or the Kepub format, so perhaps I'm wrong. But just an extra tab / panel where the user can view all their notes. Or perhaps selecting a book in their library, and viewing all the notes from that particular book on the book page.

JVT038 avatar Oct 28 '21 15:10 JVT038

Btw, I'm willing to make a PR and attempt to implement this feature myself, however I don't know how the kobo sync works, nor how the code of calibre-web works, so with some additional explanation about kobo sync and the code of calibre-web, I'm happy to try this myself

JVT038 avatar Oct 29 '21 13:10 JVT038

For syncing annotations from the Kobo database, maybe you can have a look at the calibre-annotations plugin ? Here is the forum link : https://www.mobileread.com/forums/showthread.php?t=241206 I would love to see this feature !! 🤩

Thovi98 avatar Oct 29 '21 17:10 Thovi98

Indeed it would be great if Calibre-Web could act as a sync server for Kobo annotations, then it would be possible to generate an export of those in a book view, similar to how Koreader does it.

So I took a look at it. Each time a synchronization is done, a single PATCH request is sent, containing all the annotations of all the books in its body, and several other requests are sent to interrogate / update the annotations from the server. The process is:

  1. UpdateAnnotationsCommand - "PATCH" > To: "/api/v3/content/<user/book_id?>/annotations", body containing all local annotations
UpdateAnnotationsCommand request 
Nov  1 22:52:41 nickel: (   147.720 @ 0x345a708 / sync.debug) =================== SYNC QUEUE [ SyncClient(0x36d1100) ] ==================== 
Nov  1 22:52:41 nickel: (   147.720 @ 0x345a708 / sync.debug)  ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug)  [ SyncLibraryCommand(0x30eb768) ] 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug) 	 ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug) 	 [ SyncAnnotationsCommandV2(0x31c9ec8) ] 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug) 		 ----- Queued ------- 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug) 		 [ UpdateAnnotationsCommand(0x36c93b0) ] 
Nov  1 22:52:41 nickel: (   147.722 @ 0x345a708 / sync.debug) 		 [ UpdateAttachmentsCommand(0x38986e8) ] 
Nov  1 22:52:41 nickel: (   147.722 @ 0x345a708 / sync.debug) 		 [ GetUpdatedAnnotationsCommand(0x33eb580) ] 
Nov  1 22:52:41 nickel: (   147.722 @ 0x345a708 / sync.debug)  ----- Queued ------- 
Nov  1 22:52:41 nickel: (   147.722 @ 0x345a708 / sync.debug)  [ OverDriveSyncCommand(0x36b68d0) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ PocketSyncCommand(0x36dd170) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ UpdateAllRelatedItemsCommand(0x36ac330) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ UpdateNextInSeriesCommand(0x32c6178) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ SyncRecommendationsCommand(0x36db388) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ SyncTopPicksCommand(0x36c82e0) ] 
Nov  1 22:52:41 nickel: (   147.724 @ 0x345a708 / sync.debug)  [ SyncMostPopularTabCommand(0x36d9700) ] 
Nov  1 22:52:41 nickel: (   147.724 @ 0x345a708 / sync.debug)  [ SubmitAnalyticsEventsCommand(0x329ab38) ] 
Nov  1 22:52:41 nickel: (   147.724 @ 0x345a708 / sync.debug)  [ ReviewsSyncCommand(0x32b97b0) ] 
Nov  1 22:52:41 nickel: (   147.724 @ 0x345a708 / sync.debug)  [ SyncUserGuideCommand(0x3a5af30) ] 
Nov  1 22:52:41 nickel: (   147.725 @ 0x345a708 / sync.debug)  [ SyncDictionariesCommand(0x36dcb50) ] 
Nov  1 22:52:41 nickel: (   147.725 @ 0x345a708 / sync.debug)  [ FrictionlessReadingCommand(0x379bc48) ] 
Nov  1 22:52:41 nickel: (   147.725 @ 0x345a708 / sync.debug)  [ DownloadQueuedBooksCommand(0x375acb8) ] 
Nov  1 22:52:41 nickel: (   147.725 @ 0x345a708 / sync.debug)  [ CleanupCommand(0x30c4f20) ] 
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug) =================================================== 
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug)    
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug) pumping with 3 pending commands and 0 active commands 
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug) executing UpdateAnnotationsCommand(0x36c93b0) 
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug) virtual void OAuthAuthenticatedCommand::execute() 
Nov  1 22:52:41 nickel: (   147.762 @ 0x345a708 / headerdump.debug) "Accept-Encoding" = "gzip" 
Nov  1 22:52:41 nickel: (   147.763 @ 0x345a708 / headerdump.debug) "Accept" = "application/json" 
Nov  1 22:52:41 nickel: (   147.763 @ 0x345a708 / headerdump.debug) "Authorization" = "Bearer xxx" 
Nov  1 22:52:41 nickel: (   147.763 @ 0x345a708 / headerdump.debug) "Content-Type" = "application/json" 
Nov  1 22:52:41 nickel: (   147.763 @ 0x345a708 / headerdump.debug) "x-kobo-affiliatename" = "fnac" 
Nov  1 22:52:41 nickel: (   147.764 @ 0x345a708 / headerdump.debug) "x-kobo-appversion" = "4.28.18220" 
Nov  1 22:52:41 nickel: (   147.764 @ 0x345a708 / headerdump.debug) "x-kobo-deviceid" = "xxx" 
Nov  1 22:52:41 nickel: (   147.764 @ 0x345a708 / headerdump.debug) "x-kobo-devicemodel" = "Kobo Clara HD" 
Nov  1 22:52:41 nickel: (   147.764 @ 0x345a708 / headerdump.debug) "x-kobo-deviceos" = "4.1.15" 
Nov  1 22:52:41 nickel: (   147.765 @ 0x345a708 / headerdump.debug) "x-kobo-deviceosversion" = "NA" 
Nov  1 22:52:41 nickel: (   147.765 @ 0x345a708 / headerdump.debug) "x-kobo-platformid" = "00000000-0000-0000-0000-000000000376" 
Nov  1 22:52:41 nickel: (   147.765 @ 0x345a708 / headerdump.debug) "User-Agent" = "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
Nov  1 22:52:41 nickel: (   147.765 @ 0x345a708 / headerdump.debug) "Accept-Language" = "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
Nov  1 22:52:41 nickel: (   147.766 @ 0x345a708 / packetdump.debug) -------------------------- REQUEST --------------------------- 
Nov  1 22:52:41 nickel: (   147.766 @ 0x345a708 / packetdump.debug) "PATCH" > To:  "/api/v3/content/<user_id?>/annotations" 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) 	---------------------- HEADERS --------------------------- 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) "Accept-Encoding" : "gzip" 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) "Accept" : "application/json" 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) "Authorization" : "Bearer xxx" 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) "Content-Type" : "application/json" 
Nov  1 22:52:41 nickel: (   147.768 @ 0x345a708 / packetdump.debug) "x-kobo-affiliatename" : "fnac" 
Nov  1 22:52:41 nickel: (   147.768 @ 0x345a708 / packetdump.debug) "x-kobo-appversion" : "4.28.18220" 
Nov  1 22:52:41 nickel: (   147.768 @ 0x345a708 / packetdump.debug) "x-kobo-deviceid" : "xxx" 
Nov  1 22:52:41 nickel: (   147.768 @ 0x345a708 / packetdump.debug) "x-kobo-devicemodel" : "Kobo Clara HD" 
Nov  1 22:52:41 nickel: (   147.769 @ 0x345a708 / packetdump.debug) "x-kobo-deviceos" : "4.1.15" 
Nov  1 22:52:41 nickel: (   147.769 @ 0x345a708 / packetdump.debug) "x-kobo-deviceosversion" : "NA" 
Nov  1 22:52:41 nickel: (   147.769 @ 0x345a708 / packetdump.debug) "x-kobo-platformid" : "00000000-0000-0000-0000-000000000376" 
Nov  1 22:52:41 nickel: (   147.769 @ 0x345a708 / packetdump.debug) "User-Agent" : "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
Nov  1 22:52:41 nickel: (   147.770 @ 0x345a708 / packetdump.debug) "Accept-Language" : "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
Nov  1 22:52:41 nickel: (   147.770 @ 0x345a708 / packetdump.debug) "Content-Length" : "23192" 
Nov  1 22:52:41 nickel: (   147.770 @ 0x345a708 / packetdump.debug) 	---------------------- BODY --------------------------- 
Nov  1 22:52:41 nickel: (   147.770 @ 0x345a708 / packetdump.debug) "{"updatedAnnotations": [{"clientLastModifiedUtc": "2021-11-01T13:10:10Z","highlightedText": "If something is true, no amount of wishful thinking can undo it.","id": "ede9e626-23d2-45da-864e-5d1b253cdc79","location": {"span": {"chapterFilename": "text/part0004.html","chapterProgress": 0.88888888888888884,"chapterTitle": "Introduction to 30th Anniversary Edition","endChar": 64,"endPath": "span#kobo\\.30\\.1","startChar": 0,"startPath": "span#kobo\\.30\\.1"}},"type": "Highlight"},...
Nov  1 22:52:41 nickel: (   147.772 @ 0x345a708 / packetdump.debug) -------------------------- END OF REQUEST --------------------------- 
Nov  1 22:52:41 nickel: (   147.772 @ 0x345a708 / packetdump.warning) "/api/v3/content/<user_id?>/annotations" => "Protocol "" is unknown" 
Nov  1 22:52:41 nickel: (   147.773 @ 0x345a708 / packetdump.warning) HTTP Status Code: 0 
Nov  1 22:52:41 nickel: (   147.773 @ 0x345a708 / packetdump.warning) Error: 301 
Nov  1 22:52:41 nickel: (   147.773 @ 0x345a708 / packetdump.warning) Source: "/api/v3/content/<user_id?>/annotations" 
Nov  1 22:52:41 nickel: (   147.773 @ 0x345a708 / packetdump.warning) ""
  1. DownloadContentAnnotationsCommand - "GET" > To: "/api/v3/content/<user/book_id?>/annotations?limit=100"
DownloadContentAnnotationsCommand request 
Nov  1 22:52:41 nickel: (   147.814 @ 0x345a708 / sync.debug) pumping with 1 pending commands and 0 active commands 
Nov  1 22:52:41 nickel: (   147.814 @ 0x345a708 / sync.debug) executing CheckForAnnotationChangesCommand(0x31c9db8) 
Nov  1 22:52:41 nickel: (   147.814 @ 0x345a708 / sync.debug) virtual void OAuthAuthenticatedCommand::execute() 
Nov  1 22:52:41 nickel: (   147.815 @ 0x345a708 / sync.debug) =================== SYNC QUEUE [ SyncClient(0x36d1100) ] ==================== 
Nov  1 22:52:41 nickel: (   147.815 @ 0x345a708 / sync.debug)  ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.815 @ 0x345a708 / sync.debug)  [ SyncLibraryCommand(0x30eb768) ] 
Nov  1 22:52:41 nickel: (   147.816 @ 0x345a708 / sync.debug) 	 ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.816 @ 0x345a708 / sync.debug) 	 [ SyncAnnotationsCommandV2(0x31c9ec8) ] 
Nov  1 22:52:41 nickel: (   147.816 @ 0x345a708 / sync.debug) 		 ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.816 @ 0x345a708 / sync.debug) 		 [ GetUpdatedAnnotationsCommand(0x33eb580) ] 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug) 			 ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug) 			 [ CheckForAnnotationChangesCommand(0x31c9db8) ] 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug) 			 ----- Queued ------- 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x33a16f8) ] 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug)  ----- Queued ------- 
Nov  1 22:52:41 nickel: (   147.818 @ 0x345a708 / sync.debug)  [ OverDriveSyncCommand(0x36b68d0) ] 
Nov  1 22:52:41 nickel: (   147.818 @ 0x345a708 / sync.debug)  [ PocketSyncCommand(0x36dd170) ] 
Nov  1 22:52:41 nickel: (   147.818 @ 0x345a708 / sync.debug)  [ UpdateAllRelatedItemsCommand(0x36ac330) ] 
Nov  1 22:52:41 nickel: (   147.818 @ 0x345a708 / sync.debug)  [ UpdateNextInSeriesCommand(0x32c6178) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ SyncRecommendationsCommand(0x36db388) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ SyncTopPicksCommand(0x36c82e0) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ SyncMostPopularTabCommand(0x36d9700) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ SubmitAnalyticsEventsCommand(0x329ab38) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ ReviewsSyncCommand(0x32b97b0) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ SyncUserGuideCommand(0x3a5af30) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ SyncDictionariesCommand(0x36dcb50) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ FrictionlessReadingCommand(0x379bc48) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ DownloadQueuedBooksCommand(0x375acb8) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ CleanupCommand(0x30c4f20) ] 
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug) =================================================== 
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug)    
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug) pumping with 1 pending commands and 1 active commands 
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug) executing DownloadContentAnnotationsCommand(0x33a16f8) 
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug) virtual void OAuthAuthenticatedCommand::execute() 
Nov  1 22:52:41 nickel: (   147.823 @ 0x345a708 / headerdump.debug) "Accept-Encoding" = "gzip" 
Nov  1 22:52:41 nickel: (   147.823 @ 0x345a708 / headerdump.debug) "Accept" = "application/json" 
Nov  1 22:52:41 nickel: (   147.823 @ 0x345a708 / headerdump.debug) "Authorization" = "Bearer xxx" 
Nov  1 22:52:41 nickel: (   147.823 @ 0x345a708 / headerdump.debug) "Content-Type" = "application/json" 
Nov  1 22:52:41 nickel: (   147.824 @ 0x345a708 / headerdump.debug) "x-kobo-affiliatename" = "fnac" 
Nov  1 22:52:41 nickel: (   147.824 @ 0x345a708 / headerdump.debug) "x-kobo-appversion" = "4.28.18220" 
Nov  1 22:52:41 nickel: (   147.824 @ 0x345a708 / headerdump.debug) "x-kobo-deviceid" = "xxx" 
Nov  1 22:52:41 nickel: (   147.825 @ 0x345a708 / headerdump.debug) "x-kobo-devicemodel" = "Kobo Clara HD" 
Nov  1 22:52:41 nickel: (   147.825 @ 0x345a708 / headerdump.debug) "x-kobo-deviceos" = "4.1.15" 
Nov  1 22:52:41 nickel: (   147.825 @ 0x345a708 / headerdump.debug) "x-kobo-deviceosversion" = "NA" 
Nov  1 22:52:41 nickel: (   147.826 @ 0x345a708 / headerdump.debug) "x-kobo-platformid" = "00000000-0000-0000-0000-000000000376" 
Nov  1 22:52:41 nickel: (   147.826 @ 0x345a708 / headerdump.debug) "User-Agent" = "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
Nov  1 22:52:41 nickel: (   147.826 @ 0x345a708 / headerdump.debug) "Accept-Language" = "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
Nov  1 22:52:41 nickel: (   147.827 @ 0x345a708 / packetdump.debug) -------------------------- REQUEST --------------------------- 
Nov  1 22:52:41 nickel: (   147.827 @ 0x345a708 / packetdump.debug) "GET" > To:  "/api/v3/content/<user_id?>/annotations?limit=100" 
Nov  1 22:52:41 nickel: (   147.827 @ 0x345a708 / packetdump.debug) 	---------------------- HEADERS --------------------------- 
Nov  1 22:52:41 nickel: (   147.827 @ 0x345a708 / packetdump.debug) "Accept-Encoding" : "gzip" 
Nov  1 22:52:41 nickel: (   147.829 @ 0x345a708 / packetdump.debug) "Accept" : "application/json" 
Nov  1 22:52:41 nickel: (   147.829 @ 0x345a708 / packetdump.debug) "Authorization" : "Bearer xxx" 
Nov  1 22:52:41 nickel: (   147.830 @ 0x345a708 / packetdump.debug) "Content-Type" : "application/json" 
Nov  1 22:52:41 nickel: (   147.830 @ 0x345a708 / packetdump.debug) "x-kobo-affiliatename" : "fnac" 
Nov  1 22:52:41 nickel: (   147.830 @ 0x345a708 / packetdump.debug) "x-kobo-appversion" : "4.28.18220" 
Nov  1 22:52:41 nickel: (   147.830 @ 0x345a708 / packetdump.debug) "x-kobo-deviceid" : "xxx" 
Nov  1 22:52:41 nickel: (   147.831 @ 0x345a708 / packetdump.debug) "x-kobo-devicemodel" : "Kobo Clara HD" 
Nov  1 22:52:41 nickel: (   147.831 @ 0x345a708 / packetdump.debug) "x-kobo-deviceos" : "4.1.15" 
Nov  1 22:52:41 nickel: (   147.831 @ 0x345a708 / packetdump.debug) "x-kobo-deviceosversion" : "NA" 
Nov  1 22:52:41 nickel: (   147.831 @ 0x345a708 / packetdump.debug) "x-kobo-platformid" : "00000000-0000-0000-0000-000000000376" 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) "User-Agent" : "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) "Accept-Language" : "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) 	---------------------- BODY --------------------------- 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) "" 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) -------------------------- END OF REQUEST --------------------------- 
Nov  1 22:52:41 nickel: (   147.880 @ 0x345a708 / packetdump.warning) "/api/v3/content/<user/book_id?>/annotations?limit=100" => "Protocol "" is unknown" 
Nov  1 22:52:41 nickel: (   147.880 @ 0x345a708 / packetdump.warning) HTTP Status Code: 0 
Nov  1 22:52:41 nickel: (   147.881 @ 0x345a708 / packetdump.warning) Error: 301 
Nov  1 22:52:41 nickel: (   147.881 @ 0x345a708 / packetdump.warning) Source: "/api/v3/content/<user/book_id?>/annotations?limit=100"
  1. CheckForAnnotationChangesCommand - "POST" > To: "/api/v3/content/checkforchanges", with book IDs in the body
CheckForAnnotationChangesCommand request 
> Nov  1 22:52:41 nickel: (   147.862 @ 0x345a708 / sync.debug) pumping with 4 pending commands and 2 active commands 
> Nov  1 22:52:41 nickel: (   147.862 @ 0x345a708 / sync.debug) =================== SYNC QUEUE [ SyncClient(0x36d1100) ] ==================== 
> Nov  1 22:52:41 nickel: (   147.862 @ 0x345a708 / sync.debug)  ----- Running ------ 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug)  [ SyncLibraryCommand(0x30eb768) ] 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug) 	 ----- Running ------ 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug) 	 [ SyncAnnotationsCommandV2(0x31c9ec8) ] 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug) 		 ----- Running ------ 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug) 		 [ GetUpdatedAnnotationsCommand(0x33eb580) ] 
> Nov  1 22:52:41 nickel: (   147.864 @ 0x345a708 / sync.debug) 			 ----- Running ------ 
> Nov  1 22:52:41 nickel: (   147.864 @ 0x345a708 / sync.debug) 			 [ CheckForAnnotationChangesCommand(0x31c9db8) ] 
> Nov  1 22:52:41 nickel: (   147.864 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x33a16f8) ] 
> Nov  1 22:52:41 nickel: (   147.864 @ 0x345a708 / sync.debug) 			 ----- Queued ------- 
> Nov  1 22:52:41 nickel: (   147.865 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x3a57020) ] 
> Nov  1 22:52:41 nickel: (   147.866 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x3a25868) ] 
> Nov  1 22:52:41 nickel: (   147.866 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x39e18c0) ] 
> Nov  1 22:52:41 nickel: (   147.866 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x3200400) ] 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x3578a40) ] 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug)  ----- Queued ------- 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug)  [ OverDriveSyncCommand(0x36b68d0) ] 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug)  [ PocketSyncCommand(0x36dd170) ] 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug)  [ UpdateAllRelatedItemsCommand(0x36ac330) ] 
> Nov  1 22:52:41 nickel: (   147.868 @ 0x345a708 / sync.debug)  [ UpdateNextInSeriesCommand(0x32c6178) ] 
> Nov  1 22:52:41 nickel: (   147.868 @ 0x345a708 / sync.debug)  [ SyncRecommendationsCommand(0x36db388) ] 
> Nov  1 22:52:41 nickel: (   147.868 @ 0x345a708 / sync.debug)  [ SyncTopPicksCommand(0x36c82e0) ] 
> Nov  1 22:52:41 nickel: (   147.868 @ 0x345a708 / sync.debug)  [ SyncMostPopularTabCommand(0x36d9700) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ SubmitAnalyticsEventsCommand(0x329ab38) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ ReviewsSyncCommand(0x32b97b0) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ SyncUserGuideCommand(0x3a5af30) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ SyncDictionariesCommand(0x36dcb50) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ FrictionlessReadingCommand(0x379bc48) ] 
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug)  [ DownloadQueuedBooksCommand(0x375acb8) ] 
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug)  [ CleanupCommand(0x30c4f20) ] 
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug) =================================================== 
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug)    
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug) pumping with 5 pending commands and 2 active commands 
> Nov  1 22:52:41 nickel: (   147.872 @ 0x345a708 / headerdump.debug) "Accept-Encoding" = "gzip" 
> Nov  1 22:52:41 nickel: (   147.872 @ 0x345a708 / headerdump.debug) "Accept" = "application/json" 
> Nov  1 22:52:41 nickel: (   147.872 @ 0x345a708 / headerdump.debug) "Authorization" = "Bearer xxx" 
> Nov  1 22:52:41 nickel: (   147.873 @ 0x345a708 / headerdump.debug) "Content-Type" = "application/json" 
> Nov  1 22:52:41 nickel: (   147.873 @ 0x345a708 / headerdump.debug) "x-kobo-affiliatename" = "fnac" 
> Nov  1 22:52:41 nickel: (   147.873 @ 0x345a708 / headerdump.debug) "x-kobo-appversion" = "4.28.18220" 
> Nov  1 22:52:41 nickel: (   147.873 @ 0x345a708 / headerdump.debug) "x-kobo-deviceid" = "xxx" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "x-kobo-devicemodel" = "Kobo Clara HD" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "x-kobo-deviceos" = "4.1.15" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "x-kobo-deviceosversion" = "NA" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "x-kobo-platformid" = "00000000-0000-0000-0000-000000000376" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "User-Agent" = "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
> Nov  1 22:52:41 nickel: (   147.875 @ 0x345a708 / headerdump.debug) "Accept-Language" = "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
> Nov  1 22:52:41 nickel: (   147.875 @ 0x345a708 / packetdump.debug) -------------------------- REQUEST --------------------------- 
> Nov  1 22:52:41 nickel: (   147.876 @ 0x345a708 / packetdump.debug) "POST" > To:  "/api/v3/content/checkforchanges" 
> Nov  1 22:52:41 nickel: (   147.876 @ 0x345a708 / packetdump.debug) 	---------------------- HEADERS --------------------------- 
> Nov  1 22:52:41 nickel: (   147.876 @ 0x345a708 / packetdump.debug) "Accept-Encoding" : "gzip" 
> Nov  1 22:52:41 nickel: (   147.876 @ 0x345a708 / packetdump.debug) "Accept" : "application/json" 
> Nov  1 22:52:41 nickel: (   147.877 @ 0x345a708 / packetdump.debug) "Authorization" : "Bearer xxx" 
> Nov  1 22:52:41 nickel: (   147.877 @ 0x345a708 / packetdump.debug) "Content-Type" : "application/json" 
> Nov  1 22:52:41 nickel: (   147.877 @ 0x345a708 / packetdump.debug) "x-kobo-affiliatename" : "fnac" 
> Nov  1 22:52:41 nickel: (   147.877 @ 0x345a708 / packetdump.debug) "x-kobo-appversion" : "4.28.18220" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-deviceid" : "xxx" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-devicemodel" : "Kobo Clara HD" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-deviceos" : "4.1.15" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-deviceosversion" : "NA" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-platformid" : "00000000-0000-0000-0000-000000000376" 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) "User-Agent" : "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) "Accept-Language" : "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) "Content-Length" : "430" 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) 	---------------------- BODY --------------------------- 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) "[{"ContentId": "c7b34121-0e04-4593-9d6c-7b8c823ca4e9","etag": "W/\"0\""},{"ContentId": "1a8d0291-5156-44f9-9faf-e86b149dd8e4","etag": "W/\"0\""},{"ContentId": "9753e5f4-8964-4178-92a7-9d0bc87d0445","etag": "W/\"0\""},{"ContentId": "885d68d4-92c7-4136-b318-e3d887c15495","etag": "W/\"A:820707582-Iq992+H1kkGYzpOgLXvzNA, B:462539101-KkAV6b3EYE+6aKdGXLBjow\""},{"ContentId": "258a01a6-ee56-4450-9884-8775f0b3e2df","etag": "W/\"0\""}]" 
> Nov  1 22:52:41 nickel: (   147.880 @ 0x345a708 / packetdump.debug) -------------------------- END OF REQUEST ---------------------------

The problem is that the endpoint is not the same as the store endpoint (probably api.kobobooks.com), and so we cannot redirect this request to Calibre-Web - the URL is hardcoded somewhere. I would love to be wrong, but the only solution might be to request a patch from pgaskin/kobopatch to permanently modify this URL.

Also the requests seems really weird, as if the feature was half implemented in the Kobo readers and the requested URL is never completed, so I don't have any example of a successful response.

a1ex4 avatar Nov 02 '21 13:11 a1ex4

Indeed it would be great if Calibre-Web could act as a sync server for Kobo annotations, then it would be possible to generate an export of those in a book view, similar to how Koreader does it.

So I took a look at it. Each time a synchronization is done, a single PATCH request is sent, containing all the annotations of all the books in its body, and several other requests are sent to interrogate / update the annotations from the server. The process is:

  1. UpdateAnnotationsCommand - "PATCH" > To: "/api/v3/content/<user/book_id?>/annotations", body containing all local annotations
UpdateAnnotationsCommand request 
Nov  1 22:52:41 nickel: (   147.720 @ 0x345a708 / sync.debug) =================== SYNC QUEUE [ SyncClient(0x36d1100) ] ==================== 
Nov  1 22:52:41 nickel: (   147.720 @ 0x345a708 / sync.debug)  ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug)  [ SyncLibraryCommand(0x30eb768) ] 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug) 	 ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug) 	 [ SyncAnnotationsCommandV2(0x31c9ec8) ] 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug) 		 ----- Queued ------- 
Nov  1 22:52:41 nickel: (   147.721 @ 0x345a708 / sync.debug) 		 [ UpdateAnnotationsCommand(0x36c93b0) ] 
Nov  1 22:52:41 nickel: (   147.722 @ 0x345a708 / sync.debug) 		 [ UpdateAttachmentsCommand(0x38986e8) ] 
Nov  1 22:52:41 nickel: (   147.722 @ 0x345a708 / sync.debug) 		 [ GetUpdatedAnnotationsCommand(0x33eb580) ] 
Nov  1 22:52:41 nickel: (   147.722 @ 0x345a708 / sync.debug)  ----- Queued ------- 
Nov  1 22:52:41 nickel: (   147.722 @ 0x345a708 / sync.debug)  [ OverDriveSyncCommand(0x36b68d0) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ PocketSyncCommand(0x36dd170) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ UpdateAllRelatedItemsCommand(0x36ac330) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ UpdateNextInSeriesCommand(0x32c6178) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ SyncRecommendationsCommand(0x36db388) ] 
Nov  1 22:52:41 nickel: (   147.723 @ 0x345a708 / sync.debug)  [ SyncTopPicksCommand(0x36c82e0) ] 
Nov  1 22:52:41 nickel: (   147.724 @ 0x345a708 / sync.debug)  [ SyncMostPopularTabCommand(0x36d9700) ] 
Nov  1 22:52:41 nickel: (   147.724 @ 0x345a708 / sync.debug)  [ SubmitAnalyticsEventsCommand(0x329ab38) ] 
Nov  1 22:52:41 nickel: (   147.724 @ 0x345a708 / sync.debug)  [ ReviewsSyncCommand(0x32b97b0) ] 
Nov  1 22:52:41 nickel: (   147.724 @ 0x345a708 / sync.debug)  [ SyncUserGuideCommand(0x3a5af30) ] 
Nov  1 22:52:41 nickel: (   147.725 @ 0x345a708 / sync.debug)  [ SyncDictionariesCommand(0x36dcb50) ] 
Nov  1 22:52:41 nickel: (   147.725 @ 0x345a708 / sync.debug)  [ FrictionlessReadingCommand(0x379bc48) ] 
Nov  1 22:52:41 nickel: (   147.725 @ 0x345a708 / sync.debug)  [ DownloadQueuedBooksCommand(0x375acb8) ] 
Nov  1 22:52:41 nickel: (   147.725 @ 0x345a708 / sync.debug)  [ CleanupCommand(0x30c4f20) ] 
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug) =================================================== 
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug)    
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug) pumping with 3 pending commands and 0 active commands 
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug) executing UpdateAnnotationsCommand(0x36c93b0) 
Nov  1 22:52:41 nickel: (   147.726 @ 0x345a708 / sync.debug) virtual void OAuthAuthenticatedCommand::execute() 
Nov  1 22:52:41 nickel: (   147.762 @ 0x345a708 / headerdump.debug) "Accept-Encoding" = "gzip" 
Nov  1 22:52:41 nickel: (   147.763 @ 0x345a708 / headerdump.debug) "Accept" = "application/json" 
Nov  1 22:52:41 nickel: (   147.763 @ 0x345a708 / headerdump.debug) "Authorization" = "Bearer xxx" 
Nov  1 22:52:41 nickel: (   147.763 @ 0x345a708 / headerdump.debug) "Content-Type" = "application/json" 
Nov  1 22:52:41 nickel: (   147.763 @ 0x345a708 / headerdump.debug) "x-kobo-affiliatename" = "fnac" 
Nov  1 22:52:41 nickel: (   147.764 @ 0x345a708 / headerdump.debug) "x-kobo-appversion" = "4.28.18220" 
Nov  1 22:52:41 nickel: (   147.764 @ 0x345a708 / headerdump.debug) "x-kobo-deviceid" = "xxx" 
Nov  1 22:52:41 nickel: (   147.764 @ 0x345a708 / headerdump.debug) "x-kobo-devicemodel" = "Kobo Clara HD" 
Nov  1 22:52:41 nickel: (   147.764 @ 0x345a708 / headerdump.debug) "x-kobo-deviceos" = "4.1.15" 
Nov  1 22:52:41 nickel: (   147.765 @ 0x345a708 / headerdump.debug) "x-kobo-deviceosversion" = "NA" 
Nov  1 22:52:41 nickel: (   147.765 @ 0x345a708 / headerdump.debug) "x-kobo-platformid" = "00000000-0000-0000-0000-000000000376" 
Nov  1 22:52:41 nickel: (   147.765 @ 0x345a708 / headerdump.debug) "User-Agent" = "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
Nov  1 22:52:41 nickel: (   147.765 @ 0x345a708 / headerdump.debug) "Accept-Language" = "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
Nov  1 22:52:41 nickel: (   147.766 @ 0x345a708 / packetdump.debug) -------------------------- REQUEST --------------------------- 
Nov  1 22:52:41 nickel: (   147.766 @ 0x345a708 / packetdump.debug) "PATCH" > To:  "/api/v3/content/<user_id?>/annotations" 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) 	---------------------- HEADERS --------------------------- 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) "Accept-Encoding" : "gzip" 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) "Accept" : "application/json" 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) "Authorization" : "Bearer xxx" 
Nov  1 22:52:41 nickel: (   147.767 @ 0x345a708 / packetdump.debug) "Content-Type" : "application/json" 
Nov  1 22:52:41 nickel: (   147.768 @ 0x345a708 / packetdump.debug) "x-kobo-affiliatename" : "fnac" 
Nov  1 22:52:41 nickel: (   147.768 @ 0x345a708 / packetdump.debug) "x-kobo-appversion" : "4.28.18220" 
Nov  1 22:52:41 nickel: (   147.768 @ 0x345a708 / packetdump.debug) "x-kobo-deviceid" : "xxx" 
Nov  1 22:52:41 nickel: (   147.768 @ 0x345a708 / packetdump.debug) "x-kobo-devicemodel" : "Kobo Clara HD" 
Nov  1 22:52:41 nickel: (   147.769 @ 0x345a708 / packetdump.debug) "x-kobo-deviceos" : "4.1.15" 
Nov  1 22:52:41 nickel: (   147.769 @ 0x345a708 / packetdump.debug) "x-kobo-deviceosversion" : "NA" 
Nov  1 22:52:41 nickel: (   147.769 @ 0x345a708 / packetdump.debug) "x-kobo-platformid" : "00000000-0000-0000-0000-000000000376" 
Nov  1 22:52:41 nickel: (   147.769 @ 0x345a708 / packetdump.debug) "User-Agent" : "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
Nov  1 22:52:41 nickel: (   147.770 @ 0x345a708 / packetdump.debug) "Accept-Language" : "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
Nov  1 22:52:41 nickel: (   147.770 @ 0x345a708 / packetdump.debug) "Content-Length" : "23192" 
Nov  1 22:52:41 nickel: (   147.770 @ 0x345a708 / packetdump.debug) 	---------------------- BODY --------------------------- 
Nov  1 22:52:41 nickel: (   147.770 @ 0x345a708 / packetdump.debug) "{"updatedAnnotations": [{"clientLastModifiedUtc": "2021-11-01T13:10:10Z","highlightedText": "If something is true, no amount of wishful thinking can undo it.","id": "ede9e626-23d2-45da-864e-5d1b253cdc79","location": {"span": {"chapterFilename": "text/part0004.html","chapterProgress": 0.88888888888888884,"chapterTitle": "Introduction to 30th Anniversary Edition","endChar": 64,"endPath": "span#kobo\\.30\\.1","startChar": 0,"startPath": "span#kobo\\.30\\.1"}},"type": "Highlight"},...
Nov  1 22:52:41 nickel: (   147.772 @ 0x345a708 / packetdump.debug) -------------------------- END OF REQUEST --------------------------- 
Nov  1 22:52:41 nickel: (   147.772 @ 0x345a708 / packetdump.warning) "/api/v3/content/<user_id?>/annotations" => "Protocol "" is unknown" 
Nov  1 22:52:41 nickel: (   147.773 @ 0x345a708 / packetdump.warning) HTTP Status Code: 0 
Nov  1 22:52:41 nickel: (   147.773 @ 0x345a708 / packetdump.warning) Error: 301 
Nov  1 22:52:41 nickel: (   147.773 @ 0x345a708 / packetdump.warning) Source: "/api/v3/content/<user_id?>/annotations" 
Nov  1 22:52:41 nickel: (   147.773 @ 0x345a708 / packetdump.warning) ""
  1. DownloadContentAnnotationsCommand - "GET" > To: "/api/v3/content/<user/book_id?>/annotations?limit=100"
DownloadContentAnnotationsCommand request 
Nov  1 22:52:41 nickel: (   147.814 @ 0x345a708 / sync.debug) pumping with 1 pending commands and 0 active commands 
Nov  1 22:52:41 nickel: (   147.814 @ 0x345a708 / sync.debug) executing CheckForAnnotationChangesCommand(0x31c9db8) 
Nov  1 22:52:41 nickel: (   147.814 @ 0x345a708 / sync.debug) virtual void OAuthAuthenticatedCommand::execute() 
Nov  1 22:52:41 nickel: (   147.815 @ 0x345a708 / sync.debug) =================== SYNC QUEUE [ SyncClient(0x36d1100) ] ==================== 
Nov  1 22:52:41 nickel: (   147.815 @ 0x345a708 / sync.debug)  ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.815 @ 0x345a708 / sync.debug)  [ SyncLibraryCommand(0x30eb768) ] 
Nov  1 22:52:41 nickel: (   147.816 @ 0x345a708 / sync.debug) 	 ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.816 @ 0x345a708 / sync.debug) 	 [ SyncAnnotationsCommandV2(0x31c9ec8) ] 
Nov  1 22:52:41 nickel: (   147.816 @ 0x345a708 / sync.debug) 		 ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.816 @ 0x345a708 / sync.debug) 		 [ GetUpdatedAnnotationsCommand(0x33eb580) ] 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug) 			 ----- Running ------ 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug) 			 [ CheckForAnnotationChangesCommand(0x31c9db8) ] 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug) 			 ----- Queued ------- 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x33a16f8) ] 
Nov  1 22:52:41 nickel: (   147.817 @ 0x345a708 / sync.debug)  ----- Queued ------- 
Nov  1 22:52:41 nickel: (   147.818 @ 0x345a708 / sync.debug)  [ OverDriveSyncCommand(0x36b68d0) ] 
Nov  1 22:52:41 nickel: (   147.818 @ 0x345a708 / sync.debug)  [ PocketSyncCommand(0x36dd170) ] 
Nov  1 22:52:41 nickel: (   147.818 @ 0x345a708 / sync.debug)  [ UpdateAllRelatedItemsCommand(0x36ac330) ] 
Nov  1 22:52:41 nickel: (   147.818 @ 0x345a708 / sync.debug)  [ UpdateNextInSeriesCommand(0x32c6178) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ SyncRecommendationsCommand(0x36db388) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ SyncTopPicksCommand(0x36c82e0) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ SyncMostPopularTabCommand(0x36d9700) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ SubmitAnalyticsEventsCommand(0x329ab38) ] 
Nov  1 22:52:41 nickel: (   147.819 @ 0x345a708 / sync.debug)  [ ReviewsSyncCommand(0x32b97b0) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ SyncUserGuideCommand(0x3a5af30) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ SyncDictionariesCommand(0x36dcb50) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ FrictionlessReadingCommand(0x379bc48) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ DownloadQueuedBooksCommand(0x375acb8) ] 
Nov  1 22:52:41 nickel: (   147.820 @ 0x345a708 / sync.debug)  [ CleanupCommand(0x30c4f20) ] 
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug) =================================================== 
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug)    
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug) pumping with 1 pending commands and 1 active commands 
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug) executing DownloadContentAnnotationsCommand(0x33a16f8) 
Nov  1 22:52:41 nickel: (   147.821 @ 0x345a708 / sync.debug) virtual void OAuthAuthenticatedCommand::execute() 
Nov  1 22:52:41 nickel: (   147.823 @ 0x345a708 / headerdump.debug) "Accept-Encoding" = "gzip" 
Nov  1 22:52:41 nickel: (   147.823 @ 0x345a708 / headerdump.debug) "Accept" = "application/json" 
Nov  1 22:52:41 nickel: (   147.823 @ 0x345a708 / headerdump.debug) "Authorization" = "Bearer xxx" 
Nov  1 22:52:41 nickel: (   147.823 @ 0x345a708 / headerdump.debug) "Content-Type" = "application/json" 
Nov  1 22:52:41 nickel: (   147.824 @ 0x345a708 / headerdump.debug) "x-kobo-affiliatename" = "fnac" 
Nov  1 22:52:41 nickel: (   147.824 @ 0x345a708 / headerdump.debug) "x-kobo-appversion" = "4.28.18220" 
Nov  1 22:52:41 nickel: (   147.824 @ 0x345a708 / headerdump.debug) "x-kobo-deviceid" = "xxx" 
Nov  1 22:52:41 nickel: (   147.825 @ 0x345a708 / headerdump.debug) "x-kobo-devicemodel" = "Kobo Clara HD" 
Nov  1 22:52:41 nickel: (   147.825 @ 0x345a708 / headerdump.debug) "x-kobo-deviceos" = "4.1.15" 
Nov  1 22:52:41 nickel: (   147.825 @ 0x345a708 / headerdump.debug) "x-kobo-deviceosversion" = "NA" 
Nov  1 22:52:41 nickel: (   147.826 @ 0x345a708 / headerdump.debug) "x-kobo-platformid" = "00000000-0000-0000-0000-000000000376" 
Nov  1 22:52:41 nickel: (   147.826 @ 0x345a708 / headerdump.debug) "User-Agent" = "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
Nov  1 22:52:41 nickel: (   147.826 @ 0x345a708 / headerdump.debug) "Accept-Language" = "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
Nov  1 22:52:41 nickel: (   147.827 @ 0x345a708 / packetdump.debug) -------------------------- REQUEST --------------------------- 
Nov  1 22:52:41 nickel: (   147.827 @ 0x345a708 / packetdump.debug) "GET" > To:  "/api/v3/content/<user_id?>/annotations?limit=100" 
Nov  1 22:52:41 nickel: (   147.827 @ 0x345a708 / packetdump.debug) 	---------------------- HEADERS --------------------------- 
Nov  1 22:52:41 nickel: (   147.827 @ 0x345a708 / packetdump.debug) "Accept-Encoding" : "gzip" 
Nov  1 22:52:41 nickel: (   147.829 @ 0x345a708 / packetdump.debug) "Accept" : "application/json" 
Nov  1 22:52:41 nickel: (   147.829 @ 0x345a708 / packetdump.debug) "Authorization" : "Bearer xxx" 
Nov  1 22:52:41 nickel: (   147.830 @ 0x345a708 / packetdump.debug) "Content-Type" : "application/json" 
Nov  1 22:52:41 nickel: (   147.830 @ 0x345a708 / packetdump.debug) "x-kobo-affiliatename" : "fnac" 
Nov  1 22:52:41 nickel: (   147.830 @ 0x345a708 / packetdump.debug) "x-kobo-appversion" : "4.28.18220" 
Nov  1 22:52:41 nickel: (   147.830 @ 0x345a708 / packetdump.debug) "x-kobo-deviceid" : "xxx" 
Nov  1 22:52:41 nickel: (   147.831 @ 0x345a708 / packetdump.debug) "x-kobo-devicemodel" : "Kobo Clara HD" 
Nov  1 22:52:41 nickel: (   147.831 @ 0x345a708 / packetdump.debug) "x-kobo-deviceos" : "4.1.15" 
Nov  1 22:52:41 nickel: (   147.831 @ 0x345a708 / packetdump.debug) "x-kobo-deviceosversion" : "NA" 
Nov  1 22:52:41 nickel: (   147.831 @ 0x345a708 / packetdump.debug) "x-kobo-platformid" : "00000000-0000-0000-0000-000000000376" 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) "User-Agent" : "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) "Accept-Language" : "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) 	---------------------- BODY --------------------------- 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) "" 
Nov  1 22:52:41 nickel: (   147.832 @ 0x345a708 / packetdump.debug) -------------------------- END OF REQUEST --------------------------- 
Nov  1 22:52:41 nickel: (   147.880 @ 0x345a708 / packetdump.warning) "/api/v3/content/<user/book_id?>/annotations?limit=100" => "Protocol "" is unknown" 
Nov  1 22:52:41 nickel: (   147.880 @ 0x345a708 / packetdump.warning) HTTP Status Code: 0 
Nov  1 22:52:41 nickel: (   147.881 @ 0x345a708 / packetdump.warning) Error: 301 
Nov  1 22:52:41 nickel: (   147.881 @ 0x345a708 / packetdump.warning) Source: "/api/v3/content/<user/book_id?>/annotations?limit=100"
  1. CheckForAnnotationChangesCommand - "POST" > To: "/api/v3/content/checkforchanges", with book IDs in the body
CheckForAnnotationChangesCommand request 
> Nov  1 22:52:41 nickel: (   147.862 @ 0x345a708 / sync.debug) pumping with 4 pending commands and 2 active commands 
> Nov  1 22:52:41 nickel: (   147.862 @ 0x345a708 / sync.debug) =================== SYNC QUEUE [ SyncClient(0x36d1100) ] ==================== 
> Nov  1 22:52:41 nickel: (   147.862 @ 0x345a708 / sync.debug)  ----- Running ------ 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug)  [ SyncLibraryCommand(0x30eb768) ] 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug) 	 ----- Running ------ 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug) 	 [ SyncAnnotationsCommandV2(0x31c9ec8) ] 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug) 		 ----- Running ------ 
> Nov  1 22:52:41 nickel: (   147.863 @ 0x345a708 / sync.debug) 		 [ GetUpdatedAnnotationsCommand(0x33eb580) ] 
> Nov  1 22:52:41 nickel: (   147.864 @ 0x345a708 / sync.debug) 			 ----- Running ------ 
> Nov  1 22:52:41 nickel: (   147.864 @ 0x345a708 / sync.debug) 			 [ CheckForAnnotationChangesCommand(0x31c9db8) ] 
> Nov  1 22:52:41 nickel: (   147.864 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x33a16f8) ] 
> Nov  1 22:52:41 nickel: (   147.864 @ 0x345a708 / sync.debug) 			 ----- Queued ------- 
> Nov  1 22:52:41 nickel: (   147.865 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x3a57020) ] 
> Nov  1 22:52:41 nickel: (   147.866 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x3a25868) ] 
> Nov  1 22:52:41 nickel: (   147.866 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x39e18c0) ] 
> Nov  1 22:52:41 nickel: (   147.866 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x3200400) ] 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug) 			 [ DownloadContentAnnotationsCommand(0x3578a40) ] 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug)  ----- Queued ------- 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug)  [ OverDriveSyncCommand(0x36b68d0) ] 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug)  [ PocketSyncCommand(0x36dd170) ] 
> Nov  1 22:52:41 nickel: (   147.867 @ 0x345a708 / sync.debug)  [ UpdateAllRelatedItemsCommand(0x36ac330) ] 
> Nov  1 22:52:41 nickel: (   147.868 @ 0x345a708 / sync.debug)  [ UpdateNextInSeriesCommand(0x32c6178) ] 
> Nov  1 22:52:41 nickel: (   147.868 @ 0x345a708 / sync.debug)  [ SyncRecommendationsCommand(0x36db388) ] 
> Nov  1 22:52:41 nickel: (   147.868 @ 0x345a708 / sync.debug)  [ SyncTopPicksCommand(0x36c82e0) ] 
> Nov  1 22:52:41 nickel: (   147.868 @ 0x345a708 / sync.debug)  [ SyncMostPopularTabCommand(0x36d9700) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ SubmitAnalyticsEventsCommand(0x329ab38) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ ReviewsSyncCommand(0x32b97b0) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ SyncUserGuideCommand(0x3a5af30) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ SyncDictionariesCommand(0x36dcb50) ] 
> Nov  1 22:52:41 nickel: (   147.869 @ 0x345a708 / sync.debug)  [ FrictionlessReadingCommand(0x379bc48) ] 
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug)  [ DownloadQueuedBooksCommand(0x375acb8) ] 
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug)  [ CleanupCommand(0x30c4f20) ] 
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug) =================================================== 
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug)    
> Nov  1 22:52:41 nickel: (   147.870 @ 0x345a708 / sync.debug) pumping with 5 pending commands and 2 active commands 
> Nov  1 22:52:41 nickel: (   147.872 @ 0x345a708 / headerdump.debug) "Accept-Encoding" = "gzip" 
> Nov  1 22:52:41 nickel: (   147.872 @ 0x345a708 / headerdump.debug) "Accept" = "application/json" 
> Nov  1 22:52:41 nickel: (   147.872 @ 0x345a708 / headerdump.debug) "Authorization" = "Bearer xxx" 
> Nov  1 22:52:41 nickel: (   147.873 @ 0x345a708 / headerdump.debug) "Content-Type" = "application/json" 
> Nov  1 22:52:41 nickel: (   147.873 @ 0x345a708 / headerdump.debug) "x-kobo-affiliatename" = "fnac" 
> Nov  1 22:52:41 nickel: (   147.873 @ 0x345a708 / headerdump.debug) "x-kobo-appversion" = "4.28.18220" 
> Nov  1 22:52:41 nickel: (   147.873 @ 0x345a708 / headerdump.debug) "x-kobo-deviceid" = "xxx" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "x-kobo-devicemodel" = "Kobo Clara HD" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "x-kobo-deviceos" = "4.1.15" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "x-kobo-deviceosversion" = "NA" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "x-kobo-platformid" = "00000000-0000-0000-0000-000000000376" 
> Nov  1 22:52:41 nickel: (   147.874 @ 0x345a708 / headerdump.debug) "User-Agent" = "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
> Nov  1 22:52:41 nickel: (   147.875 @ 0x345a708 / headerdump.debug) "Accept-Language" = "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
> Nov  1 22:52:41 nickel: (   147.875 @ 0x345a708 / packetdump.debug) -------------------------- REQUEST --------------------------- 
> Nov  1 22:52:41 nickel: (   147.876 @ 0x345a708 / packetdump.debug) "POST" > To:  "/api/v3/content/checkforchanges" 
> Nov  1 22:52:41 nickel: (   147.876 @ 0x345a708 / packetdump.debug) 	---------------------- HEADERS --------------------------- 
> Nov  1 22:52:41 nickel: (   147.876 @ 0x345a708 / packetdump.debug) "Accept-Encoding" : "gzip" 
> Nov  1 22:52:41 nickel: (   147.876 @ 0x345a708 / packetdump.debug) "Accept" : "application/json" 
> Nov  1 22:52:41 nickel: (   147.877 @ 0x345a708 / packetdump.debug) "Authorization" : "Bearer xxx" 
> Nov  1 22:52:41 nickel: (   147.877 @ 0x345a708 / packetdump.debug) "Content-Type" : "application/json" 
> Nov  1 22:52:41 nickel: (   147.877 @ 0x345a708 / packetdump.debug) "x-kobo-affiliatename" : "fnac" 
> Nov  1 22:52:41 nickel: (   147.877 @ 0x345a708 / packetdump.debug) "x-kobo-appversion" : "4.28.18220" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-deviceid" : "xxx" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-devicemodel" : "Kobo Clara HD" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-deviceos" : "4.1.15" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-deviceosversion" : "NA" 
> Nov  1 22:52:41 nickel: (   147.878 @ 0x345a708 / packetdump.debug) "x-kobo-platformid" : "00000000-0000-0000-0000-000000000376" 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) "User-Agent" : "Mozilla/5.0 (Linux; U; Android 2.0; en-us;) AppleWebKit/538.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/538.1 (Kobo Touch 0376/4.28.18220)" 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) "Accept-Language" : "fr-FR, fr;q=0.9, en;q=0.8, *;q=0.7" 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) "Content-Length" : "430" 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) 	---------------------- BODY --------------------------- 
> Nov  1 22:52:41 nickel: (   147.879 @ 0x345a708 / packetdump.debug) "[{"ContentId": "c7b34121-0e04-4593-9d6c-7b8c823ca4e9","etag": "W/\"0\""},{"ContentId": "1a8d0291-5156-44f9-9faf-e86b149dd8e4","etag": "W/\"0\""},{"ContentId": "9753e5f4-8964-4178-92a7-9d0bc87d0445","etag": "W/\"0\""},{"ContentId": "885d68d4-92c7-4136-b318-e3d887c15495","etag": "W/\"A:820707582-Iq992+H1kkGYzpOgLXvzNA, B:462539101-KkAV6b3EYE+6aKdGXLBjow\""},{"ContentId": "258a01a6-ee56-4450-9884-8775f0b3e2df","etag": "W/\"0\""}]" 
> Nov  1 22:52:41 nickel: (   147.880 @ 0x345a708 / packetdump.debug) -------------------------- END OF REQUEST ---------------------------

The problem is that the endpoint is not the same as the store endpoint (probably api.kobobooks.com), and so we cannot redirect this request to Calibre-Web - the URL is hardcoded somewhere. I would love to be wrong, but the only solution might be to request a patch from pgaskin/kobopatch to permanently modify this URL.

Also the requests seems really weird, as if the feature was half implemented in the Kobo readers and the requested URL is never completed, so I don't have any example of a successful response.

Perhaps try monitoring the ereader with wireshark / netcat? Then you can see the destination and figure out where all this data is going to.

I'll try this myself when I have some free time.

JVT038 avatar Nov 02 '21 22:11 JVT038

I have some notes about this here (I've been intermittently working on my own sync implementation): https://github.com/pgaskin/kobo-plugin-experiments/issues/29#issuecomment-891301151

pgaskin avatar Nov 11 '21 04:11 pgaskin

@pgaskin FYI most of the sync features have been implemented in calibre-web, books and libraries all work pretty well. It is only the notes/annotations that’s missing.

This repo is a private one, do you mind sharing your notes regarding this ? Thanks !

a1ex4 avatar Nov 11 '21 17:11 a1ex4

The reason I've been working on my own implementation is: to understand sync better, since I don't use Calibre, to have a kobo-oriented approach (rather than a calibre-oriented one), to integrate with other reading apps (specifically Lithium), for practice, and to try a different approach to the sync and sync token algorithm.

Here's the contents of the linked comment (there is a bit more information scattered in the other ones, but this is most of it):

Some rough notes about the readingservices annotations API (I'll clean these up later):

request with incorrect content-type: 
{
  "type": "https://tools.ietf.org/html/rfc7231#section-6.5.13",
  "title": "Unsupported Media Type",
  "status": 415,
  "traceId": "00-aa253bcb6d42a04bbb3072ffe7731667-bb28f051d8a5b348-00"
}

request to checkforchanges without revision IDs.

"No content ids to check."

request with invalid json:

{
  "type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
  "title": "One or more validation errors occurred.",
  "status": 400,
  "traceId": "00-7b0fa893e020bc48845d4199bb3bce20-3a6aaf0ca60ec347-00",
  "errors": {
    "$[0]": [
      "'C' is an invalid start of a property name. Expected a '\"'. Path: $[0] | LineNumber: 0 | BytePositionInLine: 2."
    ]
  }
}

etag is an opaque blob; if not provided to checkforchanges then it treats it as changed

checkforchanges returns an array of revision IDs

if any annotation changed (e.g. etag doesn't match), all are returned

kobo will fetch existing annotations before syncing new ones

for pagination, the first page returns a nextPageOffsetToken in the JSON (based on the limit url param), which is based on the next annotation ID to show sorted by the client modified date, but this is an implementation-specific detail anyways) to be passed in the url of future requests as the pageOffsetToken param. future requests should include the etag as If-Match, and if it doesn't match, the server returns 412 with the json response "The collection has changed since you retrieved the first page. Please refresh from the beginning.". The if-match header is ignored for requests without a pageOffsetToken. A blank pageOffsetToken is treated as no pageOffsetToken. If no if-match header is provided, the pageOffsetToken is still used.

Kobo's implementation is powered by asp.net core

always seems to return valid JSON no matter what

auth using Kobo OAuth bearer token

enum json fields are case-insensitive (e.g. type: dogear/note/highlight)

some random tests on the free macbeth collin's edition ebook (kepub):

HTTP/2 200 
content-type: application/json; charset=utf-8
etag: W/"A:844978244-Iq992+H1kkGYzpOgLXvzNA, B:486774625-KkAV6b3EYE+6aKdGXLBjow"
server: Kestrel
content-length: 1727
date: Mon, 02 Aug 2021 20:09:47 GMT

{
  "annotations": [
    {
      "id": "f6934933-dbea-4ffc-93bd-8ecea87a8fd2",
      "type": "highlight",
      "context": null,
      "clientLastModifiedUtc": "2021-08-02T20:09:46.1419073Z",
      "location": {
        "span": {
          "startPath": "span#kobo\\.8\\.4",
          "endPath": "span#kobo\\.8\\.4",
          "startChar": 125,
          "endChar": 220,
          "chapterTitle": "Scene II. A Camp Near Forres.",
          "chapterFilename": "OEBPS/part3.html",
          "chapterProgress": 0.6666666666666666
        }
      },
      "attachments": {},
      "highlightedText": " foe: Except they meant to bathe in reeking wounds, Or memorize another Golgotha, I cannot tell"
    },
    {
      "id": "54542baf-4306-48fe-9297-1b0cf6840496",
      "type": "dogear",
      "context": null,
      "clientLastModifiedUtc": "2021-08-01T17:39:43.1559032Z",
      "location": {
        "span": {
          "startPath": "OEBPS/part2.html#kobo.1.1",
          "endPath": "OEBPS/part2.html#kobo.1.1",
          "startChar": 0,
          "endChar": 0,
          "chapterTitle": "Act I.",
          "chapterFilename": "OEBPS/part2.html",
          "chapterProgress": 1
        }
      },
      "attachments": {},
      "highlightedText": ""
    },
    {
      "id": "a98a2a9e-897f-4781-98fd-b3a58745105c",
      "type": "dogear",
      "context": null,
      "clientLastModifiedUtc": "2021-08-02T20:05:09.8427617Z",
      "location": {
        "span": {
          "startPath": "OEBPS/part3.html#kobo.6.5",
          "endPath": "OEBPS/part3.html#kobo.6.5",
          "startChar": 0,
          "endChar": 0,
          "chapterTitle": "Scene II. A Camp Near Forres.",
          "chapterFilename": "OEBPS/part3.html",
          "chapterProgress": 0.6666666666666666
        }
      },
      "attachments": {},
      "highlightedText": ""
    },
    {
      "id": "9d8daa66-9424-4688-8e03-4c23c22d5a85",
      "type": "note",
      "context": null,
      "clientLastModifiedUtc": "2021-08-01T17:51:57.4238294Z",
      "location": {
        "span": {
          "startPath": "span#kobo\\.2\\.1",
          "endPath": "span#kobo\\.2\\.1",
          "startChar": 10,
          "endChar": 15,
          "chapterTitle": "Act I.",
          "chapterFilename": "OEBPS/part2.html",
          "chapterProgress": 1
        }
      },
      "attachments": {},
      "highlightedText": "Enter",
      "noteText": "Test 1"
    }
  ],
  "nextPageOffsetToken": null
}

http 204 on patch request

patch is done based on clientLastModifiedUtc being equal or newer and replaces the entire annotation object for each of the updatedAnnotations (i.e. it isn't a field-based delta); date is not validated to ensure it's in the past, but if it's not the right format:

{
  "type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
  "title": "One or more validation errors occurred.",
  "status": 400,
  "traceId": "00-3c328570f52ade46a15645c0f6785016-f76a7ae6868f2049-00",
  "errors": {
    "request": [
      "No updates or deletes in request."
    ],
    "$.updatedAnnotations[1].clientLastModifiedUtc": [
      "The JSON value could not be converted to System.DateTime. Path: $.updatedAnnotations[1].clientLastModifiedUtc | LineNumber: 0 | BytePositionInLine: 430."
    ]
  }
}

-HAccept:application/json -XPATCH -HContent-Type:application/json --data '[]' https://readingservices.kobo.com/api/v3/content/c67da7ab-2fc1-4429-bd24-6abf41cfb9c8/annotations

{"type":"https://tools.ietf.org/html/rfc7231#section-6.5.1","title":"One or more validation errors occurred.","status":400,"traceId":"00-b815d60689118341a0f186f15793c6d3-fa3cad1770013140-00","errors":{"$":["The JSON value could not be converted to Kobo.ReadingServices.Annotations.Api.V3.PatchAnnotationsRequest. Path: $ | LineNumber: 0 | BytePositionInLine: 1."],"request":["No updates or deletes in request."]}}

also, apparently, it doesn't validate the content ID, and you can use arbitrary identifiers there, and it'll save it

libnickel stuff:

  • sync removes all annotations not returned by removing where not updated by download annotations request
  • etag is per-book and must change if any annotations change

pgaskin avatar Nov 11 '21 18:11 pgaskin

@OzzieIsaacs are you willing to look into this and/or implement it?

Or should I close this issue?

JVT038 avatar Dec 04 '21 13:12 JVT038

Haven‘t decided yet

OzzieIsaacs avatar Dec 04 '21 13:12 OzzieIsaacs

it would be great if the annotations and note from kobo reader could be sync and view in calibre web.

DHS1001 avatar May 13 '22 04:05 DHS1001

There is maybe a way to work with Koblime?

  • https://www.reddit.com/r/kobo/comments/w5hm48/call_to_help_beta_test_wireless_highlight_backups/
  • https://kobli.me
  • https://koblime.canny.io

Thovi98 avatar Aug 26 '22 07:08 Thovi98