Wombag not downloading all articles the first time.

[karthink]

Did it download your Wallabag articles correctly, and does search work as advertised?

No, in my instance I have 89 articles and only 59 have been downloaded.

@hefistion Creating a new issue to track the download issue. The fact it downloaded exactly 30 articles less than you have is a helpful clue, I'll look into it.

For now, could you do the following?

1. Open wombag (M-x wombag)
2. Call wombag-sync with a prefix argument (C-u G)

It will ask you for a date to sync forward from. Enter a date older than your oldest article, something like 1990-01-01. Does it download all 89 articles this time?

[hefistion]

1. Open wombag (`M-x wombag`)

2. Call `wombag-sync` with a prefix argument (`C-u G`)

It will ask you for a date to sync forward from. Enter a date older than your oldest article, something like 1990-01-01. Does it download all 89 articles this time?

The error persists, the 30 most recent posts are still missing.

I have tried to delete 5 articles in my wallabag instance and it syncs fine, now I have 54 articles.

[hefistion]

The new entries do appear after wombag-sync, the problem seems to be that during the first sync it does not download the first 30 posts.

[karthink]

The new entries do appear after wombag-sync, the problem seems to be that during the first sync it does not download the first 30 posts.

Yeah, this is what I suspected. It's a little work to test since I have to start with an empty DB and download all my articles (all 15,000 of them!) every time.

Now that all the articles are there, do the other problems still persist? (tags, archived articles missing etc)

[hefistion]

Now that all the articles are there, do the other problems still persist? (tags, archived articles missing etc)

So far I have found the following problems:

• In the help (?) to search for articles it indicates that to search by tags it must be with +tag but for it to work you have to put tag
• The first 30 articles do not appear on the list

Everything else seems to be working well, now the archived publications are displayed (&&), it's ok., great job, thank you very much.

[krisbalintona]

I'd like to chime in here.

@hefistion seemed to indicated that all was well for him. On the other hand, This issue (I think) persists for me. No matter what, Wombag always shows 30 entries less than what I have, no matter the way I use wombag-sync. Wombag seems to retrieve all of them properly (based on the updates shown in the header line), but 30 less are shown and reported in the total count.

Despite my attempts, I haven't been able to pinpoint the problem.

[goofansu]

In my case, it downloaded all the articles (74 articles), and then deleted 30 articles, remaining 44 articles in the first sync.

[krisbalintona]

For some reason this is a persistent issue for me. Wombag will inconsistently not download all my entries. I did some digging and:

1. All the entries are being downloaded, but some of they are immediately being deleted. The "Retrieving... X entries added" indicator communicates this.
2. The wombag-sync function is recursive, retrieving pages until all are retrieved. The function is recursively called via a run-with-idle-timer. Calling wombag-sync immediately rather than via timer fixes the issue for me. I don't know if there are any unintended side effects, but I haven't noticed any — what is the role of the idle timer? I think that the idle timer is unintentionally causing items to be deleted somehow, given that within wombag-sync there is a call to wombag-db-delete: wombag-db-delete is probably being called too late then deleting entries that were just added but shouldn't be deleted.

@karthink Thoughts?

[krisbalintona]

Update:

I think I found a fix. I pushed the commit to my fork here: https://github.com/krisbalintona/wombag/commit/32f87c349254abb7b53ba985b98ee9cd2911a0f6.

Seems like this is an issue with local-ids not being "in sync" with server-ids -- or something in the ballpark of that.