tomboy-ng
tomboy-ng copied to clipboard
tomboy-notes
synchronization: add support for automated synchronization with frequency
In the synchronization setting screen, it would be great to be able to set up automated synchronization with frequency:
- hourly
- daily
- weekly
- monthly
Thanks!
Yes, I agree, good idea. I'll look at that at some stage. (Sorry for slow response, have been traveling and seriously out of contract).
Davo
OK, thinking out loud here. Drop the Sync Enabled box and instead offer - Sync : [] Off [] Hourly [] Daily [] Weekly
Will require replacing existing settings in config file, FileSyncEnabled and GitSyncEnabled (both boolean) with new ones, perhaps string, off, hour, day, month. Old setting must remain to ensure backward compatibility. Day and Month will also require saving the time and date of last sync. Will need to dwell on this, hate changing config meanings ....
But overall, sounds reasonable.
No, 'Off' does not mean the same as not auto. If not auto, user may still trigger a sync manually.
David
This seems OK, superficially tested. Clearer code, slightly simpler user interface, good ! Dropped checkboxes 'enabled' and 'auto'. Combo box, 'manual', 'hourly' etc.
OK, there is a beta with 0.36e in the file name, here - https://github.com/tomboy-notes/tomboy-ng/releases/tag/v0.36c
I don't remember what platform you use and there are a lot of files in that directory. Look for the one with 'e' in its version number.
If you are using Linux and Qt5, you must get a new libqt5pas library, see link in that page.
Davo
installed 0.36e. I can see the new sync option, thanks!
however, it's only available for local filesystem sync, but it's disabled for Github sync (which is my use case).
Oh, that is not intentional ! I'll look into it ...... Fortunately, my debug code is still in there. Davo
OK, seems to work for me. You realise you set the frequency for each sort of sync individually ? Can you please see if any messages are being dropped to console ? I wasted an hour or so tracing a bug here that was being caused by what I discovered to be a very slow (ie timeout) DNS. You should see an error message if that happening but I am pretty sure its a local to our network problem rather than github.
What OS do you use ? (if Linux, gtk2 or Qt5 ?) - I could send you a debug binary that might be helpful.
Davo
OK, GTK2, 64 bit linux binary attached. In your existing tomboy-ng, make sure File Sync is set to manual, github to hourly, exit. Then just untar the attached, drop the binary in, eg, your home directory. Make sure your real tomboy-ng is NOT running, start this one from the command line. It will pick up your notes and config so behaves just like your existing one except that drops messages to console and it's sync cycle is running 100x faster. So, we get several cycles in 4 or 5 minutes. Thats all I need, take a screen shot or a screen scrap and copy to here. Pretty sure nothing of private nature should be shown but do check before posting !
Davo
OK, this issue resolved in https://github.com/tomboy-notes/tomboy-ng/wiki/Download_Release
So closing this issue, thanks for your report.
Davo
Hi Davo, FYI, in settings, I set and see "Github sync". But with right click, synchronize, it says cannot synchronize to /tmp as if it considers a local folder for synchronization.
Hi Martin, sorry, this ticket was closed so I did not notice you message until just now. I have reopened it.
Did you setup your tomboy-ng repo on github ? Or are you telling me it went ahead and tried to sync without a proper config ? That would be messy indeed.
Davo
Tried again. The actual problem is that I cannot complete the first sync. It ends with "List index (3) out of bounds."
Sorry, your really need to need to spell out exactly what you are doing/trying to do I am afraid. Without that information, I really am pretty useless. Just to focus down a little -
Can you please tell me what version of tomboy-ng you are using ? On what platform ? And, platform specific, do you know the version of openssl you are using ? That has become quite critical now that many platforms are using openssl 3.0 and some versions of tomboy-ng are not happy about that.
You are trying to do a github sync ? Assuming yes, you have added a valid security token to your tomboy-ng config ? And how far did you get with that sync ? An initial sync is a two stage thing, it does a dummy run to see if everything is OK and then asks you to confirm and save. Did you get that far ?
Does the private github repository (TB_notes) now exist in your github account ? You can only see it when logged into github. Does it contain an index and some notes ?
Background info. I am particularly concerned about an openssl issue I just found out about in the last couple of weeks. But I don't think that is your problem because the error "should" be well trapped and you get a sensible error message to the effect that it cannot initialize openssl. Whereas the error you see about a list index could come from anywhere and is that dreaded un-handled error IMHO. Some background - I believe, but am not certain that tomboy-ng compiled with fpc322 will not work with on 'some' platforms with openssl 3.0. In Linux, its easily fixed, install libssl-dev. I don't know of any easy fix on Windows (but generally, Windows lags way behind with things like openessl anyway. I have not had time to test on the Mac.
Davo
Can you please tell me what version of tomboy-ng you are using ? On what platform ?
0.39 on Linux
And, platform specific, do you know the version of openssl you are using ?
3.0.2-0ubuntu1.15
You are trying to do a github sync ?
yes
you have added a valid security token to your tomboy-ng config ?
yes
And how far did you get with that sync ?
Starts, "Save & Sync", detects clashes, ask for what to do, say "Use local for all", then fails with "List index (3) out of bounds", does not complete sync.
Martin, I am currently traveling and cannot setup a test rig. But will make a few suggestions.
Firstly, I still have some concerns about version 0.39 of tomboy-ng and the openssl you are using. I believe you "should" get a clear openssl error message if that the problem (and I'd be conducting tests in that direction if I could).
I do have a beta test version out at present that will be much happier with your openssl, maybe worth looking at.
I don't, without some testing, understand what the list index issue is all about, I cannot see how you'd get there if you do have openssl issues. I understand you would not be able to connect at all.
Can you please look, in your github repository and see if - Does the private github repository (TB_notes) now exist in your github account ? You can only see it when logged into github. Does it contain an index and some notes ?
From your description, it sounds to me as if you did not, at any stage, get to the "Save and Sync" stage ? Crash while still reading your local repo (and presumably NO remote repo) ? Thats important info for me, sounds just a bit OpenSSL again. But maybe, github's extra level of security, they recently added two factor authentication, you may need to do that two factor thing before proceeding ?? I will have already done so because I commit to my github account all the time . So I would not see this problem. Maybe ....
This extra level of github security could turn our to be the problem now I think about it, may be a serious problem...
Davo
Does the private github repository (TB_notes) now exist in your github account ? yes
Does it contain an index and some notes ? yes
it sounds to me as if you did not, at any stage, get to the "Save and Sync" stage ? maybe partially since the repo exists and contain some content
thanks!
Sorry for slow response, I have been travelling.
I have confirmed that the existing tomboy-ng v0.39 will not work with openssl 3.0 so it would seem likely this is your problem. If you do have some files in your github notes repo, could that repo have been made before you updated your desktop OS ? In particular, that would have got you a new openssl.
The beta I have just pushed to https://github.com/tomboy-notes/tomboy-ng/releases/tag/v0.39a solves this openssl problem. (note github tag says v0.39a but content is v0.39c, released 18th April).
Sorry about this. Problem is that the current formal FPC release does not support openssl 3.0, I have built these with a (very well tested) beta compiler. Sigh ...
Davo
I downloaded the last version. Same problem. I'm sure OpenSSL works because some notes are pushed.
Looking at .config/tomboy-ng/tomboy-ng.cfg
[SyncSettings]
Autosync=false
SyncOption=AlwaysAsk
SyncType=file
SyncNextCloudRepo=
SyncRepo=/tmp/
AutosyncGit=false
FileSyncEnabled=false
GitSyncEnabled=false
SyncRepoGithub=
GHPassword=<valid key>
GHUserName=monperrus
SyncTimingFile=daily
SyncTimingGithub=manual
What should be SyncType for Github? What's the format of SyncRepoGithub value?
Sorry Martin, been distracted.
From memory, SyncType is not used with GitHub sync. Looking at my config, whats important appears to be -
SyncRepoGithub=https://github.com/davidbannon/
It also observes the common ones about timing Autosync etc.
So, your GitSyncRepo appears to be blank (unless you blanked it before posting ?). If its really blank, no sync is going to happen. Your GitSync is also disabled. That says to be it was once configured, is not usable now.
I'm guessing you have tried to re-configure with the "change sync repository" button.
Its (intended to be) always safe to resync or reconfigure sync with tomboy-ng, unlike Tomboy where you would end up with a heap of duplicated notes. If you have not tried to resync, please do so.
Next step might be to blow away the repo, allow -ng to create a new one, again, should be safe.
Why is going on ? Beats me. One thing that keeps nagging me is the possibility you had a working sync, one way or another, your openssl was updated, bad things happened. That bad thing is fixed in that beta you are now using, don't go back to the old one. I accept the old one would not be able to sync, however, surprised that what ever went wrong left it unrecoverable. If I get some time, I'll try and simulate that situation on a test rig and see what happens.
Davo
SyncRepoGithub=https://github.com/davidbannon/
GHPassword=
GHUserName=davidbannon
AutosyncGit=true
GitSyncEnabled=true
SyncRepo=tb_notes
Thanks, aligned accordingly. I understand that SyncRepoGithub is concatenated with SyncRepo.
More debug info:
- I rightlick on the tomboy tray, then synchronize
- in the console, I see
syncgui.pas, ManualSync(), line:352 : Test Transport Failed.
OK, thats useful, when that console msg appears, I expect you to be doing a Manual Sync and you should see a (hopefully) more useful message in the tomboy-ng window that is open at that time. Is that the message you mentioned on the 17th March ? eg "List index (3) out of bounds." ?
If so, firstly, thats not one of my messages, that in itself is interesting. It appears you are calling my function TSync.TestConnection(): TSyncAvailable; and getting one of the TSyncAvailable enumerated types back. All the "error" conditions should give a further error message and you are obviously not seeing that.
So, I suspect you are getting that error from Transport.TestTransport(), "Transport", in this case being the github transport class. Its the only one that can return an error without me translating a low level error into a more meaningful one. (Because, I suspect, I did not expect, or could not trigger, an error of that nature.)
This method makes initial contact with Github, gets back and decodes some JSON, makes some decisions about if its safe to proceed. The JSON decoding and returned data all use lists and that likely whats going wrong.
Please give me a couple of days (travelling again, sorry !) to put together a binary that will generate some much more explicit error message (useful to have in the code anyway now you have demonstrated its need).
Davo
Martin, I have now added some better error checking to the github sync, will make you up a tarball if you tell me if you use the Qt5, Qt6 or gtk2 version of tomboy-ng please ?
So far, this "better error checking" has revealed, for me, an occasional timeout when talking to github's servers. I am unsure if it has been happening in the past, it could be due to my current poor network connection (my phone left sitting on an upturned bucket on a small ridge near our caravan). But it is happening. I have not, at this stage, built in some retries following a timeout, I would prefer to try and catch the problem "red handed" with your very tolerant help !
Sadly, a closer examination of the code has not revealed any list likely to be the cause of the exception you see. And I still cannot replicate your problem !
Davo
Ah, well timed ! We are in town with a decent connection !
Attached a zip containing "tomboy-ng-text", unzip, don't bother to install, just run, from command line, must use eg
$> ./tomboy-ng-test --debug-sync > log.txt
It will generate a lot of noise during a sync, including listing names of notes so check content before posting !
Davo tomboy-ng-test.zip
Here is the result of Click on tomboy icon in system tray >> Synchronise
TFormSync.FormShow
Remote address is tb_notes/
Local Config /home/martin/.config/tomboy-ng/
Notes dir /home/martin/.local/share/tomboy-ng/
Reading local mainfest /home/martin/.config/tomboy-ng/manifest.xml
TSync.TestConnection() : failed Transport.TestTransport, SyncNoRemoteDir : Perhaps sync device or shared drive is not mounted ?
syncgui.pas, ManualSync(), line:354 : Test Transport Failed, SyncNoRemoteDir : Perhaps sync device or shared drive is not mounted ?
Failed testConnection
TFormSync.FormShow
TGithubSync.SetTransport - called
Remote address is
Local Config /home/martin/.config/tomboy-ng/SyncGithub/
Notes dir /home/martin/.local/share/tomboy-ng/
ReadLocalManifest set an empty LocalLastSyncDateSt, probably local manifest does not exist.
syncgui.pas, ManualSync(), line:354 : Test Transport Failed, SyncNoLocal : We dont have a local manifest, only an error if config thinks there should be one
Failed testConnection
with
[SyncSettings]
Autosync=false
SyncOption=AlwaysAsk
SyncType=file
SyncNextCloudRepo=
SyncRepo=tb_notes
AutosyncGit=true
FileSyncEnabled=false
GitSyncEnabled=true
SyncRepoGithub=https://github.com/monperrus/
GHPassword=XXXXXX
GHUserName=monperrus
SyncTimingFile=daily
SyncTimingGithub=manual
thanks!
OK, key here is this line -
TSync.TestConnection() : failed Transport.TestTransport, SyncNoRemoteDir : Perhaps sync device or shared drive is not mounted ?
I understand you do, definitly, have a 'tb-notes' repo in github account. The above problem then could be caused by two things :
- The github token does not allow reading of the remote (that is on github) manifest or serverID file.
- The above files do not exist.
Easy check, in your browser, login to github, go to your the tb_notes repository, look in the Meta directory. You should see two files, serverid and manifest.json. serverid has a single line with an GUID, manifest has a some json listing the notes in the repo. If files are present and not empty, then I strongly suspect your github token does not have "repoository scope".
If either file is not present, then we have to assume that 'something' has broken during a sync and its not recoverable. I am not sure just what that could be, I do a "journal like" write to github and all my tests, including breaking in the middle of a sync, were recoverable on the next run.
Anyway, if its broken, and you don't have notes on github that are not present on your local PC, I suggest you remove the github tb_notes repo and start again. Again, my test indicate that should work, -ng will detect no repo present at the other end, will offer to create one for you and then do a full sync.
Davo