Database merge and/or network sync

[eliotb]

I have run hamster on more than one machine. It would be nice to have a way to get an overview of the activity on them all. This might be accomplished by...

Merge data from ssh://user@host or even manually getting the sqlite database and importing locally

Or if machines are all running and network accessible, automatically sync between them

Or a way to build a report that combines all the data without actually merging the databases.

[elbenfreund]

I was investigating a similar use case myself recently and as far as I see there seem to be at least two ways about this:

• Create an alternative Storage class in db.py that provides the same interface but access a potentialy remote db server instance (Postgres?). that way all db transactions get handlet transparently and no changes to the "client" should be needed.
• Investigate the work done on the flask/webinterface and make the client talk to the flask exposed API

btw: is there any -dev mailinglist available? Is there interest in having one?

[elbenfreund]

I don't want to nag, but is there any medium for discussing general design/development milestones/goalposts? I feel that hamster would profit a great deal from a distributed architecture. A goal much further down the road may be support for groups and collaborative tasks. But for now, even a relatively simple rest based approach would be a huge step forward...

[benzoid]

How about simply an import function that will read in the XML format that is generated in the already existing "Save report..."->XML (or TSV)?

[elbenfreund]

I once wrote a basic python script to do something like that. But besides not beeing polished enogh to be shown the problem with batch processing data like this is that one needs to have an (potentially ellaborate) way to handle overlaps/duplicates in order to maintain data integrety.

[elbenfreund]

As part of my current work on hamster I have summerized the state of afairs regarding this issue here

[FelixSchwarz]

I had a very brief conversation with @tbaugis about this last year (by email) and I think his answer might give some guidance on how this could be implemented:

After simple configuration in which you would tell hamster which categories/tasks you would like to synchronize, hamster would perform regular updates and synchronize only the total time spent (so no punchcards). Such project would pave way for other integrations (redmine / basecamp whatever), but should be the primary means. The backend API is fully exposed via DBUS as the application utilizes it itself - so i don't think any work there is necessary. Improved integrations with other tools could be a plus (like the inklings of the GTG integration)

I'm not sure how we should approach this. Personally I don't think a storage-layer sync is the best idea here. For example I like the ability to work offline and only sync when I'm online again (e.g. I'm working in a plane but of course time-tracking needs to be updated once I'm at the airport/hotel). So this requires some "queuing" mechanism and I think an explicit sync protocol would be best.

Also using some kind of separate process just listening for DBUS (which could be started by hamster automatically) might be a nice way to keep the complexity small, start with good test coverage and no/few extra dependencies for people who only use local tracking. (Nevertheless I fully agree with using SQLAlchemy to get rid of hand-written SQL.)

In the end I feel we should agree on the desired features/properties of a sync mechanism which will lead to a certain design which could be implemented as a proof of concept then.

[don-philipe]

Is somebody still working on it? I would really like to see this feature coming up. So maybe I can help you guys with some minor task (as my python skills are not that good)?

[tstriker]

just drop your db in dropbox or similar and symlink from there to ~/.local/share/hamster-applet/ hamster checks for the db file changes and reloads the db when that happens

[don-philipe]

just drop your db in dropbox or similar and symlink from there

Well I tried this already. But then I have sometimes problems with the database being locked. No starting or stopping of tasks then possible then.

[sm8ps]

I do have the same issue as @don-philipe. After some research I could sort of identify the cause and come up with a work-around. The syncing of the db over my OwnCloud instance works well; that is, there does not seem to be any conflict with open db handles or such. However, Switching computers, there does not seem to be a way of adding new tasks. hamster-cli spits out an error message that the database is locked. The cause for this does not seem to be intrinsic to the db but rather due to hamster-service running in the background. As a work-around, I kill that process an re-launch hamster-indicator (in my case) and all is fine and merry. It would seem useful to have an option in the indicator/applet menu for re-launching Hamster, i.e. killing hamster-service and restarting hamster-indicator. Would that make sense?

[sm8ps]

@elbenfreund commented in #316 regarding the general state of the project which seems to apply here as well.

For the time being, I have added a simple script to re-launch hamster-service after sleep mode. It is not perfect but fits my workflow pretty well. For reference: on SystemD-based Ubuntu 16.04 I created '/lib/systemd/system-sleep/hamster' with the following content:

#!/bin/sh
if [ $1 = post ] && [ $2 = suspend ]
then 
        /usr/bin/killall hamster-service
        /usr/bin/killall hamster-indicator
        /usr/bin/hamster-indicator &
        #/bin/date >> /tmp/hamster.log
fi

This script does not, however, solve issue #316 as the last active task still is prolonged across the sleep period.

I found some helpful comments in http://askubuntu.com/questions/661715/make-a-script-start-after-suspend-in-ubuntu-15-04-systemd. NB: other systems may use '/etc/pm/sleep.d' or other directories and don't forget to make the script executable! :)

[GeraldJansen]

Concerning one of the opening posts' options, namely "Merge data ... even manually getting the sqlite database and importing locally", I have written a tool which can pipe "export tsv" output from one hamster database to a "load tsv" into another database: https://github.com/GeraldJansen/hamster-tool.