cgru

cgru copied to clipboard

Deleting a job makes afanasy completely forget about it. But sometimes it can be interesting to come back to those jobs, even though it is something exceptional.

So it could be interesting to have a mechanism to move jobs to a so called Archive, whatever it means technically. Archived jobs would be completely ignored, unless the user explicitly ask for them. There would be an action to archive a job (equivalent to delete it) and also a way back from archive to main job pool.

In practice, I think that an Archive could be a directory, organized the same way as /var/tmp/afanasy/jobs, so that it would easily be managed by a JobContainer object. Hence it would actually be easy to have several archives. There would be a way to open an archive in afwatch, but for job pools not to mix it should not be loaded into the same JobContainer object.

The only thing that would change for a job while being moved from or to an archive is its ID. There is no need to keep the same ID, and so this let the different JobContainers manage the IDs by themselves.

Quite a bit of work, but no real new concept to introduce. I believe it would be a good occasion to stress test the current design and eventually fix it. ;)

Hi. This can be a useful feature. Note, that there is already some jobs and tasks archive in SQL (Postgres). And you query get some useful statistics within different queries. And also there is already some GUI for this SQL database. This GUI is web-based, it works with apache+php+php-mod-psql. But you can't get a job back from archive. Also not all job and tasks information is stored there.

I noticed, indeed, but the "back from archive" feature really interests me. ;)

edit: I started implementing it, and have a archive_mask argument slightly spreading in the code. Hope it won't be too much of an issue. The other solution would be to have some kind of MetaContainer, but I was not so happy with it. We'll see how it turns! I'll commit through a PR for you to review it before merging. ;)

Ok) Yes, that feature is very different, it is not archive, it is statistics.

@timurhai Which one do you think is the best:

• Using a single instance of JobContainer holding Jobs loaded from different archives. Requires the JobContainer to be able to hold several jobs with the same ID.
• Using one instance of JobContainer per loaded archive, and so introducing some kind of JobContainerContainer, a.k.a. JobMetaContainer or ArchiveContainer (less clear). Clearer from a design perspective, but requires to track down a lot of things like all the JobContainer locks to see whether they should be generalized to this new container.

Making IDs unique first would solve the problem of option 1. I would vote for that aynways. Its easier to communicate about jobs when IDs don't change. Would this be a big deal?

You need several archive to group (show) jobs just from a specified archive later? Or you need different folders in /var/tmp/afanasy for different archives for some other program?

It would be nice to have several archives, but not neccesary, imo. If its a side-effect of the design, why not.

Can different archives can be hold by 1 JobContainer? And a job will have an extra attribute archive, and GUI can get jobs just of an exact archive? ( it will be more easy, so i asking will it suit you archive needs ? )

@timurhai Your last comments corresponds to my first bullet point ;) @sebastianelsner Making IDs unique is hard, because the archive lives independently from the rest of the program, which will reassign IDs of deleted/archived jobs to new ones.

And yes, there is no express need for several archives, but I think that it will be a side effect.

If you do not need different folders in /var/tmp/afanasy, different archives can be organized by job.archive attribute and can be hold by 1 JobContainer. This way i think it can have an unique IDs for all archives jobs. And ID will change only when a job comes to archive and when you want job back from archive.