Gaffer
Gaffer copied to clipboard
JobTracker tracks all operations, not just jobs
The documentation of Job, and the general title/JavaDoc of JobTracker indicates that, when enabled on a store, the JobTracker should track only Jobs.
However, inspection of Store.java (see https://github.com/gchq/Gaffer/blob/1fb6823082ef9dd88f65fab33ac1541e8a2a76cc/core/store/src/main/java/uk/gov/gchq/gaffer/store/Store.java#L389 and https://github.com/gchq/Gaffer/blob/1fb6823082ef9dd88f65fab33ac1541e8a2a76cc/core/store/src/main/java/uk/gov/gchq/gaffer/store/Store.java#L924) indicates that, when the JobTracker is enabled, any and every operation executed on the store is tracked as if it was a Job.
I suggest this is a defect, and that regular non-job operations should not be tracked by the JobTracker.
If this is, in fact, intended behaviour, then I suggest the documentation and Javadoc should be updated, and maybe the name of JobTracker changed to OperationTracker, as I'd argue the existing behaviour is surprising given the name of the class and documentation.
Impacts of this defect include:
- Performance degradation when running multiple fast operations caused by multiple writes to the cache for every operation
- Performance degradation when instantiating a Store (see the initialise method in Store.java), as initialising a store with the job tracker enabled requires iterating through everything tracked by the JobTracker (even jobs for other stores)
- In an environment where the JobTracker cache has a maximum size; early expiry of 'real' jobs as they are forced out of the cache early
- Data being persisted to the cache that the user might not expect
As always, happy to submit a PR (I think the solution to this issue is quite straightforward) if the maintainers agree that this is indeed a defect.
Thanks,
P.S. This issue has partially blocked the testing of gh-2457
We believe this is a bug.
On discussion with @GCHQDev404 it appears that this is no longer blocking completion of #2457 so it can stay on the post-2.0 backlog.