cache: slowdown on export

[markus2330]

Steps to Reproduce the Problem

I have a large multiresolver configuration, from which exports into 24 CSV files are done. (Using kdb export)

Expected Result

That with cache, the previous generation time of 358s will be improved.

Actual Result

With cache, the exports take longer: 559s

System Information

• Elektra Version: master

[mpranj]

I can not confirm this.

Here's what I did:

# Create some large CSVs in current dir
for j in `seq 1 100`; do echo "user/test/$j/name;user/test/$j/other;user/test/$j/something" > $j.csv; for i in `seq 1 1000`; echo "$i:$i;$i" >> $j.csv; done

# Mount the CSVs using multifile and csvstorage
kdb mount -R multifile -c storage="csvstorage",pattern="*.csv",resolver="resolver" `pwd` system/test/

# Clear caches and run some exports
kdb cache clear

time kdb ls system/test > /dev/null
# kdb ls system/test > /dev/null  0.69s user 0.15s system 97% cpu 0.870 total
time kdb ls system/test > /dev/null
# kdb ls system/test > /dev/null  0.24s user 0.12s system 99% cpu 0.360 total
time kdb ls system/test > /dev/null
# kdb ls system/test > /dev/null  0.24s user 0.12s system 99% cpu 0.365 total

I do get a speedup by using the cache. Is it possible you used "/" or "system/" when exporting? Maybe something in there can't be cached (#2454).

[markus2330]

1. I compared with an Elektra where the cache plugin was not available at all not the first with the second execution (e.g. by not having it mounted or remove the plugin).
2. (Maybe not relevant but) I copied it the other way round: multiresolver+INI is mounted and the export uses csvstorage.

[mpranj]

Then I think you used a version of Elektra where INI caching was disabled. Caching for ini was enabled in #2604 much later than this issue was opened.

[markus2330]

So you think that the way the disabling was implemented caused the slowdown?

[mpranj]

I think so, but I'll verify it. The disabling always caused a cache miss and a rebuild of the cache.

[markus2330]

This is very plausible.

[stale[bot]]

I mark this issue stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping the issue by writing a message here or create a new issue with the remainder of this issue. Thank you for your contributions :sparkling_heart:

[stale[bot]]

I mark this stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping by writing a message here or create a new issue/PR with the remainder of this issue/PR. Thank you for your contributions :sparkling_heart:

[github-actions[bot]]

I mark this stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping by writing a message here or create a new issue with the remainder of this issue. Thank you for your contributions :sparkling_heart:

[github-actions[bot]]

I closed this now because it has been inactive for more than one year. If I closed it by mistake, please do not hesitate to reopen it or create a new issue with the remainder of this issue. Thank you for your contributions :sparkling_heart: