scons icon indicating copy to clipboard operation
scons copied to clipboard

WIP: expeimental support for diskcache-style (sqlite backed) sconsign

Open mwichmann opened this issue 1 year ago • 0 comments

This is a low-touch conversion to use an sqlite backend instead of a flat "dblite" file for the sconsign database, without really changing SCons logic - but includes some hacks where rather than full detection, things are just commented out.

New SCons/sdiskcache.py module supports diskcache usage.

Definitely not ready to merge.

diskcache: https://pypi.org/project/diskcache/ and https://grantjenks.com/docs/diskcache/

Experimental: with a --convert flag, sconsign utility will read an existing sconsign file and set up a sqlite-backed Diskcache in .sconsign.sqlite.

========================= TODO on the testing front:

  • test/Repository: fiddling the "writable" attribute in the framework doesn't help, sqlite db opened unwritable is always unwritable, Get: OperationalError : attempt to write a readonly database. Affects:
    test/Repository/Install-Local.py
    test/Repository/Local.py
    test/Repository/SharedLibrary.py
    test/Repository/StaticLibrary.py
    test/Repository/VariantDir.py
    test/Repository/link-object.py
    test/Repository/option-n.py
    test/Repository/variants.py
  • test/Configure/VariantDir-SConscript.py: hardcodes .dblite as the DB suffix.
  • test/Configure/implicit-cache.py: hardcodes .dblite as the DB suffix.
  • test/SConsignFile/default.py: hardcodes .dblite as the DB suffix
  • test/SConsignFile/explicit-file.py: hardcodes .dblite as the DB suffix
  • test/option/hash-format.py: why does this do an assert, anyway? e.g. Missing files: /tmp/scons/testcmd.305909.0g2s8ny7/.sconsign.dblite
  • test/sconsign/corrupt.py: hardcodes .dblite. Since it's not a flat file, corrupt is different. Expose Cache.check()?
  • test/sconsign/script/Configure.py: currently hitting unpickled data
  • test/sconsign/script/dblite.py: possibly should just not run, but maybe add parallel one for sdiskcache
  • test/option/option-n.py: see FW
  • testing/framework/TestSCons.py: get_sconsignname, unlink_sconsignfile don't work for sqlite db
  • Some tests should use unlink_sconsignfile(possibly renamed) that don't. From a grep:
  test/Configure/VariantDir-SConscript.py
  test/MSVS/vs-10.0-exec.py
  test/MSVS/vs-10.0Exp-exec.py
  test/MSVS/vs-11.0-exec.py
  test/MSVS/vs-11.0Exp-exec.py
  test/MSVS/vs-14.0-exec.py
  test/MSVS/vs-14.0Exp-exec.py
  test/MSVS/vs-14.1-exec.py
  test/MSVS/vs-14.2-exec.py
  test/MSVS/vs-14.3-exec.py
  test/MSVS/vs-6.0-exec.py
  test/MSVS/vs-7.0-exec.py
  test/MSVS/vs-7.1-exec.py
  test/MSVS/vs-8.0-exec.py
  test/MSVS/vs-8.0Exp-exec.py
  test/MSVS/vs-9.0-exec.py
  test/MSVS/vs-9.0Exp-exec.py
  test/option/option-n.py
  test/option/hash-format.py
  • test/SConsignFile` - Check - may need work, or may be unique to file-based. may need equivalent one for DB versions?

Contributor Checklist:

  • [ ] I have created a new test or updated the unit tests to cover the new/changed functionality.
  • [ ] I have updated CHANGES.txt and RELEASE.txt (and read the README.rst).
  • [ ] I have updated the appropriate documentation

mwichmann avatar Oct 18 '24 20:10 mwichmann