chalk icon indicating copy to clipboard operation
chalk copied to clipboard

integrate libcon4m

Open ee7 opened this issue 1 year ago • 0 comments

con4m was rewritten, most importantly to:

  • Decrease the chalk startup time
  • Supported multi-threaded attribute access

Current flow

Currently, building chalk execs the con4m executable to generate c4autoconf.nim for Nim-native data structures and getter procs:

https://github.com/crashappsec/chalk/blob/4157b586e484f12c6b37ee2c68afd5bd5ac330dd/chalk.nimble#L54-L60

Then running chalk:

  1. Runs con4m code to load and validate the chalk config (which doesn't change)
  2. Uses that environment to load and validate the built-in configs
  3. Uses that environment to load and validate any user-supplied config

This also involves flushing the con4m attribute state to the Nim-native data structures a few times, so it stays synced.

Future flow

libcon4m allows chalk to save and restore runtime state by embedding an object file into the chalk binary, such that chalk can perform only the necessary work that would change per-invocation (like user-supplied config). This should reduce complexity overall in configuration loading - there's no more stacking.

That is, with libcon4m, building chalk should:

  1. Run the chalk configuration, generating an object file
  2. Embed that object file into chalk

Then running chalk should:

  1. Restore the state from the object file
  2. Partially process the command-line to find external configs
  3. Process those external configs
  4. Perform the requested operations

Eventually we may want to allow pre-compiling user-configs for later runs, but that introduces significant complexity for much less important gains.

There are also breaking changes in the language.

Refs: https://github.com/crashappsec/chalk/issues/26 Refs: https://github.com/crashappsec/chalk/issues/214

ee7 avatar Apr 11 '24 10:04 ee7