pisa icon indicating copy to clipboard operation
pisa copied to clipboard

Safer and more expressive expression evaluation

Open jllanfranchi opened this issue 8 years ago • 2 comments

E.g. in DataProcParams (fields specified in data_proc_params.json and interpreted by evaluateExpr) and Events (applyCut method) classes could benefit from more generic and safe evaluation of the contents.

Note that it will probably be necessary to more fully encapsulate what is an HDF5 path vs. what is not to allow for more complicated mathematical expressions (and to guarantee that ambiguous things like / or Python keywords are interpreted correctly as not being part of an HDF5 path).

Numexpr: https://github.com/pydata/numexpr

  • fast for simple numpy operations over multiple arrays
  • limited in what it can do
  • possibly slower (though probably not by much) than direct evaluation of numpy for really simple things
  • not sure about how "safe" it is from security standpoint, but its limited syntax may suffice for this concern

ASTEVAL: http://newville.github.io/asteval/index.html

  • more expressive (can do looping, almost everything you can specify in Python itself)
  • in order to specify arbitrary things, a fully valid Python syntax must be passed (so including line breaks and indentation for subsequent lines), which is awkward to specify in a JSON or CFG file.
  • supports numpy if numpy is installed (though whether e.g. sqrt is from math or from numpy isn't clear to me yet)

ast.literal_eval: https://docs.python.org/2.7/library/ast.html#ast.literal_eval

  • Simplest solution, no external package necessary
  • Just useful for reading in Python objects without doing anything bad, not for "doing" something

jllanfranchi avatar Oct 16 '16 11:10 jllanfranchi

Also in the config_parser, replace simple eval expressions by ast.literal_eval for safer parsing

philippeller avatar Mar 26 '17 16:03 philippeller

I can see this being useful, but then again, it has worked like this for 6+ years now :thinking:

LeanderFischer avatar May 02 '24 12:05 LeanderFischer