csvtk icon indicating copy to clipboard operation
csvtk copied to clipboard

Change expression evaluation package

Open shenwei356 opened this issue 4 years ago • 6 comments

Since https://github.com/Knetic/govaluate is not active anymore, I plan to replace it with https://github.com/antonmedv/expr.

Related commands: filter2, mutate2.

It looks great! https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md

shenwei356 avatar Nov 03 '21 07:11 shenwei356

https://github.com/benhoyt/goawk seems a good choice too.

shenwei356 avatar Feb 04 '22 16:02 shenwei356

and https://github.com/google/cel-go

shenwei356 avatar Mar 19 '22 02:03 shenwei356

https://github.com/PaesslerAG/gval looks great!

shenwei356 avatar Mar 30 '23 11:03 shenwei356

I hacked up a quick and dirty mutate3 implementation using https://github.com/expr-lang/expr to see what it could do.

Expr does date math out of the box (which could solve #213):

$ cat ../testdata/datesub.csv 
ID,In,Out
1,2023-08-25 11:24:00,2023-08-27 08:33:02
2,2023-08-25 11:28:00,2023-08-26 14:17:35
3,2023-08-26 11:29:00,2023-08-29 20:43:00

$ cat ../testdata/datesub.csv | ./csvtk mutate3 -n Duration -e 'date($3) - date($2)'
ID,In,Out,Duration
1,2023-08-25 11:24:00,2023-08-27 08:33:02,45h9m2s
2,2023-08-25 11:28:00,2023-08-26 14:17:35,26h49m35s
3,2023-08-26 11:29:00,2023-08-29 20:43:00,81h14m0s

$ cat ../testdata/datesub.csv | ./csvtk mutate3 -n Duration -e 'date($Out) - date($In)'
ID,In,Out,Duration
1,2023-08-25 11:24:00,2023-08-27 08:33:02,45h9m2s
2,2023-08-25 11:28:00,2023-08-26 14:17:35,26h49m35s
3,2023-08-26 11:29:00,2023-08-29 20:43:00,81h14m0s

$ cat ../testdata/datesub.csv | ./csvtk mutate3 -n Duration -e '(date($Out) - date($In)).Hours() | int()'
ID,In,Out,Duration
1,2023-08-25 11:24:00,2023-08-27 08:33:02,45
2,2023-08-25 11:28:00,2023-08-26 14:17:35,26
3,2023-08-26 11:29:00,2023-08-29 20:43:00,81

I basically copied mutate2.go, threw out the custom functions, and replaced govaluate with expr.

moorereason avatar Mar 18 '24 04:03 moorereason

@moorereason That's great!

shenwei356 avatar Mar 18 '24 07:03 shenwei356