baker
baker copied to clipboard
Add lua filter
:question: What
Add a LUA filter that allows to add filter logic (implement the Process function) in lua. Needs further testing/benchmarking/polishing but first results shows a relatively good performance with respect to the same filter coded in Go (5% slower for a filter with a single lua operation).
Every Lua
baker filter compiles a given script and will run a user-specific function (which should have the following prototype):
-- rec is a record object
-- next is function next(record)
function myLuaFilterFunction(rec, next)
lua script knows the record
type which is the lua-equivalent of the Record
Go interface. record
has the following methods:
-
get(int) string
-
set(int, string)
-
clear()
-
copy() record
In the lua context (called lua state), some utility functions are already defined and can be used:
-
validateRecord(record) bool, int
-
createRecord record
-
fieldByName(string) int|nil
different from Go which returns(baker.FieldIndex, bool)
-
fieldNames
which is a lua table, same asfieldNames
Go slice
:hammer: How to test
- List all steps necessary;
- To test this pull request.
:white_check_mark: Checklists
This section contains a list of checklists for common uses, please delete the checklists that are useless for your current use case (or add another checklist if your use case isn't covered yet).
- [x] Is there unit/integration test coverage for all new and/or changed functionality added in this PR?
- [x] Have the changes in this PR been functionally tested?
- [x] Have new components been added to the related
all.go
files? - [x] Have new components been added to the documentation website?
- [x] Has
make gofmt-write
been run on the code? - [x] Has
make govet
been run on the code? Has the code been fixed accordingly to the output? - [x] Have the changes been added to the CHANGELOG.md file?
- [x] Have the steps in CONTRIBUTING.md been followed to update a Go module?
Codecov Report
Merging #110 (e5da03b) into main (3ff24dd) will increase coverage by
0.79%
. The diff coverage is82.29%
.
@@ Coverage Diff @@
## main #110 +/- ##
==========================================
+ Coverage 49.43% 50.22% +0.79%
==========================================
Files 56 57 +1
Lines 3880 3976 +96
==========================================
+ Hits 1918 1997 +79
- Misses 1752 1764 +12
- Partials 210 215 +5
Impacted Files | Coverage Δ | |
---|---|---|
filter/lua.go | 82.29% <82.29%> (ø) |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 3ff24dd...e5da03b. Read the comment docs.
Codecov Report
Merging #110 (68a1060) into main (40a57bc) will decrease coverage by
3.75%
. The diff coverage is81.25%
.
:exclamation: Current head 68a1060 differs from pull request most recent head 4fe5623. Consider uploading reports for the commit 4fe5623 to get more accurate results
@@ Coverage Diff @@
## main #110 +/- ##
==========================================
- Coverage 53.95% 50.20% -3.76%
==========================================
Files 60 57 -3
Lines 4003 3976 -27
==========================================
- Hits 2160 1996 -164
- Misses 1646 1764 +118
- Partials 197 216 +19
Impacted Files | Coverage Δ | |
---|---|---|
filter/lua.go | 81.25% <81.25%> (ø) |
|
filter/set_string_from_url.go | 37.50% <0.00%> (-19.03%) |
:arrow_down: |
topology.go | 60.89% <0.00%> (-13.41%) |
:arrow_down: |
config.go | 65.00% <0.00%> (-13.00%) |
:arrow_down: |
filter/metadata_lastmodified.go | 76.19% <0.00%> (-12.70%) |
:arrow_down: |
input/list.go | 60.78% <0.00%> (-10.51%) |
:arrow_down: |
filter/notnull.go | 70.58% <0.00%> (-9.42%) |
:arrow_down: |
filter/clear_fields.go | 56.25% <0.00%> (-8.04%) |
:arrow_down: |
filter/partial_clone.go | 65.00% <0.00%> (-7.23%) |
:arrow_down: |
filter/string_match.go | 69.56% <0.00%> (-6.63%) |
:arrow_down: |
... and 26 more |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update a209176...4fe5623. Read the comment docs.