fluent-plugin-record-modifier icon indicating copy to clipboard operation
fluent-plugin-record-modifier copied to clipboard

how to use .match(/pattern/) when using record_modifier

Open PikaByter opened this issue 3 years ago • 1 comments
trafficstars

I want to add an api field to record_modifier, but it doesn't seem to work when I search with Kibana, it reports:

This field is present in your Elasticsearch mapping but not in the 500 documents shown in the doc table. You may still be able to visualize or search on it.

config:

    <filter {{ .Values.sourceTag }}>
      @type record_modifier
      <record>
        api ${record['log'].match(/\/api\/.*? /)}
        level
      </record>
    </filter>

record['log'] examples(one per line):

2022/09/26 16:34:30 DBG abc/xyz.go:46 > Register Route - DELETE, /api/role/v2
2022/09/26 16:34:30 DBG cmdbase/parser.go:46 > Register Route - GET, /api/role/v2/sys_feat
2022/09/26 16:34:30 Serving gRPC-Gateway on http://0.0.0.0:8070
2022/09/26 16:34:30 INF gwserver/gateway_server.go:152 > Start rest server in 0.0.0.0:7847
2022/09/26 16:35:36 DBG middleware/api_entry.go:97 > <- GET /api/ping to_xyz=infra-abc 4d65822107fcfd52
2022/09/26 16:35:36 DBG middleware/perm.go:93 > perm check, userID: 0, partyID:  4d65822107fcfd52
2022/09/26 16:35:36 /app/paas_go/cofu/models/party/party.go:311 record not found
2022/09/26 16:35:36 WRN monitor/instrument.go:75 > record metrics failed, Err: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: address $[MONITOR_GRPCSERVICEURL]: missing port in address" 4d65822107fcfd52
2022/09/26 16:35:53 DBG middleware/api_entry.go:97 > <- GET /api/ping to_xyz=infra-yao 78629a0f5f3f164f
2022/09/26 16:35:53 DBG middleware/perm.go:93 > perm check, userID: 0, partyID:

When I change the configuration to the following, it can get data what doesn't match what I need exactly

    <filter {{ .Values.sourceTag }}>
      @type record_modifier
      <record>
        api ${record['log'].split(' ')[7])}
        level
      </record>
    </filter>

how to edit the config?

PikaByter avatar Sep 26 '22 09:09 PikaByter

I also try this,but I can not get data either.

    <filter {{ .Values.sourceTag }}>
      @type record_modifier
      <record>
        api ${record['log'].match(/.*/)}
        level
      </record>
    </filter>

PikaByter avatar Sep 26 '22 09:09 PikaByter