prometheus-nginxlog-exporter icon indicating copy to clipboard operation
prometheus-nginxlog-exporter copied to clipboard

Multiple formats support

Open pawellrus opened this issue 3 years ago • 3 comments

Is your feature request related to a problem? Please describe. Some log strings doesn't contain $upstream_response_time value. So exporter throws many errors like "error while parsing $upstream_response_time: value '' could not be parsed into float"

Describe the solution you'd like I would be nice if exporter could parse string against several formats (without $upstream_response_time for example) and select more apropriate format for each string.

pawellrus avatar Nov 30 '21 10:11 pawellrus

Thanks for the suggestion! :+1: The exporter was built under the general assumption that it would be fed uniform log lines. Supporting multiple log formats in a single file is not a priority for me ATM, but I'd be willing to accept pull requests for this feature, provided that they're backwards-compatible and have test coverage.

martin-helmich avatar Dec 13 '21 19:12 martin-helmich

@martin-helmich, hello. Another one question regarding this issue. If we can have only one format template, may be we can define possible null values inside. I mean something like that:

format = "$remote_addr - $remote_user [$time_local] \"$request\" ($status|) ($body_bytes_sent|) \"($http_referer|)\" \"($http_user_agent|)\" \"($http_x_forwarded_for|)\""

Or it should not work?

UPD: already tested this - not possible. Seems that only way to get rid of exceptions like "error while parsing $upstream_response_time: value '' could not be parsed into float" is to implement multiple formats. :(

pawellrus avatar Jan 12 '22 10:01 pawellrus

You may want to try to define your nginx pattern with the nullable field at the end. It may work since the exporter won't have any else to parse.

Nereis avatar Oct 13 '22 07:10 Nereis