vector
vector copied to clipboard
Request for parse_aws_cloudfront_log or parse_w3c_extended_logfile function
The documentation for parse_regex
suggests opening a ticket to request a parse_*
function be added for a log format that isn't already available.
My request is for AWS Cloudfront Logs, the format is W3C Extended Log File format. Which is made up of tab-delimited log lines with 2 "comment" lines prepended with "#" at the top of the file, one with a version # and one with a list of the fields. https://www.w3.org/TR/WD-logfile.html
And is further defined on this doc.: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html
e.g.:
#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type cs-protocol-version fle-status fle-encrypted-fields c-port time-to-first-byte x-edge-detailed-result-type sc-content-type sc-content-len sc-range-start sc-range-end
2023-02-25 06:06:53 LAX50-P3 429 129.146.75.29 GET xxxx.cloudfront.net /menu/guiw 200 - Mozilla/5.0%20(Windows%20NT%2010.0)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Chrome/89.0.4389.114%20Safari/537.36 id=3&nsbrand=1&nsvpx=phpinfo&protocol=nonexistent.1337%22%3E startupapp=st LambdaGeneratedResponse O0woGPklvWn5Hx7ahYw1uQdUBF-CrSBEl8EK-kgi4u-Tacc1df1Skg== foo.bar.com https 330 0.115 - TLSv1.3 TLS_AES_128_GCM_SHA256 LambdaGeneratedResponse HTTP/1.1 - - 35024 0.115 LambdaGeneratedResponse - 0 - -
2023-02-25 06:06:54 LAX50-P3 434 129.146.75.29 GET xxxx.cloudfront.net /.//WEB-INF/web.xml 200 - - -- LambdaGeneratedResponse 0c6ffuHcpguIKuR1kOgR-fE0IRp2-qbuL_AqhtcrgIWSxHbKoN3jCA== foo.bar.com https 100 0.113 - TLSv1.3 TLS_AES_128_GCM_SHA256 LambdaGeneratedResponse HTTP/1.1 - - 35032 0.113 LambdaGeneratedResponse-0