modsecurity-parser
modsecurity-parser copied to clipboard
Published
20 hours ago •
molu8bits
molu8bits
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed
Hi Guys, run this script, met issue(as title), can help? thanks
my modsecurity(3.0.9) logs format:
{"transaction":{"client_ip":"10.200.101.16","time_stamp":"Thu May 11 02:13:58 2023","server_id":"6c63a629cf8ef75665cbe6abb55daaf9d4fa7b2b","client_port":33042,"host_ip":"172.22.0.2","host_port":80,"unique_id":"168377123884.428748","request":{"method":"GET","http_version":1.1,"uri":"/pub/","headers":{"Connection":"Keep-Alive","Host":"10.200.101.18","User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36"}},"response":{"body":"<!--\n\n Copyright © 2016-2023 The Thingsboard Authors\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n\n-->\n<!doctype html>\n<html lang=\"en\" style=\"width: 100%; height: 100%;\">\n<head>\n <meta charset=\"utf-8\">\n <title>ThingsBoard</title>\n <base href=\"/\">\n\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">\n <link rel=\"icon\" type=\"image/x-icon\" href=\"thingsboard.ico\">\n <link rel=\"preload\" href=\"assets/fonts/MaterialIcons-Regular.ttf\" as=\"font\" type=\"font/ttf\" crossorigin=\"anonymous\"/>\n <link rel=\"stylesheet\" href=\"assets/fonts/material-icons.css\"/>\n <style type=\"text/css\">\n\n body, html {\n height: 100%;\n overflow: hidden;\n background-color: #eee;\n }\n\n .tb-loading-spinner {\n margin: auto;\n z-index: 1;\n position: absolute;\n top: 0;\n bottom: 0;\n left: 0;\n right: 0;\n width: 136px;\n height: 30px;\n text-align: center;\n }\n\n .tb-loading-spinner > div {\n width: 30px;\n height: 30px;\n margin-right: 10px;\n background-color: rgb(43,160,199);\n\n border-radius: 100%;\n display: inline-block;\n -webkit-animation: tb-bouncedelay 1.4s infinite ease-in-out both;\n -moz-animation: tb-bouncedelay 1.4s infinite ease-in-out both;\n animation: tb-bouncedelay 1.4s infinite ease-in-out both;\n }\n\n .tb-loading-spinner .tb-bounce1 {\n -webkit-animation-delay: -0.32s;\n -moz-animation-delay: -0.32s;\n animation-delay: -0.32s;\n }\n\n .tb-loading-spinner .tb-bounce2 {\n -webkit-animation-delay: -0.16s;\n -moz-animation-delay: -0.16s;\n animation-delay: -0.16s;\n }\n\n @-webkit-keyframes tb-bouncedelay {\n 0%, 80%, 100% { -webkit-transform: scale(0) }\n 40% { -webkit-transform: scale(1.0) }\n }\n\n @-moz-keyframes tb-bouncedelay {\n 0%, 80%, 100% { -moz-transform: scale(0) }\n 40% { -moz-transform: scale(1.0) }\n }\n\n @keyframes tb-bouncedelay {\n 0%, 80%, 100% {\n -webkit-transform: scale(0);\n -moz-transform: scale(0);\n transform: scale(0);\n } 40% {\n -webkit-transform: scale(1.0);\n -moz-transform: scale(1.0);\n transform: scale(1.0);\n }\n }\n\n </style>\n<link rel=\"stylesheet\" href=\"styles.10895964a4a3aa21d65a.css\"></head>\n<body class=\"tb-default\">\n <tb-root></tb-root>\n <div id=\"tb-loading-spinner\" class=\"tb-loading-spinner\">\n <div class=\"tb-bounce1\"></div>\n <div class=\"tb-bounce2\"></div>\n <div class=\"tb-bounce3\"></div>\n </div>\n<script src=\"runtime.286f6982886cb90bbe7a.js\" defer></script><script src=\"polyfills.e2023dc347cde42f7c8d.js\" defer></script><script src=\"scripts.d93c5ee41f6da54bd100.js\" defer></script><script src=\"vendor.3f3611f892c51888617d.js\" defer></script><script src=\"main.ed39576ce9947da26638.js\" defer></script></body>\n</html>\n","http_code":200,"headers":{"Accept-Ranges":"bytes","Vary":"Origin","Vary":"Access-Control-Request-Method","Vary":"Access-Control-Request-Headers","Connection":"keep-alive","Last-Modified":"Tue, 07 Feb 2023 14:18:35 GMT","Last-Modified":"Tue, 07 Feb 2023 14:18:35 GMT","Cache-Control":"no-cache, no-store, max-age=0, must-revalidate","Content-Type":"text/html;charset=UTF-8","Content-Length":"3345","Date":"Thu, 11 May 2023 02:13:58 GMT","Server":"nginx/1.22.1","X-Content-Type-Options":"nosniff","X-Content-Type-Options":"nosniff","X-XSS-Protection":"1; mode=block","Pragma":"no-cache","Content-Language":"en","Expires":"0","X-Frame-Options":"SAMEORIGIN"}},"producer":{"modsecurity":"ModSecurity v3.0.9 (Linux)","connector":"ModSecurity-nginx v1.0.3","secrules_engine":"Enabled","components":["OWASP_CRS/4.0.0-rc1\""]},"messages":[{"message":"Host header is a numeric IP address","details":{"match":"Matched \"Operator `Rx' with parameter `(?:^([\\d.]+|\\[[\\da-f:]+\\]|[\\da-f:]+)(:[\\d]+)?$)' against variable `REQUEST_HEADERS:Host' (Value: `10.200.101.18' )","reference":"o0,13o0,13v48,13","ruleId":"920350","file":"/etc/modsecurity.d/owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf","lineNumber":"744","data":"10.200.101.18","severity":"4","ver":"OWASP_CRS/4.0.0-rc1","rev":"","tags":["modsecurity","application-multi","language-multi","platform-multi","attack-protocol","paranoia-level/1","OWASP_CRS","capec/1000/210/272","PCI/6.5.10"],"maturity":"0","accuracy":"0"}}]}}
I think the problem is with newest matplotlib. Can you check if you are using recommended versions of python packages?
You can always try to use docker version
yes, using modsecurity-parser docker for analysis logs. found fix method, using native format, not json with modsecurity logs, modsecurity-parser will be working fine.
Can you provide me command line options you used? For the first time when error was generated and when it works? I will add it to test cases.
as https://github.com/coreruleset/modsecurity-docker describe, nginx's MODSEC_AUDIT_LOG_FORMAT default value is JSON. I changed to Native, modsecurity-parser can working.
Well, json output for modsecurity3 is much different than modsecurity3. Current parser doesn't work for version3 + json output. I am going to implement this case soon.