graylog2-server
graylog2-server copied to clipboard
Graylog2
Lookup table: DSV File from HTTP without Quote character
When using the feature of "DSV File from HTTP" the option for "Quote character" must be set. There is no option to use files without quote characters. I want to use lists like http://rules.emergingthreats.net/blockrules/compromised-ips.txt or https://zeustracker.abuse.ch/blocklist.php?download=ipblocklist to enrich data. But in these lists there a only one ip address per line.
Example: 103.80.36.19 103.99.1.193 103.99.2.150 103.99.2.172 103.99.3.222 ... .. .
It would be a good option, when there is an option to leave quote character blank.
To extend this - the quote character needs always to be set and if the value is not in the quote characters they are not present.
The IP Lists that are provided by users and orgs usually do only contain one IP per line without any other characters.
We should make those lists usable.
Did this ever get implemented? We'd like to use a lookup table/data adapter to query Palo Alto's External Dynamic Lists so we can (for example) check for a CIDR match in a pipeline rule.
As far as I can see its neither implemented in Graylog 4.3.9 or Graylog 5.0.0
@EvaZg Can you clarify requirements for this? I don't think it is about the quote character at all: You can already use tables without quotes (specify a dummy character as quote, that does not occur in any of the values).
CSV from file and DSV from HTTP are designed for key/value lookups. This feature request however specifies only a single column. I assume they only ever want to call lookup_has_value(). It would make more sense to define a new data adapter type for presence tests rather than trying to twist CSV/DSV to accomodate this - for clarity of the UI as well as the code.
@Graylog2/product Unclear how to proceed with this issue. Can you weigh in on my idea of adding a new data adapter type for presence tests?
Patrick - I agree with your assessment that a single value list is not equivalent to a CSV, DSV file. I think your recommended solution is appropriate.
