Confusion about updating MISP after generating new warninglist list.json files

[hkelley]

Maybe we've been doing it all wrong, but we have been running the tools/generate-crl-ip-domains.py script periodically in order to keep our warninglist updated.

Today, I went to update MISP via the web UI and hit this error while it was updating the submodule for misp-warninglists.

error: Your local changes to the following files would be overwritten by checkout:
	lists/crl-hostname/list.json
	lists/crl-ip/list.json
Please commit your changes or stash them before you switch branches.
Aborting
Unable to checkout '99c64074101b3b3a18727564350070895a4e0216' in submodule path 'app/files/warninglists'

The web UI seems to read from these git-managed files when it updates the warninglists, so I assume we are meant to update at this location:

 /var/www/MISP/app/files/warninglists/lists/crl-ip/list.json

But if we update there, how are we meant to clean up so that we can do MISP updates?

[adulau]

It's indeed a good point.

Following a discussion with @iglocska MISP core should have a dedicated custom directory for the misp-warning-lists next to the upstream github repository in the future. For the time being, many users are maintaining a custom branch and merge it at update via a script.

[hkelley]

Thank you. If the custom directory feature (a server setting?) is coming soon then I can keep doing my current stash/delete via CLI before using the MISP UI for the update. We can then regenerate any warninglist after the MISP update pulls the list.json files locally.

If that setting isn't coming soon, then could you please elaborate a bit on this branch+merge strategy? Are you saying that users are doing something like this?

1. forking this repo
2. switching the upstream target of their local warninglists folder to that fork+branch