vulnerability-lookup
vulnerability-lookup copied to clipboard
cve-search
User management system
Definition of the scope
We need support of:
- [ ] users (login, password, email, api key)
- [ ] admin dashboard: creation, edition, validation of accounts, etc.
- [ ] account creation and validation
- [ ] password reset
- [ ] ownership: editor, contributor to a vulnerability. Editor of a comment. etc.
- [ ] organizations
- [ ] admin dashboard: creation, edition
- [ ] org admin permissions ?
A user can be member of several organizations ? Why not...
Regarding storage, I don't mind a SQL database (as long as I'm not the one developing it :laughing: ), but it really needs to be an optional feature. We must keep the option to install the project as a single-user read only interface that doesn't requires to create and manage users and permissions.
Regarding creating vulnerability, we still haven't decided a format, required fields, if we go for vulnogram and so on. This is something we at some point need to decide on, but there is no obvious winner yet (afaict).
OK, i understand the fact that it must stays a single-user project.
Concerning the way to deal with vulnerability edition I've checked vulnogram. It seems nice. I must take a deeper look. I will.
We could use JSON schemas to define the (required) fields. I know this nice JSON editor. It generates a HTML form in order to create and edit objects. The form is generated from a JSON schema. And the objects created thanks to the HTML forms will be validated by the JSON schema, ofc...
Some times ago I tried to edit the same CVE but with 3 different JSON schemas:
- GitHub schema: https://objects.monarc.lu/object/view/5253
- NIST schema: https://objects.monarc.lu/object/view/5212
- CVESearch: https://objects.monarc.lu/object/view/5211
The editor looks like this for example:
it's all generated automatically.
Just an idea... vulnogram is probably more what we need !
that's nice!
but yes, vulnogram is the way to go if the format we use is the one used by CVE. This format is a lot and not necessarily super userfriendly. It really depends the level of technical capability we expect from the users of the platform.
I've started to implement a standard system of sessions. Users are able to Log In as normal users or as admin.
As discussed this is optional and not used by default. This is why I decided to use Flask Blueprints.
The existing views that we have currently in the file /web/__init__.py are in a default Blueprint. There is absolutely no change for the user. Endpoints are the same.
The existing views that we have currently in the file /web/__init__.py that are related to the API are in another Blueprint. Again no change for the user.
So the new code related to the user sessions is in a dedicated Blueprint. We could even deactivate it via the configuration file. Not implemented properly but I've tested this deactivation.
For the moment there is only a form to login and to edit the profile of the currently connected user (password, email, etc.). Users (normal and admin) must be created via Flask command. But I will add a form to Sign In with a verification via email (with a time based token in a URL provided inside the email, standard...).
By default this uses a SQLite database, so we do not impact the existing database. Of course we can change to PostgreSQL for example. With @adulau we think that SQLite should be enough and scale. Anyway, we must be able to change easily the database. Of course as everything is optional the app do not need this database to operate properly. And the app still works as it used to (I mean to start the back-end WITH the web server, importers, etc.)
For the first release of this feature I plan to have:
- self-account creation with a form available to not connected users ofc (with a verification via email)
- account creation by the admin (via the web interface and Flask command (done))
- basic profile page for currently connected user (edit his/her information). Almost done
- admin views: check status and edition of users, validate accounts, etc.
- and ofc the feature to comment a vulnerability: else why implementing all this... 😆 We can say that it will be the first feature dedicated to a normal user.
I must refine the permissions that will be available: edit, create new vulnerabilities (that will be stored in kvrocks), etc. ? This depends on the future features for the authenticated users. For the moment it's quite basic since I must implement the first bricks related to users sessions. As said for the first feature dedicated to the users, let's focus on commenting vulnerabilities. No need to impact the kvrocks database.
Must I implement Organizations ? It's not big deal. But better to do it now if we want this. I have an idea how to do it. I can explain it in details now or a bit later.
Since we will have email notifications for the accounts creation/validation. I will add a little "notifications" Python module. In this module we will be able in the future to implement things related to notifications via email... Idk... Maybe notifications of a comment to review, etc. FYI...
So there is still "some" work and I am in vacations. But I can progress relatively fast. It's just a lot and it must be tested with care, especially the fact that the current functions of the project are not impacted.
Do you want that I open a draft PR ? Maybe it's a good idea. No need to review the PR since a lot of work is still needed. But at least it's tracked and you can see easily.
Ofc, I am documenting everything ;-)
I'll try to document everything here: https://vulnerability-lookup.readthedocs.io/en/latest/architecture.html#blueprints WIP WIP...
I'm happy with you implementing a PR as soon as possible, and if possible merging it incrementally so we don't have a monster PR to review in a month or two.
And regarding the scope, the only one I have is a user has to be able to create a vulnerability (but don't have a format nor the specs yet). Comments is cool, orgs, I really don't know if it's useful. I'd like to avoid feature creep if we don't have usecases yet: in which context will we have vuln lookup installed in a multi org environment where many users in one organization will add commends in a vulnerability lookup instances managed by a third party? At some point, we will need to implement synchronization too and then we re-implement MISP :cry:
About orgs, to be honest it's something I've kept in my mind because we briefly talked about it while eating... I am totally fine to not implement Organization. And you are right when you say that we will start re-implement MISP, because then we will need to speak about Org Admin, etc. Then it becomes way too much for what we need. I prefer to only have normal users (able to create vulns, comment, etc) and admins.
I'll clean the code as much as possible before opening a draft PR. Actually for the moment it's not big changes. Just new blueprints with existing code and few new code for the sessions management.
Sounds good. I'm not going to teach you anything there, but my policy in general is to implement the absolute strict minimum that solves our current practical usecase, while keeping in mind that more things will come up and we will need to be flexible. And not try to implement everything now because it may be useful in a year or two.
OK, cool ! I do my best to reach the feature of creating a vuln with few code ;-)
In the blueprint dedicated to the existing API, I will certainly (surely) need to add a POST method related to the existing Vulnerability Resource. It will be authenticated. So that authenticated users will be able to use the API in order to create a vulnerability.
Later we will need a nice web interface, vulnogram, that will use this endpoint. It should... Probably first with a simple form. Maybe it's best if the POST endpoint checks the format of the submitted data (with jsonschema Python module for example - when will have a schema). We can indeed use hooks and format_checker with flask-restx.
And when this works with the simple form. Maybe we can start to look closer at vulnogram in order to use it.
We're almost done for the admin views. Account creation (self account or by an admin), confirmation via email, password recovery via email are working. OTP as well.
This is an admin view:
And here the user interface available to a user (commenter or reporter) :
As you can see the user can generate an API key for the API. Since I am adding a POST endpoint in order to create new vulnerabilities. Consequently we have now one authenticated endpoint for the API. The only one. The authentication method is quite standard, I uses Flask-restx capabilities. To summarize, an already authenticated user can make POST requests to the API or if not authenticated can use the HTTP header X-API-KEY :
- https://github.com/cedricbonhomme/vulnerability-lookup/blob/user-amanagement/website/web/api/v1/common.py#L20
- https://github.com/cedricbonhomme/vulnerability-lookup/blob/user-amanagement/website/web/api/v1/init.py#L15
the curl request should be something like this:
curl -X 'POST' \
'http://127.0.0.1:5000/api/cve/' \
-H 'accept: application/json' \
-H 'X-API-KEY: BrehnVEvEHFpKWSB-x55J1o-pfqw8ciHpAQdunnNtGFpCP1QijgPsOmMaZFhKYYnsl0rXyzSL-rLGsNC5fDQIw' \
-d ''{$JSON_DATA}"