aiida-core
aiida-core copied to clipboard
aiidateam
Docs: Topic storage
The documentation would benefit from a topic on "storage". This should be aimed at more advanced users who want more in-depth and detailed information about what is meant with "storage" in AiiDA. Note that it should still be focused on the end-user, and not for aiida-core developers.
In order to slim down the how-to on finding data (#3996), we're moving the more advanced functionality of the QueryBuilder to the topics section. I think it would make sense to make a section on this page on "Advanced Queries", which includes more advanced features of the appender method and information on the queryhelp.
I'd also add a more extensive discussion on certain database related terminology like entities, vertices, query path etc.
@csadorf what do you think?
Another option would be to put it as a sub section of the "Provenance" topic? There are already quite a few top-level sections there, and maybe we should be a bit careful of letting this number grow uncontrolledly
I agree with limiting the number of top-level sections. What I meant was adding the advanced queries as a sub-section in the "Database" topic, sorry if that wasn't clear.
I agree with limiting the number of top-level sections. What I meant was adding the advanced queries as a sub-section in the "Database" topic, sorry if that wasn't clear.
Haha, actually, it is me who is the idiot here. Clearly didn't read the context properly and realize in what issue this was being posted. I think both Database or Provenance could work.
I think the "Database" topic makes sense as this would also be a good place to explain the database terminology used by the QueryBuilder. In the "Provenance" topic, concepts are explained via nodes, links, etc. So it might be confusing to suddenly end with a sub-section that talks about entities, vertices, edges, etc.
I see, but the concepts of the provenance graph very much apply to querying. In order to effectively use the joining of entities in the query builder, one should understand how these are linked through edges in the provenance graph. The way I see it, the QueryBuilder is exactly built to explore the provenance graph. A lot of its functionality and terminology only exists because the data it is querying is represented by a graph. There are plenty of other data structures that you can store in a database that are not necessarily best represented by a graph. The only connotation is that the QueryBuilder also allows to query for data that is not explicitly represented in the graph by nodes, but tangentially, such as the User of a node, or its Computer.
@csadorf @giovannipizzi what do you think? Just to be clear, I don't want to make this a hugely contentious point, as I would be fine with both, but think it is interesting to discuss it a bit
I'd have expected this under "Data", but I see @sphuber 's point about the QB specifically designed to explore the provenance graph. I have no strong opinion, but lean towards "Data".
@csadorf Should I still clean up the Advanced querying part of this topics section and do a PR? The topics section won't be complete, but at least the links in the querying how-to will work. ^^
There is a TODO for this in https://github.com/aiidateam/aiida-core/blob/develop/docs/source/topics/database.rst, but it is unclear what actually needs to be done here.
@sphuber, can you update the initial comment with the steps to close this issue, or simply create a PR deleting the TODO in the docs and closing this issue
can you update the initial comment with the steps to close this issue, or simply create a PR deleting the TODO in the docs and closing this issue
Done. updated the title also to show that this should concern the storage as a whole and no longer just the database.
Done. updated the title also to show that this should concern the storage as a whole and no longer just the database.
Do you think #5424 closes this, or no?
Not sure, will have to look,. but the architecture is different from the topic section. The former should be aimed at aiida-core developers whereas the latter is for users. The difference would be that the former expounds on design decisions of the internal implementation, whereas the latter goes more into the direction of providing information and details that make the more advanced features of AiiDA easier to understand and use.