jackson-dataformats-text
jackson-dataformats-text copied to clipboard
FasterXML
Emit comments
Is it please possible to emit comment lines with the keys? I could imagine this being done with an annotation for the respective class member. Also, adding whitespace, e.g. empty line would be a nice feature as well.
If this is not currently possible, could anyone please point me to the code where this magic would be happening so that I can add this feature?
Many thanks! :)
There is no way to emit comments, in general, by Jackson, so it would be necessary to first figure out why this is something to add (use case, and where contents of comments should come from). After that it would be necessary to figure out how piping of comment contents should occur (what should added, based on metadata).
So... what is the use case? Why comments? Containing what?
Using a YAML file as a user-setup file in which an user will be able to change some settings in an easy way, using a notepad.
If you want to clarify some of the options you use comments so the user will be able to understand what he should modify and what will cause any of the modifications.
I guess I am more used to people adding comments manually, and more trying to understand how a machine-processable generation would add comments. I am familiar with comment usage in XML, where generators may add sort of prefix comment containing diagnostics metadata (generation timestamp, system etc).
I think it would be possible to add something like writeComment in base class (JsonParser) and let formats implement it if available, or throw exception if not.
Although there's further question of style of comments to use for formats that allow multiple styles (json does not have official comments, but there are 3 distinct styles that are sometimes used).
Even if Jackon finds a way to create API to emit comments, SnakeYAML explicitly does not support dumping comments. It is not supported by the YAML spec.
@asomov Hmmh. Ok, that would indeed be a blocker. I understand that retaining comments can be tricky (as they really do not play well with object models in general), wrt reading. But was assuming that there might be a way to generate them -- this was useful with XML, for example, adding comments as human-debuggable headers for things like "file generated by [tool name] on [date]" and such, instead of (or in addition to) actual machine-readable (meta)data.
@asomov I understand that. I just find this... unexpected. Thank you for pointing out point in specification.
Just one point however: while this clarifies that comments are essentially not to be exposed when reading YAML (or preserved within model), they clearly exist in textual representation. As such it would seem odd that generators could not produce them. Presumably that would mean that only humans with with text editors were to use them as... annotations?
Put another way: specification does not seem to have much to say on output/generation side, which is what this issue is about.
The spec says that the generation must be controlled by the composed Node tree. The comments are not part of the Node tree. You can see that as the white space. It is present and important but it is difficult to control while dumping the output. Of course, if a parser (jackson) could find a way to dump the comments it will be NOT against the spec.
@asomov Ok. I am not familiar enough with the specification there -- it is rather big spec, after all. Thank you for pointers so far.
Another way to view this would also be whether any other YAML tools allow emission or not. I tend to prefer working similarly to existing tools, and if (for example) such generation is not supported elsewhere, would not want to be introducing such non-compliant behavior.
At this point it does sound like adding such output may not be a good idea.
@asomov @cowtowncoder Are you sure that comments can't be included? http://yaml.org/spec/1.1/#id863676 There seems to be part of representation tree, even more, you can include whole styling here. For sure you can't read this, but you can include this. Note that you can include that outside the node, just pass additional argument to emitter with data about comments/style, in my projects I had special object to store comments mapped by path.
Where do you see that they are a part of the tree? This is what the spec says:
Comments are a presentation detail and must not have any effect on the serialization tree or representation graph. In particular, comments are not associated with a particular node.
" just pass additional argument to emitter with data about comments/style" - well, if you have a proposal how to implement that then feel free to share it with us.
@asomov you ofc need own emitter then, as SnakeYaml does not support anything like that, so it is probably too complicated to include and support inside jackson
- this is the issue tracker for jackson
- SnakeYAML is not written in stone, it can be changed if we know how.
- You said there is already a project which emits comments. Why not to share with us the API ?
@asomov this was my "private" project and internal code - so I didn't really care about being compilat with yaml specification, I just needed comments, so it was more like a hack. So it is not anything you would want to use in more popular and used project :)
Than I do not get the reason to start the conversation here (with the references to the spec).
I just wanted to throw few ideas here (like passing that comments object next to nodes, as this seems to be fine with specification?), as once I also needed feature like that but needed to do it alone i a bit hacky way as there isn't any ready-to-use solution for generating nice configuration files with comments, but would be nice to finally have some better library for this. Sadly jackson can be problematic anyway with yaml, as after all yaml have many features that don't map well to json model, like object keys.
Sorry then :)
On plus side, I think there is agreement that:
- Support for writing comments, if any, makes most sense on SnakeYAML side -- it handles all low-level generation and decoding
- If such support is added, Jackson can probably be modified to take advantage: there are other data formats with support for comments (XML has them as legal thing; JSON has non-standard extension; CSV similarly has means to embed)
I think I will leave this open as a placeholder just in case someone else was looking for this, so it is easy to see our current thinking.
I just wanted to throw few ideas here (like passing that comments object next to nodes, as this seems to be fine with specification?), .
Sorry then :) Without any API proposal this idea is completely unclear for me. What is "that comments object" ?
@asomov I don't know if there is any reason for me to describe this, as this was created for simple configuration files, I didn't need to care about many corner cases.
in my code I just created CommentsDocument object that was storing header, footer and comments data for each node described by path inside yaml structure, like: some.object.field with comment hello to get simple yaml:
some:
object:
# Hello
field: ...
For lists I was just ignoring them, so it would also work for:
some:
object:
-
# Hello
field: ...
And for nested objects in other maps I had a wildcard token: some.other.*.node
Additionally there was a class that was able to read comments from annotations over classes and include them to final document while/before serializing.
But my code was only designed for simple configuration files, and it just needed to work ;) PS: nice feature is to have option to only include comment once, like having this same comments in each list element might be pretty ugly, but again: I only did that for configuration files so I still think only about that case.
@cowtowncoder SnakeYAML Engine is going to support dumping comments. I wonder if we can somehow use it in Jackson. @lovasko @GotoFinal @IhToN can you try to marry the new feature in SnakeYAML with Jackson ?
Adding support may be tricky as a general feature, although adding methods in YAMLGenerator should be relatively easy. Not sure how this could be abstracted at higher levels though.
@cowtowncoder if no one wishes to contribute, than the feature is not that useful. We have done our part and we are ready to help the volunteers to make the dream come true.
@asomov I appreciate your adding functionality that makes it possible, and will be happy to at least add one mechanism on YAMLGenerator. Just wondering how this could be generalized: handling of comments in data formats is generally tricky, but also useful: XML format module would benefit from similar mechanism, for example.
Marking as "good first issue" in case someone has time to just do the simple thing first.
@cowtowncoder unfortunately I cannot help you. The support for comments in SnakeYAML ends on the Node level (before Java instances come into play). I am not sure it can be generalized to be used in Jackson. You can check the folder with tests to see how comments can be used: https://bitbucket.org/asomov/snakeyaml-engine/src/master/src/test/java/org/snakeyaml/engine/v2/comments/
Important: support for comments is not yet released ! I wanted to give it a try. May be we will need to change the API. This is exactly want I expected - somebody could try SnakeYAML before we release the new API
That level makes sense. If I understood it right with a quick look, emission of comments would need Jackson to associate comments with other node types to emit (like value that is written) next. That's probably quite doable. Still, would be good to know what users are sort of trying to achieve. For example, ability to add "header comments" is a common use case for at least xml -- add a note about time file was generated, for example.
I am very interested in this but am not sure I have the time to do a contribution. Thanks for keeping it alive.
Even today, I am looking for this solution. Let me know if anyone finds any solution to achieve this
Hi All, Thank you for this informative discussion. I am too looking for this feature in snakeyaml & it is definitely a good thing to have. For example even microconfig uses comments to mark / annotate things in YAML. When there is merging of YAML files, it does create issues since comments are erased in the final outcome.
@asomov - I cannot access this https://bitbucket.org/asomov/snakeyaml-engine/pull-requests/1 .. Can you provide access to this ? Also you mentioned that snakeyaml is going to add support to dump comments - can you please point me to some reference related to this?
Btw, I just found this -- https://bitbucket.org/snakeyaml/snakeyaml/issues/497/how-to-use-the-new-process-comments .. Havent tried it yet, but seems like something was done around this. Can you please confirm