stringtemplate4
stringtemplate4 copied to clipboard
Separator not emitted for nulls until a value is hit
We use string templates for a variety of data exports. One of which is simple CSV.
If we have an input from a table of data where as long as the first value in each row is non null then the CSV is correctly formed. When the first value is a null a separator is not emitted until there is a value.
Imagine the following table of data
Name, Age, Color Ron, 10, Pink Paul,15,Brown
and the template
printRow(row) ::= <%
<items.Columns:{ k | <row.(k)>}; separator=","><\n>
%>
outputTemplate(items) ::= <%
<items.Columns; separator=","><\n>
<items.Rows:printRow()>
%>
this will output Name, Age, Color Ron, 10, Pink Paul,15,Brown
if however the input data was Name, Age, Color Ron, 10, Pink null,15,Brown
Then the output would be
Name, Age, Color Ron, 10, Pink 15,Brown
and not
Name, Age, Color Ron, 10, Pink ,15,Brown
I can see it is because of this section of code
because the seenAValue is false until the first value in the enumerable is non null.
In this case there is a work around, which is to substitute nulls with "" which will generate for the example above
Name, Age, Color Ron, 10, Pink "",15,Brown
However I was looking at the possibility of adding another option like emitAll="true" or something like that which would be used along with the separator option to still output, something like
boolean needSeparator = (seenAValue || (emitAll && separator!=null)) &&
separator!=null && // we have a separator and
(iterValue!=null || // either we have a value
options[Option.NULL.ordinal()]!=null); // or no value but null option
However I do not have a clear idea of the impact this could have and was hoping someone for familiar with the code could advise me
I can't remember but I thought we had a template option that would test whether value is null or not and use a replacement. Oh, yeah, this thing: options[Option.NULL.ordinal()]!=null
.
At this point its worth mentioning I am debugging the c# version so its possible there are differences. The codebase is largely neglected. However as those parts of the code are identical I thought I would ask here. Seems like the need to gobble up nulls at the start of a series is necessary to support some of the formatting allowed in the templates. Its the fact that the same code is also used with the separator option that gives me some hope it can be tweaked.
I will get a simple Java version going if I can to test it, but it seems in this context that null never gets triggered
My 'List' that is being iterated is actually an OrderedDictionary so it's possible that in a simpler context of just a plain list the null gets realised but here it doesn't. I'll have another play later now I have some more leads to follow.
Something else is that the value ultimately gets written here
and not in the with options variant, I don't quite understand how those opcodes are generated at this point
Here is a very simple example project in Java exhibiting the same behaviour
https://github.com/rrs/StringTemplateEmitSeparator
Dang. Does seem broken (haven't run code yet though).