Line continuation in curly braces

[dbohdan]

Jim does not process line continuations in curly braces, which causes incompatibility with Tcl 8 code.

$ cat demo.tcl 
puts "$tcl_platform(engine) [info patchlevel]"
puts "foo\
      bar"
set s {lorem\
       ipsum}
puts $s
puts [string length [lindex $s 0]]

$ tclsh demo.tcl 
Tcl 8.6.8
foo bar
lorem ipsum
5

$ jimsh demo.tcl 
Jim 0.77
foo bar
lorem\
       ipsum
6

[dbohdan]

Another way the difference manifests:

$ cat demo2.tcl 
puts [list {*}{foo\
               bar}]
$ tclsh demo2.tcl 
foo bar
$ jimsh demo2.tcl 
{foo } bar

[dbohdan]

If you're interested, I've just pushed a number of tests for Tcl 8-compatible handling of line continuations to Picol's repo.

[msteveb]

Thanks for the report. I was aware of this incompatibility with Tcl but I didn't see it as a significant feature. (i.e. I've never needed it). I would be happy to hear specific issues with this missing feature aside from simply incompatibility.

[dbohdan]

The issues I have seen come down to long string literals with meaningful whitespace being corrupted in code that otherwise works in Jim. The most common has probably been help messages displayed incorrectly when written like the one below.

puts stderr {Lorem ipsum dolor sit amet, consectetur\
             adipiscing elit. Aenean non pharetra\
             metus, vitae tristique sem. Praesent\
             at sapien non nibh suscipit ultrices.}

Those are only an annoyance. More important, though far less common, have been problems with multiline regular expressions and document templates. For example,

if {[regexp {a very long regular expression split on a word\
             boundary} $foo]} ...

set template {
...
        <deeply>
          <nested>
            <tags foo="a" bar="b" \
                  baz="c">
              %5$s
            </tags>
...
}

[msteveb]

Here, is a fix: https://gist.github.com/msteveb/bc468decd86293c185e498adff9a2199

But unfortunately it breaks line numbering:

source-1.2 ERR Line numbers after command with escaped newlines
At      : stacktrace.test:53
Expected: 'stacktrace.test 43'
Got     : 'stacktrace.test 41'

source-1.3 ERR Line numbers after string with newlines
At      : stacktrace.test:56
Expected: 'stacktrace.test 47'
Got     : 'stacktrace.test 45'

source-1.4 ERR Line numbers after string with escaped newlines
At      : stacktrace.test:59
Expected: 'stacktrace.test 51'
Got     : 'stacktrace.test 47'

 stacktrace.test: Total    45   Passed    42  Skipped     0  Failed     3

I don't see any easy way around this. I prefer line numbering to be correct.

[dbohdan]

Thanks for the patch! Yes, I agree it's a bad regression. So far I've spent a couple of hours trying to hack up a solution localized to just ScriptObjAddTokens and JimMakeScriptObj. I see a way to do it by attaching metadata about the squashed newlines' offsets to the script object, but this solution would be cumbersome and ugly. An alternative is to refactor the parser. I'll get back to you if I implement either.

[The-Markitecht]

I wonder if this is also the root cause of the error I get whenever I break a long list of proc parameters into 2 lines, in version 0.79:

$ cat proc.tcl 
proc p {a \
    b} {
    puts $b
}

p one two

$ ./jimsh proc.tcl 
proc.tcl:1: Error: argument with no name
at file "proc.tcl", line 1

I run into that often, and it's annoying. I'm in the habit of doing it that way, from Tcl 8.x

Just now I did find that removing the backslash makes it work:

$ cat proc.tcl 
proc p {a
    b} {
    puts $b
}

p one two

$ ./jimsh proc.tcl 
two

Is that the same workaround y'all use? I wouldn't want any interp changes that break line numbering in stack dumps. The line numbering is one of THE MAJOR reasons I've adopted Jim across the board. Its absence in Tcl 8.x is simply ASTONISHING to me.

[msteveb]

I agree regarding line numbers and I wouldn't want to give it up for this feature. If anyone implements an elegant solution to this, I'm happy to consider it. In the meantime, yes I simply remove the backslashes.