sclang: Parser does not allow class definitions after extensions

[jleben]

[Issue migrated from SourceForge | ID: 1901169 | Submitted by 'rbrown46'] [http://sourceforge.net/support/tracker.php?aid=1901169]

Compilation errors result if you add an extension to a class and then define another class in the same file. Order matters. I'll post a specific example later.

[nhthn]

Can reproduce:

NathansTest { }

+ NathansTest {
    foo { ^"bar"; }
}

NathansSecondTest { }

Here is the error:

ERROR: syntax error, unexpected CLASSNAME, expecting $end
  in file '/home/nathan/.local/share/SuperCollider/Extensions/test/Test.sc'
  line 7 char 17:

  NathansSecondTest { }
  ^^^^^^^^^^^^^^^^^

-----------------------------------
ERROR: file '/home/nathan/.local/share/SuperCollider/Extensions/test/Test.sc' parse failed
error parsing
Library has not been compiled successfully.

Evidently mixing extensions and classes is not supported by the parser.

[scztt]

@snappizz You can mix them in the same file, as long as the extensions are at the end. This is a parser issue - I tried solving it a while ago and it was a bit tricky (but I'm all thumbs when it comes to parser stuff....).

This section of the lang11d has to be rewritten, probably so that 'classes' and 'classextensions' are a single entry, which can contain either 'classdef's or 'classextension's. It's a worthwhile (and probably not TOO hard) experiment for anyone that wants to learn a bit more about the lang11d, and it would be backwards compatible so it could be introduced without breaking existing things.

[scztt]

Actually, I wonder if this would work:

root    : classes
            { gRootParseNode = (PyrParseNode*)$1; gParserResult = 1; }
        | INTERPRET cmdlinecode
            { gRootParseNode = (PyrParseNode*)$2; gParserResult = 2; }
        ;

classes : { $$ = 0; }
        |   classes classdef
            { $$ = (intptr_t)linkNextNode((PyrParseNode*)$1, (PyrParseNode*)$2); }
        |   classes classextension
            { $$ = (intptr_t)linkNextNode((PyrParseNode*)$1, (PyrParseNode*)$2); }
        ;

That appears to compile.... someone want to try it?

[nhthn]

I tried something similar but ran into this bug: https://github.com/supercollider/supercollider/issues/2259

[mossheim]

@scztt Was able to compile with those changes with Bison 2.3; however, I get this:

pass 1 done
ERROR: There is a discrepancy.
numClassDeps 2149   gNumClasses 4300

[scztt]

Huh - so, it should be the case that: 2*numClassDeps == gNumClasses. Looks like numClassDeps is one off - that's pretty close, I'd bet that there's one case somewhere that isn't parsing correctly? You might try printing out classes in newClassDependancy and indexClassTree, and see which one ends up missing from the former?

[mossheim]

OK I think I fixed it (deleted the last comment where I was a little more sure of that).

I managed to compile a class library including this file successfully, and exec:

TestClass {
	classvar <>var1;

	method1 {
		"method1".postln;
	}
}

+ String {
	junkMethod {
		"hi".postln;
	}
}

TestClass2 {
	classvar <>var2;

	method2 {
		"method2".postln;
	}
}

And then running

(
TestClass.var1.postln;
TestClass.var1_(3);
TestClass2.var2.postln;
TestClass2.var2_("a string");

TestClass.var1.postln;
TestClass2.var2.postln;
TestClass.new.method1;
TestClass2.new.method2;
"String".junkMethod;
""
)

for sanity's sake gave the expected results

nil
nil
3
a string
method1
method2
hi

It's a long story, and I need to go to sleep now, but basically the parser fix you gave above was perfect. The problem was this beautiful block:

	} else if (token == '+') {
		token = yylex();
		if (token == 0) return false;
		scanForClosingBracket();

		newClassExtFile(fileSym, startPos, textpos);
		return false;

That immediately cuts to the end of the file as soon as a + is found, through an abuse of scanForClosingBracket(). It took some time and lots of rebuilding with added post()s for me to realize it's being called before a opening bracket is ever identified. So it'll just keep scanning to the end of the file even if it never actually finds a closing bracket, and then quietly return.

Will make a combined PR in the morning for this and #2259 and I'd really like to get someone else to test! This is issue fricking 52 after all.

[scztt]

This is awesome! If you have a sec, can you push your change as a topic branch to the main repo? If you do that, travis will spin a build we can hand out for testing.

[mossheim]

I don't have permissions to do that, but there's a PR ^

[telephon]

btw. if I remember correctly: originally, the implementation allowed only either extensions or class files separately. Then, it allowed extensions after class files. That was communicated, but never documented. Good that it is more flexible now!

[mossheim]

Interesting! Hm.. I might look into this a little more. We could potentially simplify it even further if class extensions can be parsed as soon as they're found.

[LFSaw]

is this actually an issue to be fixed or something we should document clearly somewhere?

[mossheim]

@LFSaw issue to be fixed. A fix was presented in #2630 and I can make the fix sometime soon once other more important fixes are done.

[LFSaw]

Ah, I got mixed up with #2630 being closed (thought this also implies merged but no...). Thanks for the heads-up!

[Evrikast]

I am facing the exact same problem and i dont understand why parser faces it as a problem

I'm wiriting down:

~Buf1 = Buffer.read(s, "C:/Users/user/Desktop/September programming 2/Samples/GtrMono1.wav"); ~Buf1.numChannels = 1;

and the PostWindow shows this!

ERROR: syntax error, unexpected CLASSNAME, expecting NAME or WHILE in interpreted text line 1 char 5:

~Buf1 = Buffer.read(s, "C:/Users/user/Desktop/September programming 2/Samples/GtrMono1.wav"); ^^^^

/////////////////////////////////////////////////

Why is this happening pls help!

[capital-G]

@Evrikast the problem you mentioned is a different problem from the one this issue is about.

Your problem can be fixed by writing ~buf1 instead of ~Buf1 - this is because class parsing (indicated by an upper case letter) takes precedence over environment variable parsing (indicated by a ~).

[Evrikast]

Oh thank you alot!!