Goto definition not reliable

[TurkeyMan]

Hey there, just trying out your new build. Other recent issues appear to be fixed. Today I wanted to try and get to the bottom of some of these issues with goto definition not working reliably.

The first sample case I bumped into looks like this:

char[] tconcat(Args...)(ref Args args)
{
	import urt.string.format : concat;
	char[] r = concat(cast(char[])tempMem[allocOffset..$], args);
	if (!r)
	{
		allocOffset = 0;
		r = concat(cast(char[])tempMem[0..TempMemSize / 2], args);
	}
	allocOffset += r.length;
	return r;
}

You can see the import urt.string.format : concat; there, and 2 calls to concat inside this function. If I cursor over concat and goto definition, it doesn't go anywhere. I figure the first thing to do is check the log for an error explanation, but it didn't say anything:

19:25:19 - ba64 -   schedule GetDefinition: D:\Code\monitor\src\urt\mem\temp.d
19:25:19 - ba64 - GetDefinitionResult: 
19:25:20 - ba64 -   schedule GetDefinition: D:\Code\monitor\src\urt\mem\temp.d
19:25:20 - ba64 - GetDefinitionResult: 
19:25:21 - ba64 -   schedule GetDefinition: D:\Code\monitor\src\urt\mem\temp.d
19:25:21 - ba64 - GetDefinitionResult: 
19:25:21 - ba64 -   schedule GetDefinition: D:\Code\monitor\src\urt\mem\temp.d
19:25:21 - ba64 - GetDefinitionResult: 

What's my next port of call to try and track this down?

I did wonder; given a template function without an instantiation like this, how does the frontend even perform semantic to determine what these local symbols are? Can DMD begin to meaningfully compile this code without knowing what Args is?

[rainers]

I did wonder; given a template function without an instantiation like this, how does the frontend even perform semantic to determine what these local symbols are?

This is the problem we are indeed facing here. The frontend doesn't do any semantic analysis unless an instance is created. dmdserver grabs the first of these instances assuming all are similar, but if there are none, it has nothing to work with.

In C++, you can select template arguments explicitly that should be used for analysis, but that never really worked for me.

Maybe some annotation or comment in the source code could be used, but then you could also just write

version(VisualD) // assuming this is added 
    enum instance = tconcat!string;

after the template delaration.

[TurkeyMan]

Why can't the compiler begin semantic on an uninstantiated template? Imagine the AST for an uninstantiated template; the template args could be some void or something like that, but from there I don't see any reason semantic couldn't begin, and resolve all the way down until it reaches a reference to one of the undefined template args... it would be able to run semantic on a whole lot of typical code.

Do you have any idea what C++ does in this situation?

[rainers]

Visual D could try to instantiate the template with some arbitrary types, but they might not fit at all or fail constraints or static if checks. Maybe this is good enough for most templates, I could give it a try.

Do you have any idea what C++ does in this situation?

MSVC used to be rather sloppy, too. Clang always complained when it could not infer types or expressions, so you have to add typename, this. or similar to clarify. Microsoft followed suite some time, I guess it has been added to the standard. Intellisense for templates works pretty well these days (from browsing the STL a bit), but they probably invested quite a bit of effort.

[TurkeyMan]

version(VisualD) is an interesting idea, and trying some default thing in lieu of that makes sense... many/most templates can make a strong go at semantic before they hit a template arg. Many templates are surprisingly loosely coupled to their template args; one of D's biggest challenges really; hyper-over-use of templates.

What surprises me about my case is that an instantiation SOMEWHERE wasn't picked up... it's like I never used that function before; but reality is I use that function literally everywhere! Maybe it wasn't called inside that module, or a module I had open in the editor...? Does VisualD make effort to compile everything in the project to prime the intellisense database, or does it only compile single modules as they appear for editing?

[rainers]

The dmdserver only looks at modules currently being edited (and files imported by them). Template instances are gathered from any file it analyzed, but as soon as you edit the file containing the template, the module AST including all generated template instances are thrown away and they are only rebuilt if the file instantiating them is edited again. That's why it's easy to end up with no template instance. There is some simple dependency tracking based on imports, but IIRC rebuilding all affected modules was too expensive even in moderately sized projects.

I tried creating a default template instance with generic template arguments (mostly typed void), but some templates behave worse than just generating errors, e.g. std.typecons.GetOverloadedMethods produces an infinite recursion causing the frontend to bail out completely. Still looks like the way to pursue, especially if using types named identical to the template arguments. Choosing defaults for value arguments can be even more problematic, though.

[TurkeyMan]

Okay, so if the template instances are discarded regularly, and the instantiating code may not be in the editing vicinity, and if you're already carrying state like dependency tracking; maybe carry around known valid instantiation arguments? So even if the ast is discarded, you might remember from some time in the past a set of actual valid template arguments. Then we can have confidence they should work? Maybe even carry around several known instantiations to try and fill out internal static branches?