Allow a library prefix (not an import prefix) to support access to shadowed names

[eernstg]

This proposal is a slight generalization of a proposal from @lrhn that occurred here.

The idea is that a library directive should be able to introduce a name for the library itself. This name can be used in a way that is similar to an import prefix to denote top-level or static member declarations that are shadowed by other declarations in intermediate scopes. It differs from an import prefix in that it can be used to denote a declaration whose name is private, and also in that it uniquely denotes the top-level namespace of the current library (import prefixes can denote other libraries, and they can be shared by more than one imported library).

A library 'lib.dart' would name itself (as an example, we're using the name this_library) as follows:

// --- Library 'lib.dart'.
library as this_library;

final int _foo = 10; // Example private declaration.
class E {} // Example public declaration.

We could then use it as follows, in 'lib.dart' or in a part which is a part of 'lib.dart':

class A<E> { // `E` shadows a top-level declaration.
  void _foo() {} // Shadows a top-level declaration.
  void f() {
    print(this_library._foo); // OK.
    new this_library.E(); // OK.
  }
}

Note that the underlying problem (shadowing) cannot always trivially be removed by renaming some declarations in the current library: A public static member can be used from other libraries, so we may not be in a position to rename that static member. However, it could still shadow some other declaration which is imported, which means that we also cannot easily rename that other declaration.

Similarly, we may not want to rename a class or a type variable with a shared public name even when they are declared in the same library: The class may be used by many clients outside this library, and the type variable name may be widely known because it occurs in documentation.

With generated code or macros, we may need to denote a declaration (possibly private) in a way that unambiguously resolves to a certain top-level or static member declaration, even in the case where the declarations in intermediate scopes are not known. This can safely be done in a named library, using constructs like this_library._foo.

Grammar

<libraryName> ::= <metadata> 'library' <dottedIdentifierList>? ('as' <typeIdentifier>)? ';'

Static analysis

Assume that a library L has been equipped with the name N using library as N; or library D as N; where D is derived from <dottedIdentifierList>, and N is derived from <typeIdentifier>.

A compile-time error occurs if a declaration or import prefix in L has the name N. Otherwise, this introduces the name N into the library scope of L.

N does not belong to the exported namespace of L.

This means that N shadows imported declarations named N, if any. Also, it behaves just like import prefixes with respect to exports.

Assume that N is an identifier expression that occurs in L and resolves to the prefix which is introduced by <libraryName> (that is, it isn't shadowed). It is a compile-time error if it is not immediately followed by ..

Just like an import prefix, N cannot be used as an expression, and it does not have a static type. Also note that N?.id is a compile-time error for any identifier id, and so is any cascade starting with N.. or N?...

Assume that a qualified identifier of the form N.id occurs in L, where id is a private or a public name. This expression resolves to a declaration named id in the top-level scope of L, if it exists. A compile-time error occurs if it does not exist.

This determines all further static analysis and the dynamic semantics of expressions starting with N.

Discussion

This is a very simple mechanism with a clear purpose. The ability to denote a top-level, imported, or static member declaration in a situation where some other declaration shadows it is generally useful, even though it is probably not used very often (we don't have it today because it wasn't needed that badly). It should be noted, though, that the ability to rule out shadowing in crucial for mechanically created code (such as plain code generation or macro execution), because there is no other way to ascertain that no shadowing can occur.

As an alternative, we could modify the rules about imports and exports such that an import of the library L that occurs in L itself could include private names, and such self-imports could then be used to overcome the shadowing problem.

However, that's a more complex undertaking because it changes the meaning of imports (now we need to learn that, as an exception, self imports do include private declarations), and the resulting code is less explicit on the purpose and meaning of the self import. Even a style rule contributes to the confusion: The self import may occur somewhere in the middle of a number of other imports, and it may be confusing that this particular import "can see private names". Next, how about self imports via exports of other libraries? Do they not include private names? ... my private declarations definitely can't be included when the exporting library is imported into some other library.

I think this adds up to a strong hint that we should use a separate mechanism rather than generalizing imports.

[eernstg]

@johnniwinther, is this hard to implement?

[ghost]

Won't it be easier to reserve a standard prefix like _this_library or __this_library or _$this_library or something? (As a bonus, your chosen pattern could be used for other prefixes, like __dart.int).

[johnniwinther]

This should be fairly simple to implement.

[eernstg]

@tatumizer wrote:

Won't it be easier to reserve a standard prefix

That would be good for immediate readability, but such a standard prefix would presumably have to be a reserved word—what would we do if this name is itself shadowed? And we don't add reserved words lightly, if at all, ever. The safe bet is probably to require a declaration and develop a convention for which word to use.

[lrhn]

Maybe not a reserved word, but at least a built-in identifier, since it works as a prefix for types.

That said, the name would have to be one that is unlikely to conflict with other code, so either a longer name or something starting with _. We could use _, but that's not particularly readable, and we plan to make _ non-binding as a prefix otherwise, but at least that means it wouldn't conflict with those.

The point of allowing the user to choose the prefix, and not, say, always import package:.../banana.dart with a prefix of banana is to let the user avoid name conflicts. Any automatically chosen name could again be shadowed, and then we're back to square one. At least if it's a private name, the user can avoid declaring anything with the same name, so _this, _library or _top might work.

@eernstg This says:

This expression resolves to a declaration named id in the top-level scope of L, if it exists.

The top-level scope includes both declarations in the library and prefixes and imported names not shadowed by declarations. Is it intended that you can use id.importedName to refer to an imported name, not only a declaration?

I guess it's useful, because then you can access shadowed imports without needing to add a second prefixed import too, but it was more than I expected. Maybe just because of bias from coming at this from the import "" as self; direction.

It's actually so useful I can see myself always adding a library as top; to my files, and only removing it if I end up not using it.

I would probably use library as top; or library as _top; as my go-to names, if it denotes the top-level scope. (Can I write top.top.top.top.name and get the same result, that is, is the library-self-name in the top-level scope?)

[eernstg]

Is it intended that you can use id.importedName to refer to an imported name, not only a declaration?

It seemed useful and benign... it would always be possible to use an import prefix, but reusing the library prefix to do this job might be convenient if the shadowing situation is rare and no other concerns justify having the import prefix.

Can I write top.top.top.top.name and get the same result

I didn't think about that. Doesn't seem harmful, either, by the way. ;-)