C++: introduce "using" role to "namespace" kind

[masatake]

"foo" in "using namespace foo;" was captured as using kind. However, "foo" is not defined in the statement. "foo" is referred as a namespace defined somewhere.

Therefore, ctags should not capture "foo" as a definition tag. Instead, ctags should capture it as a reference tag.

The original code captures "foo" as a definition tag of "using" kind. This change captures "foo" as a reference tag of "using" role of "namespace" kind.

$ cat /tmp/foo.hh using namespace std::cout; $ ./ctags -o - /tmp/foo.hh $ ./ctags -o - --extras=+r /tmp/foo.hh std::cout /tmp/foo.hh /^using namespace std::cout;$/;" n $ ./ctags -o - --extras=+r --fields=+K /tmp/foo.hh std::cout /tmp/foo.hh /^using namespace std::cout;$/;" namespace $ ./ctags -o - --extras=+r --fields=+Kr /tmp/foo.hh std::cout /tmp/foo.hh /^using namespace std::cout;$/;" namespace roles:using

$ ./ctags --list-roles=C++.namespace #KIND(L/N) NAME ENABLED DESCRIPTION n/namespace using on specified with "using namespace"

Signed-off-by: Masatake YAMATO [email protected]

[masatake]

@pragmaware, could you look at this one?

[coveralls]

Coverage decreased (-0.004%) to 84.384% when pulling 13ef94a7f3e0d2b01c55c81161f2b65921bb401d on masatake:cxx-using-as-reftag into 28a95d08e1b71ed721757a1fa1f7b1720edd05b4 on universal-ctags:master.

[pragmaware]

While I agree that this is a different usage of a namespace, and it is certainly a reference, and the code looks perfectly fine, I'm not 100% positive with using roles here. For a couple of reasons:

• "name", the symbols imported via "using" statement fall exactly in the same category but can't be described by roles because we don't know their real category (i.e. we don't know if the imported symbol is a function, a variable or a type). Extending the reasoning we could even say that function prototypes are really different roles for functions. Handling them as roles would break backward compatibility though. It's hard to make the usage of roles consistent.

• roles make both emission of tags and parsing of the tags file more complicated with no real gain. The users just know that "using" is a reference to a namespace: there is no ambiguity. ctags is already a quite complicated software to control. I'm a developer and still I'm always quite in trouble when I have to switch on/off kinds and fields...

Having said that... if you go for the roles, then this pr looks fine :)

[masatake]

Thank you for comments.

"name", the symbols imported via "using" statement fall exactly in the same category but can't be described by roles because we don't know their real category (i.e. we don't know if the imported symbol is a function, a variable or a type). Extending the reasoning we could even say that function prototypes are really different roles for functions. Handling them as roles would break backward compatibility though. It's hard to make the usage of roles consistent.

About function, it is good point. I thought I would like to introduce prototype role of function kind. It could be replacement the prototype kind...At least, we can introduce the prototype role without removing the prototype kind. The prototype kind can be enabled/disabled from command line, so a user can choose 'using prototype kind and not using prototype role' or using prototype role and not using prototype kind'.

Having said that... if you go for the roles, then this pr looks fine :)

I would like to go for the roles. I would like to see what kind of application could we write on the roles.

[masatake]

Like prototype kind, externver kind can be replaced with 'prototype' role of 'variable' kind (or 'extern' role of 'variable' kind).

[masatake]

About 'N name' kind, we can make the same.

Real kinds for 'name' in C++ are resolved if we solve https://github.com/universal-ctags/ctags/pull/1495 :-)

[masatake]

"used" will be better than "using". "used" is more consistent with other languages like python. In python "imported" is used as the role name.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (24f8524) 87.35% compared to head (1a95c56) 87.37%. Report is 1667 commits behind head on master.

:exclamation: Current head 1a95c56 differs from pull request most recent head 75c547f. Consider uploading reports for the commit 75c547f to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1783      +/-   ##
==========================================
+ Coverage   87.35%   87.37%   +0.01%     
==========================================
  Files         200      200              
  Lines       47839    47865      +26     
==========================================
+ Hits        41792    41821      +29     
+ Misses       6047     6044       -3     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[masatake]

I understand the suggestion about "name".

using std::cout;

Currently, ctags emits:

cout	foo.cc	1;"	kind:name	line:1	language:C++	roles:def

This should be:

cout	foo.cc	1;"	kind:name	scope:namespace:std	roles:used	extras:reference

name is similar to "unknown" in Python parser. for foo.py:

from X3 import Y3

ctags emits:

X3      input.py        /^from X3 import Y3$/;" kind:module     roles:namespace
Y3      input.py        /^from X3 import Y3$/;" kind:unknown    scope:module:X3 roles:imported

(See https://docs.ctags.io/en/latest/man/ctags-lang-python.7.html for more details.)

Ideally, std should be tagged, too:

std	foo.cc	1;"	kind:namespace	line:1	language:C++	roles:namespace

The role namespace is a bit odd. The role name comes from the fact that std is referred to as a namespace.

What I don't decide what I should do is foo::bar in

using foo::bar::baz;

We can expect foo is a namespace. How about foo::bar ? Can I expect foo::bar is a namespace? I need more study in this area.

[pragmaware]

using std::cout;

Ideally, std should be tagged, too:

Not sure...

Well, technically it could be a solution, but it's quite a convoluted one. If many names are imported with the same namespace path then there would be a lot of repeated "fake" namespace declarations. Not ideal.

It's cout that is being imported in the current namespace. std is just used to "point" at cout but is not imported nor used in any other way.

We could probably just stick std as scope of name cout: that would be unambiguous...

[masatake]

@pragmaware, thank you for your comments.

About tagging std in using std::cout;, I would like to withdraw the idea.

I implemented the other ideas.

I have two remained items.

If using specified a name started from :: like using ::std::cout;, what we should do? I think an answer is given for this question when an answer is given for the next more important question.

namespace X {
   using A::B;
}

With changes, I proposed in this pull request, B is tagged as a reference tag with name kind and name:A as scope. How can we represent the relationship between X and B?

I'm thinking about a new field "context:" or "where:".

X       .../;"     kind:namespace  roles:def       end:3 
B       ...;"    kind:name       scope:name:A    roles:used       context:namespace:X     extras:reference

This question is not only about namespace. Most of all reference tags are relevant.

void foo(void) { }
void bar(void)
{
    foo();
}

In the input, how the reference tags for foo();?

foo     ...;"   kind:function   typeref:typename:void   signature:(void)        roles:def       end:1                                                                                         
bar     ...;"   kind:function   typeref:typename:void   signature:(void)        roles:def       end:5                                                                                         
foo     ...     /^    foo();$/  kind:function   roles:called    context:function:bar    arguments:()

I have no plan to implement called role. However, this is one of good materials for thought experiment about reference tags.

struct s {
  int i;
};
...
struct s *S;
...
void f(void) {
S->i = 1;
}

When making a reference tag for i in S->i = 1;, how the tag should be?

... scope:variable:S    context:function:f...

[pragmaware]

namespace X {
   using A::B;
}

With changes, I proposed in this pull request, B is tagged as a reference tag with name kind and name:A as scope. How can we represent the relationship between X and B?

OK, now I remembered this thing better. I was wrong: std:: should not be used as the scope of std::cout.

The name kind denotes a name being imported in a certain namespace. For consistency with other kinds the scope field should be the destination namespace, not the source one: it's where the symbol should be attached, not where it is imported from. So in your example above the name being imported is really A::B and the scope is namespace X.

... I'm thinking about a new field "context:" or "where:".

X       .../;"     kind:namespace  roles:def       end:3 
B       ...;"    kind:name       scope:name:A    roles:used       context:namespace:X     extras:reference

Exactly. But scope and context contents should be reversed. And for clarity context could be named source.

X       .../;"     kind:namespace  roles:def       end:3 
B       ...;"    kind:name       scope:namespace:X    roles:used       source:A     extras:reference

The difference between the name and the using kind is simply that in the former case a single symbol is imported while in the latter all the symbols included in a namespace are imported. Currently the using kind tags the fully qualified imported symbol and in the source code there is a note about a "nameref" field (which is exactly what you have called context here). The current behavior isn't that bad either, it could be replicated to name:

X       .../;"     kind:namespace  roles:def       end:3 
A::B       ...;"    kind:name       scope:namespace:X    roles:used     extras:reference

This question is not only about namespace. Most of all reference tags are relevant. ...

void foo(void) { }
void bar(void)
{
    foo();
}

Yes, they are similar in nature. However I think that the using statement is far more important than a simple function reference. using and name attach a section of the symbol tree to the current namespace and are essential for an editor/client to be able to fully resolve the symbols used in a file. While a simple function call/reference serves no other purpose and it's presence is non critical.

[masatake]

Thank you. The source and destination are keywords that I'm looking for.

About B you show two ideas:

B       ...;"    kind:name       scope:namespace:X    roles:used       source:A     extras:reference

or

A::B       ...;"    kind:name       scope:namespace:X    roles:used     extras:reference

The latter one, A::B looks source scope version of fully qualified tag. In the former one, A::B can be synthesized from the tag name B and the source field A. So the former one is more fundamental (basic).

About nameref, I don't convince I understand your idea or not. If we use the nameref field, the tag for B will be:

B       ...;"    kind:name       scope:namespace:X    roles:used       source:A     nameref:A::B extras:reference

Am I correct?

I would like to implement the source field as you suggested . Before jumping into the implementation task, I would like to revise the way for tagging an import statement of Python. https://docs.ctags.io/en/latest/man/ctags-lang-python.7.html When I designed this, I have the critical concept source and destination scopes. So I used scope where I should use source. To introduce the source field for representing the source scope, I have to change the python parser. It breaks what I wrote on the man page. So...we should change the version number from 5.9.x to 5.10.x.

[pragmaware]

About B you show two ideas:

[...]

Yes, both solutions can work. Your call.

About nameref, I don't convince I understand your idea or not.

Ignore it. It was my first idea of what was needed, long time ago. Now it's really source.

I would like to implement the source field as you suggested .

Great!