Need an API to Unescape Characters in Identifiers

[ThisaruGuruge]

Description: When the Semantic API returns values of identifiers, the escape characters are not unescaped. For example:

import ballerina/graphql;

service on new graphql:Listener(4000) {
    resource function get name(string 'version) returns string {
        return 'version;
    }
}

In the above code, the parameter name of the resource method is returned as 'version from the getName() API. There should be a way to remove the leading single quote and also unescape the escaped characters.

There is an existing API for this, but it is not exposed to external users. It is better to expose this, as some compiler plugins need this functionality.

[lochana-chathura]

@dulajdilshan is getName() returning 'version, the expected behavior? Don't we expose both the original value and the unescaped value? if we expose unescaped value don't we remove the quote?

[pubudu91]

+1 for adding a public API for this. Currently there's no way to compare identifiers when it's a mix of escaped and unescaped identifiers.

[pubudu91]

@dulajdilshan is getName() returning 'version, the expected behavior? Don't we expose both the original value and the unescaped value? if we expose unescaped value don't we remove the quote?

This is the expected behaviour. The behaviour in the Semantic API is that we return the names exactly as they were written in the source code. e.g., if the user wrote 'name in the code, getName() will also return 'name.

[lochana-chathura]

I remember keeping both values internally. Was wondering if both of them were exposed via separete methods.

[dulajdilshan]

I remember keeping both values internally. Was wondering if both of them were exposed via separete methods.

Yeah. Internally we keep both of the values. But we only expose the name that the user has defined. I'm also +1 to have another API to get the unescaped string value of the name

[hasithaa]

There are different variations when it comes to escaping and unescaping, We can't expose all these variations as part of the syntax or semantic API. We have this identifier-util which is used by many other components including runtime. I guess the best option is to expose this Util as a proper API.

[lochana-chathura]

I'm not sure to which team this change belongs. Most of the methods are written by the runtime team.

We have this identifier-util which is used by many other components including runtime. I guess the best option is to expose this Util as a proper API.

looping @HindujaB @warunalakshitha. would we be able to expose this as a proper API?

[ThisaruGuruge]

I'm not sure to which team this change belongs. Most of the methods are written by the runtime team.

In that case, will the compiler plugins be able to use them in the compiler plugins?

[lochana-chathura]

In that case, will the compiler plugins be able to use them in the compiler plugins?

Yes of course. New API should be visible everywhere.

Currently, it is exported as follows.

exports io.ballerina.identifier to io.ballerina.lang, io.ballerina.runtime, io.ballerina.shell,
            io.ballerina.testerina.runtime, io.ballerina.lang.runtime, io.ballerina.lang.error,
            ballerina.debug.adapter.core, io.ballerina.jsonmapper;

[ThisaruGuruge]

I also noticed that most of the compiler APIs return the escaped value, and some are not. For example,

PathSegment pathSegment = // get path segment
String signature = pathSegment.signature();

In the above code, the signature will return the value without the ' character. The typeSymbol.getName().get() also returns the value without the ' character. But the input parameters do not behave this way.

[lochana-chathura]

In the above code, the signature will return the value without the ' character. The typeSymbol.getName().get() also returns the value without the ' character. But the input parameters do not behave this way.

This seems to be a bug. @dulajdilshan thoughts?

[HindujaB]

New API should be visible everywhere.

The identifier-util module contains the APIs that are for internal usages - like jvm encoding etc. So, it is only exported to specific modules. In this use-case, we will need to export only the escape/unescape APIs to public. The others should be restricted.

[warunalakshitha]

Yes. We should not expose internal encoding/decoding methods as apis. Lets expose unescape method only in Identifier-utils module as a separate api package method.

[pubudu91]

Yes. We should not expose internal encoding/decoding methods as apis. Lets expose unescape method only in Identifier-utils module as a separate api package method.

Need a method for escaping as well. That's needed for cases where we generate Ballerina code.

[ThisaruGuruge]

In the above code, the signature will return the value without the ' character. The typeSymbol.getName().get() also returns the value without the ' character. But the input parameters do not behave this way.

This seems to be a bug. @dulajdilshan thoughts?

Any thoughts on this? We are using these APIs in our packages. What is the bug here? Including the ' character or not including it? Is this fixed now? We are using these APIs in compiler plugins. If the bug is fixed, how should we update our code?

Also, when we do introduce new APIs like this, can we please at least include some examples in the PR? We've mentioned this previously as well. It is really hard to find APIs and their usage without any proper documentation.

[lochana-chathura]

Any thoughts on this? We are using these APIs in our packages. What is the bug here? Including the ' character or not including it? Is this fixed now? We are using these APIs in compiler plugins. If the bug is fixed, how should we update our code?

@dulajdilshan this is regarding the semantic API behavior. Shall we create a separate issue to check this and fix it?

Also, when we do introduce new APIs like this, can we please at least include some examples in the PR? We've mentioned this previously as well. It is really hard to find APIs and their usage without any proper documentation.

+1

[dulajdilshan]

Any thoughts on this? We are using these APIs in our packages. What is the bug here? Including the ' character or not including it? Is this fixed now? We are using these APIs in compiler plugins. If the bug is fixed, how should we update our code? We are working on it right now.

Also, when we do introduce new APIs like this, can we please at least include some examples in the PR? We've mentioned this previously as well. It is really hard to find APIs and their usage without any proper documentation.

Sure, we will include examples in future PRs

[dulajdilshan]

Any thoughts on this? We are using these APIs in our packages. What is the bug here? Including the ' character or not including it? Is this fixed now? We are using these APIs in compiler plugins. If the bug is fixed, how should we update our code?

@dulajdilshan this is regarding the semantic API behavior. Shall we create a separate issue to check this and fix it?

Also, when we do introduce new APIs like this, can we please at least include some examples in the PR? We've mentioned this previously as well. It is really hard to find APIs and their usage without any proper documentation.

+1

Sure I'll create a one