[feature] Custom attributes as metadata in the the output language [Java (but relevant to all languages)]

[ofirm93]

As of today, besides of specific group of attributes, any other attribute that is mentioned in the schema has no effect on the output classes. I think that it can be represented in the class as a Java annotation, as it allows it to be mentioned in the code in non obtrusive way. In other languages there are equivalent mechanisms so it can be reflected in other languages too.

For example, for this fbs:

namespace com.example;

attribute "Ref";

table Example {
  field:string (Ref: "example.field");
}

root_type Example;

The generated code will include an annotation with the info:

public final class Example extends Table {
  ...
  @Ref("example.field")
  public String field() { int o = __offset(4); return o != 0 ? __string(o + bb_pos) : null; }
  ...
}

[aardappel]

What if people add a custom attribute somewhere that wasn't intended for use with Java? Is Java ok with any annotation name? Or should we somehow indicate that an attribute is intended for annotation (with a particular language)?

See, we could even use attributes on attribute annotations! :P

attribute "Ref" (java);

[ofirm93]

What if people add a custom attribute somewhere that wasn't intended for use with Java? Is Java ok with any annotation name? Or should we somehow indicate that an attribute is intended for annotation (with a particular language)?

See, we could even use attributes on attribute annotations! :P

attribute "Ref" (java);

At least for java, if you add an anotation and do not intentionally read it, it has no effect on the code execution.

Usually, another piece of code reads the annotation and use either in compile time or runtime.

So java annotations are metadata just like flatbuffers attributes.

Regarding naming, we can take the general approach in flatbuffers and convert the name to a language specific convention.

[github-actions[bot]]

This issue is stale because it has been open 6 months with no activity. Please comment or this will be closed in 14 days.

[tomCohen123]

Hi :)
I am working with Ofir, and what we actually want to do is allow user to add annotation to a java class with a variable package inside:

Example: (added just the important lines of code in the example):

For this example.fbs:

attribute hello (java_package: "x"); table A { field_a:string (hello: "World");

} root_type A;

We will get this AT.java:

public class A { @Hello("World") private string field_a; }

And this Hello.java (annotation file):

package x;

public @interface Hello { String value() default ""; }

To implement, the Functionality we thinking to do the following:

1. change the attribute decl grammar to: attribute_decl = attribute ident | "ident" metadata ;
2. add java package as predefined attribute
3. change the idl_gen_java.cpp such that if user define attribute contains the java_package predefined attribute (like in example.fbs ) the last 2 files will be created.
4. Add this Functionality to field (top priority), table, enum, struct.

@aardappel I would be happy to get feedback from you.

Thanks :)

[aardappel]

That generally seems reasonable, though I am not sure if we want to be in the business of generating Java attribute definition files?

@paulovap opinion?

[paulovap]

I am not sure it would be a good feature, because I don't see how this functionality connects directly with flatbuffers per se. Looks more like piggybacking on flatc to add more functionality to the generated classes. I guess this is something that could be accomplished by other tooling

It makes more sense to me, if we really want to customize code generation, to add a more generic approach, where one could intercept every "write event", allowing user to append or prepend what they want, like an AST visitor. That would be beneficial for all languages and leave the complexity and problems for the user to handle.

[ofirm93]

@paulovap On one hand, I do agree that this not the most cross-language implementation, but I think that this is the most appropriate way to pass the metadata Flatbuffers attributes contain to the generated code. FlatBuffers allows user-defined attribute while practically they do not have any effect on the generated code, and Java Annotations is the most similar java definition both syntacticly and semantically. Let's say I want to define a new attribute "max" which defines the max value of an integer. Passing it on as Java annotation is almost natural to make this functionality without requiring any functionality to be tied to Flatbuffers.

[ofirm93]

@paulovap May I ask your response on that thread. Also, I want to ask what is the benefit of having user defined in FlatBuffers as it doesn't have any effect right now on the generated code.

[paulovap]

@paulovap May I ask your response on that thread. Also, I want to ask what is the benefit of having user defined in FlatBuffers as it doesn't have any effect right now on the generated code.

Since we don't see a language neutral solution in the horizon, I guess we can introduce this feature for java specific as long as it's not very intrusive. @aardappel has the final saying.

[aardappel]

I would agree with @paulovap that this seems very niche / language specific .. so yeah, depends on how intrusive it would be.

[dbaileychess]

@ofirm93 Plans to implement this with a PR? Otherwise I will close.

[github-actions[bot]]

This issue is stale because it has been open 6 months with no activity. Please comment or label not-stale, or this will be closed in 14 days.

[github-actions[bot]]

This issue was automatically closed due to no activity for 6 months plus the 14 day notice period.

[ofirm93]

Let me update regarding this feature. We forked the code somewhere after 2.0, and we use this feature regularly. Let me give some example for usages of this feature:

1. We used attributes to annotate the Object API classes with existing Java Annotations. That way we got some validations out of the box for free (like min, max value etc...). Flatbuffers became the codec which we use to encode and decode messages over multiple transports. It was very easy to integrate after we add the previous feature of ObjectAPI to java, and the additional annotations made the code be very Java native. Developers don't need to bother about the underlying encoding as they are only aware of the Java Object API classes which look by all means like regular classes.
2. Additional custom attributes allows us to use reflection libraries in java to analyze the code in both compile time and runtime, and implement similar mechanism to the AST visitor @paulovap mentioned. This made flatbuffers our source-of-truth to all information that flows in the system.

I think that this one is crucial feature for java developers, yet I think that the investment in rebasing our code on top of current main branch is substantial. We have plans to contribute that at some point if there will be enough interest. I think this can allow people to use flatbuffers as codec in frameworks like springboot/quarkus pretty easily.

[paulovap]

@ofirm93 would it work for you if one adds the ability to generate annotations from the schema? Example:

table A (annotate: "@MyAnnotation(\"hello\")" {
field_a:string (annotate: "@Max(10)");

}
root_type A;

This would generate something like: AT.java

@MyAnnotation("hello")
public class A {
    @Max(10)
    private string field_a;
}

That way we add a new attribute that flatc understands, other languages can implement if makes sense, we avoid changing the grammar, avoid adding specific use-cases to the java generator (like this package specification mentioned before). With that you can add an extra step on you build toolchain after flatc generation to parse the generated files, read the annotations and do whatever you want with them.

2. Additional custom attributes allows us to use reflection libraries in java to analyze the code in both compile time and runtime, and implement similar mechanism to the AST visitor @paulovap mentioned.

Out of curiosity, what kind of reflection libraries are you using? In what step of the build (I am assuming you are doing additional code generation at compile time)?

[ofirm93]

@paulovap I don't mind doing that, yet I think this is too permissive. Different languages probably need different annotations so it's not the most cross-language feature. If the problem is related to the metadata we need in Java to select the package I can think of many ways to do that without changing the grammer:

1. We can pass that as the parameter for 'flatc'.
2. We can use the namespace as the package, although last time I suggested that I understood the namespace do not apply to flatbuffers attributes which are considered non-names paced. Is there a reason for that? I didn't see any such hard limitation on the docs or in the code, but maybe you can shed the light on that.

Regarding your question about how we use it, currently we just use plain Java reflection with some utilities of spring to read the annotations. We read them both in compilation as part of the build process to generate other classes and docs. And also use them in runtime to make decisions about the validity of the received values.

[github-actions[bot]]

This issue is stale because it has been open 6 months with no activity. Please comment or label not-stale, or this will be closed in 14 days.

[github-actions[bot]]

This issue was automatically closed due to no activity for 6 months plus the 14 day notice period.