JS generator missing jspb.generate_from_object option

[hochhaus]

trafficstars

The comment for jspb.Message.GENERATE_FROM_OBJECT states:

 *     NOTE: By default no protos actually have a fromObject method. You need to
 *     add the jspb.generate_from_object options to the proto definition to
 *     activate the feature.

However, no such option is currently implemented in the open sourced version. I see the code which generates output for this option (GenerateClassFromObject) but it is invoked 0 times.

~/protobuf$ grep -r GenerateClassFromObject .
./src/google/protobuf/compiler/js/js_generator.cc:void Generator::GenerateClassFromObject(const GeneratorOptions& options,
./src/google/protobuf/compiler/js/js_generator.h:  void GenerateClassFromObject(const GeneratorOptions& options,

Can the jspb.generate_from_object option be open sourced?

[hochhaus]

Also, the jspb.generate_xid option appears to be missing as well.

/**
 * The xid of this proto type (The same for all instances of a proto). Provides
 * a way to identify a proto by stable obfuscated name.
 * @see {xid}.
 * Available if {@link jspb.generate_xid} is added as a Message option to
 * a protocol buffer.
 * @const {!xid.String|undefined} The xid or undefined if message is
 *     annotated to generate the xid.
 */
jspb.Message.prototype.messageXid;

[hochhaus]

@xfxyjwf Any chance this bug could be addressed prior to the beta 4 release? It is a large issue for certain use cases of the JS implementation.

[pcj]

+1

[hochhaus]

Adding @haberman as he was able to help with a few other JS issues.

Any chance this could be looked into?

[haberman]

Many of the options we have internally are legacy/deprecated, so we haven't open-sourced them. The ones that we want to keep we will likely want to integrate into descriptor.proto.

Could you say more about how you want to use toObject() and fromObject()? I ask because this format is not standardized, and it is similar to (but not the same as) proto3 JSON.

Also how do you want to use xids?

[hochhaus]

Thanks Joshua. All I need is proto3 JSON support (I was not aware of the different toObject() JSON format). Also I have no current need for xid, I was only trying to be complete.

Could the proto3 JSON support be open sourced?

[haberman]

We don't currently have proto3 JSON support implemented for JavaScript (even internally). Inside Google most JavaScript projects use a wire format called JSPB that is specific to JavaScript and encodes protobufs as arrays indexed by field number like [123, "abc", true]. This format has code-size benefits over binary or proto3 JSON formats (basically it can be parsed with JSON.parse() without the need for any generated parsing code). But the JSPB format isn't very standardized (even inside Google) and changes a lot so we haven't open-sourced it.

For JavaScript we've implemented binary support which works across proto2/proto3 and can interoperate with other implementations. proto3 JSON support would be possible to implement, but we haven't done it yet.

[hochhaus]

@haberman Thanks for the additional context.

For what it is worth, the JSPB format that you mentioned looks very similar to the goog.proto2 PBLite format from closure-library. I've been using this for some time as part of the protoclosure project.

Given that proto3 JSON isn't implemented would it be possible to include the toObject and fromObject JSPB support?

[haberman]

For what it is worth, the JSPB format that you mentioned looks very similar to the goog.proto2 PBLite format from closure-library.

Indeed -- that is an example of a format that is similar but not the same as JSPB. I didn't realize that had been open-sourced.

[pcj]

I was under the impression from https://groups.google.com/d/msg/closure-library-discuss/oRZIfaoFGRQ/0FZpl-RIAgAJ that someone is working on grpc support for javascript that may or may not be supported by closure library goog.net.rpc.*. Any insight on JSON support with those efforts?

[haberman]

@stanley-cheung do you have any insight on @pcj's question?

[stanley-cheung]

I am not sure about the full context here, but yes we are working on a javascript client library that will use JS protobuf and the Closure library to support making RPC calls from browsers to gRPC backend via a gRPC gateway. It is still early in that project so nothing concrete is released yet. The current estimate is late Q3 the earliest, probably Q4.

[pcj]

@stanley-cheung Nice. Keep me posted, I may be interested in helping with that.

[seishun]

@haberman

Could you say more about how you want to use toObject() and fromObject()?

Here's my use case. I maintain a library that interoperates with a network that uses protobufs (proto2) extensively. Since I don't want to tie my library's API to a specific protobuf implementation, its methods accept protobufs as plain objects - same goes for events that provide protobufs (more detailed description here).

There's also the issue that I don't convert to camelCase, but toObject in this implementation does and there's no way around it. I was hoping to address this after protocolbuffers/protobuf#1922 is resolved.

[alexeykomov]

Hello, all, I'm interested in this issue being resolved as well.

Answering @haberman on question in https://github.com/google/protobuf/issues/1591#issuecomment-237001956. My use case is to use toObject and fromObject in order to use JSON.stringify and JSON.parse with them correspondingly, in order to transmit human readable JSON over the wire.

I see that at least fromObject being missing could be resolved by adding GenerateClassFromObject(options, printer, desc); after this line https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/js/js_generator.cc#L1841.

Is there any dangers with this solution?

[wiktortomczak]

What's the status of this? Are there plans to include fromObject method in the generated JS proto message classes?

My use case is same as @alexeykomov's. Also, Alexey's solution just works. See wiktortomczak/protobuf@68ad1c80cefc93e17f26b3a337b253102bbece27.

[someValue]

+1

[jonnyreeves]

I would also like to make use of fromObject; our use-case is to incorporate it with https://github.com/improbable-eng/ts-protoc-gen and provide a less verbose API for constructing Protobuf objects for transmission via https://github.com/improbable-eng/grpc-web

I'm happy to raise a PR for this if it will be considered by the grpc maintainers.

[shanshanzhu]

What is the status of this one? We would like to have fromObject exposed as well.

[anandolee]

@haberman any updates?

[hartmut-co-uk]

having the same requirement as @jonnyreeves...

less verbose API for constructing Protobuf objects

[metled]

+1, any updates?

[met-pub]

hope fromObject() generated.

[amgadani]

Pinging this ticket again, any update ?

[parano]

Tried the following workaround and it seems to work correctly. Although I haven't tested it extensively: https://github.com/protocolbuffers/protobuf/compare/master...parano:js-fromobject-hack?expand=1

First add the following line to js_generator.cc:

diff --git a/src/google/protobuf/compiler/js/js_generator.cc b/src/google/protobuf/compiler/js/js_generator.cc
index 5779d5e2..439893b4 100644
--- a/src/google/protobuf/compiler/js/js_generator.cc
+++ b/src/google/protobuf/compiler/js/js_generator.cc
@@ -1910,6 +1910,7 @@ void Generator::GenerateClass(const GeneratorOptions& options,


     GenerateClassToObject(options, printer, desc);
+    GenerateClassFromObject(options, printer, desc);
     // These must come *before* the extension-field info generation in
     // GenerateClassRegistration so that references to the binary
     // serialization/deserialization functions may be placed in the extension

The Generator::GenerateClass method now looks like:

void Generator::GenerateClass(const GeneratorOptions& options,
                              io::Printer* printer,
                              const Descriptor* desc) const {
  ...
  if (!NamespaceOnly(desc)) {
    printer->Print("\n");
    GenerateClassConstructor(options, printer, desc);
    GenerateClassFieldInfo(options, printer, desc);

    GenerateClassToObject(options, printer, desc);

    GenerateClassFromObject(options, printer, desc); // ADD THIS LINE

    // These must come *before* the extension-field info generation in
    // GenerateClassRegistration so that references to the binary
    // serialization/deserialization functions may be placed in the extension
    // objects.
    GenerateClassDeserializeBinary(options, printer, desc);
    GenerateClassSerializeBinary(options, printer, desc);
  }
  ...
}

Rebuild protoc following instructions here, now the protoc generated js code should include fromObject method. Although it is missing goog.isDef and goog.array method. A quick hack, I added the following to all the generated file:

goog.isDef = function(a) {
  if (a === undefined || a === null) {
    return false;
  }
  return true;
}

goog.array = {
  map: function(arr, func) {
    return arr.map(func);
  }
}

It works for most simple cases. The only problem I still have is the generated fileld name are not what I'd expected. For example, for field repeated myObjects = 1, the json object field name will be myobjectsList.

Looks like I need the Use proto field name instead of lowerCamelCase name option that is documented here: https://developers.google.com/protocol-buffers/docs/proto3#json_options Which I believe is not yet implemented in js.

[sophiawisdom]

@TeBoring any thoughts/plans on this?

[thetimetraveler]

any updates on this?

[elvizlai]

LOL? a 2016 issue still not solved.

[BichengWang]

we need fromObject.......

[4nte]

.fromObject would be neat, +1