google-cloud-rust icon indicating copy to clipboard operation
google-cloud-rust copied to clipboard

Huge generated files cause issues with tools

Open Dig-Doug opened this issue 6 months ago • 3 comments

Some of the generated files, e.g. src/generated/cloud/aiplatform/v1/src/model.rs here are huge (11MB).

In IntelliJ, the default max size for analysis is 2.5Mb. If you do not increase it, the IDE will complain that it does not know about any of the types defined in these large files. It would also be nice to be able to look at the source on GitHub.

Would you consider breaking up the files into smaller chunks?

Dig-Doug avatar Jun 30 '25 11:06 Dig-Doug

How difficult is it to increase the IntelliJ limit? And if one does increase the limit: what are the downsides?

Would you consider breaking up the files into smaller chunks?

Yes, though we probably would work on core functionality first.

The first idea would be to split the submodules to their own files. But there are only 167 of those in aiplatform, and 900 top-level structs. It would help, but not solve the problem.

A second idea would be to split the serialization and deserialization impls to their own files. That would probably cut the size by a factor of 3, still not enough.

Another alternative is to split alphabetically, yuck. Probably only enabled by a flag? Or auto-enabled for very large files?

coryan avatar Jun 30 '25 12:06 coryan

It's easy to increase the limit. Details are here.

Dig-Doug avatar Jun 30 '25 13:06 Dig-Doug

FWIW, we implemented this change:

A second idea would be to split the serialization and deserialization impls to their own files. That would probably cut the size by a factor of 3, still not enough.

About a dozen files are over the 2.5MiB limit, as expected, aiplatform/v1/src/model.rs is still one of them.

coryan avatar Sep 18 '25 16:09 coryan