`docs.rs` cannot build `datafusion-proto` crate

[andygrove]

Describe the bug

[INFO] [stderr]    Compiling datafusion-proto v12.0.0 (/opt/rustwide/workdir)
[INFO] [stderr] error: failed to run custom build command for `datafusion-proto v12.0.0 (/opt/rustwide/workdir)`
[INFO] [stderr] 
[INFO] [stderr] Caused by:
[INFO] [stderr]   process didn't exit successfully: `/opt/rustwide/target/debug/build/datafusion-proto-4369f54fa6f25371/build-script-build` (exit status: 1)
[INFO] [stderr]   --- stdout
[INFO] [stderr]   cargo:rerun-if-env-changed=FORCE_REBUILD
[INFO] [stderr]   cargo:rerun-if-changed=proto/datafusion.proto
[INFO] [stderr]   Running: "/usr/bin/protoc" "--include_imports" "--include_source_info" "-o" "/tmp/prost-buildQPnxTC/prost-descriptor-set" "-I" "proto" "proto/datafusion.proto"
[INFO] [stderr] 
[INFO] [stderr]   --- stderr
[INFO] [stderr]   Error: "protobuf compilation failed: Read-only file system (os error 30)"

To Reproduce Steps to reproduce the behavior:

Expected behavior A clear and concise description of what you expected to happen.

Additional context Add any other context about the problem here.

[andygrove]

@avantgardnerio @alamb This is one side-effect of the recent change to generate protobuf into the src directory. I don't have a strong opinion on how much of an issue this is, although it would be nice to have docs.

[avantgardnerio]

@andygrove do you have a link to the build in question? The error message isn't sufficient in and of itself, but I feel like this is fixable.

[andygrove]

@andygrove do you have a link to the build in question? The error message isn't sufficient in and of itself, but I feel like this is fixable.

https://docs.rs/crate/datafusion-proto/12.0.0

[avantgardnerio]

Got it, so https://docs.rs/crate/datafusion-proto/12.0.0/builds/630103

[avantgardnerio]

This looks broken because the source directory is closed to a readonly file system https://docs.rs/about/builds#read-only-directories

But should be fixable using conditional compilation flags https://docs.rs/about/builds#cross-compiling

[tustvold]

Docs builds are still failing

[avantgardnerio]

Docs builds are still failing

Can you explain what you mean please? I see the results and the build log contains no errors (only warnings).

[alamb]

Perhaps https://docs.rs/crate/datafusion-proto/13.0.0

With https://docs.rs/crate/datafusion-proto/13.0.0/builds/646416 failure:

[alamb]

BTW the fact that generated code definition jump to works in datafusion-proto is awesome!

[avantgardnerio]

With https://docs.rs/crate/datafusion-proto/13.0.0/builds/646416 failure:

For the record:

[INFO] [stderr]   Running: "/usr/bin/protoc" "--include_imports" "--include_source_info" "-o" "/opt/rustwide/target/x86_64-unknown-linux-gnu/debug/build/datafusion-proto-7d222b4b383775ea/out/proto_descriptor.bin" "-I" "proto" "proto/datafusion.proto"
[INFO] [stderr] 
[INFO] [stderr]   --- stderr
[INFO] [stderr]   thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 30, kind: ReadOnlyFilesystem, message: "Read-only file system" }', build.rs:76:10

It looks like my previous fix got the main build to work on docs.rs, but specifically not the datafusion-proto crate. And unfortunately docs.rs will only let us generate into the OUT_DIR.

I see 3 possible solutions:

1. generate into OUT_DIR, but only for docs.rs (getting complicated and painful here)
2. we always generate to OUT_DIR and as a CLion user, I enable the experimental feature that make it work. vscode users aren't supported
3. We merge https://github.com/apache/arrow-datafusion/pull/3950

at this point I'm fine with the 3rd option. I just want to keep momentum going.