Rahul Penti

Docker on M1 leverages aarch64 which is causing the failure. The docker build code currently is based on x86_64 architecture. Support for x86_64 in Mac is provided using Rosetta. We...

This is resolved with the recent change of installing openssl instead of building it from source. It handles the above architecture discrepancy

Sorry for the delay. I think these changes are redundant given the recent changes in the docker build steps (specifically installation of openssl through apt-get which ensures the right architecture)

@andygrove Can you share the contents of `python/target/wheels/` which should ideally have the python package whl. If there are no files, can you rebuild ballista-builder Docker

Thats weird. Given you rebuilt the docker, can you manually run this [command](https://github.com/apache/arrow-ballista/blob/master/dev/build-ballista-docker.sh#L24) and share the logs. The `maturin build`[command](https://github.com/apache/arrow-ballista/pull/434/files#diff-615c017da4454ef856f22a705786f27f36b1b44317dc49dab5be52a5fba85f01R35) in the entrypoint script is responsible for building the wheels

@andygrove I was able to do a clean build (docker system prune and git clone to ensure there are no old artifacts) on Intel Mac and M1. Which OS are...

Can I work on this ? Additionally is the expectation here is to add multiple FileScanTask based on the split size and the scan file size