Issue: Compiling Large Scarb Project Never Completes

[raphaelDkhn]

Problem

When attempting to compile a large Scarb project, the process never completes and causes excessive system resource usage.

Steps

1. Access the minimal repository I made here.
2. Navigate to the inference directory.
3. Execute the command scarb build to try compiling the project.

Possible Solution(s)

The interminable compilation time is probably due to the large size of the node_fc1_weight file available here.

This issue is a significant blocker we faced at Giza, affecting numerous builders using our products. We represent neural network weights as Cairo code, leading to immense file sizes.

To address this, we seek a more efficient and scalable approach to represent these weights. Considering that all weights in Giza share a common structure (with only the number of elements varying), a solution I thought involves representing weights directly in Sierra. However, the challenge lies in importing these Sierra weights into Cairo code when we want invoke it (e.g; here). I'm not sure it's even conceivable to import Sierra code into a Cairo file..

I'd like to know what you think and what you suggest for building large Cairo projects. Thank you!

Notes

No response

Version

Scarb v2.5.3