chore!: simplify webhook code

[davidcavazos]

Main changes

⚠️ Note: This PR includes breaking changes, but will make the code and maintenance simpler at the long run.

Terraform

• Simplify terraform to make it easier to maintain
• Simplify required permissions and roles
• Include unique prefix on resources to avoid name clashes with other solutions
• Update terraform package versions
• Update Makefile to generate metadata
• Move BigQuery schema to its own file

Webhook code

• Simplify webhook code
• Simplify webhook tests
• Use Document AI instead of Vision API
• Use Gemini instead of PaLM
• Improve webhook github actions linter and tests
• Add Python lint check with ruff
• Add Python type checking with mypy
• Upgrade Python dependencies
• Remove notebook since it's already covered by the Neos tutorial

TO DO before merging

• [x] [cl/625435521] Update Neos tutorials
• [x] [pricing-calculator] Update pricing
• [ ] [planned] Update documentation
• [x] [same PR] Update architecture diagram
• [x] [cl/626184064] Update solution metadata in Console
• [ ] Update the sic-jss tag

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB