brickflow
brickflow copied to clipboard
[FEATURE] Increase task limit for workflows to 200
Is your feature request related to a problem? Please describe. Databricks team have increased the task limit to 200 for react workspace. It would be helpful to have this feature supported in brickflow.
Cloud Information
- [x] AWS
- [ ] Azure
- [ ] GCP
- [ ] Other
Describe the solution you'd like
Update the max_tasks_in_workflow variable to 200
Describe alternatives you've considered
Additional context
Currently, brickflow throws the following error
@shwen-gine can you try to override it in your code as it is mentioned in the comment. Thanks!
Wont't fix this as it is a Databricks limit