procfwk
procfwk copied to clipboard
mrpaulandrew
A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.
Hi, I ran the proc: EXECUTE [procfwk].[ResetExecution] for a failed batch middway of execution. How then do I get it restarted because the proc says it will be ready for...
There are use cases where a pipeline may be dependent on another pipeline within the same stage. For such use cases it would be easier to set a dependent pipeline...
Sometimes it may happen that "Check For Alerts" activity can fail for whatever reason. This will fail the whole pipeline and extend the whole execution by another 10 minutes (current...
Add more monitoring and troubleshooting capabilitites using Azure Workbooks.
Deal with database connection failures ensure metadata is left in good state and ready for a restart.
Hello, Thanks for the great framework! I ran into one issue: when running the "Check running pipeline" task (I am using Synapse pipelines instead of Data Factory), I get an...
**Describe the bug** After downloading and extracting the current zip file (Code => Download Zip) on running DeployProcFwkComponents.ps1 the deployment fails with this message: "Publish-procfwkadf : The term 'New-AdfPublishOption' is...
In data warehousing scenario, a data load batch/session ID would be good to have in the loaded data, in order to preserve history and distinguish between load runs. There is...
Currently, the ForEach activity in the pipeline "03-Child" triggers all worker pipelines in a given stage at once, in parallel. I had a source system (API) which returned me error...
**Describe the bug** When the 04-Infant pipeline sets the pipeline result, if there is a connectivity issue and the store proc call to set the pipeline times out, the PipelineStatus...
Metadata
Owner
Metadata
A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.