Manage Azure Data Factory Monitoring with Nodinite
Take control of your Azure Data Factory monitoring with Nodinite. This guide shows you how your teams securely manage, monitor, and act on Data Factory and Pipelines with Remote Actions, without direct Azure Portal access.
- ✅ Proactively monitor Data Factory and pipelines for issues
- ✅ Enable secure, role-based self-service for business and support teams
- ✅ Access actionable details and remote actions from a single interface
- ✅ Review a pipeline's Run history and trigger a new run without opening Azure
Animated walkthrough: the Data Factory and its pipelines in a Monitor View, a pipeline's Action menu, and History with the Run history widened to 4 weeks. No pipeline ran in that time frame, so the list is empty here; a failed run shows its error in the Message column.
Nodinite lets you delegate management and insights for selected Data Factory resources in Azure. Your support and maintenance teams gain deep insights for root cause analysis, while you minimize risk by limiting direct Azure Portal access. This approach reduces potential attack vectors and safeguards mission-critical services from accidental disruption.
Start with Monitoring Azure Data Factory if you want the collection overview and threshold context before working with the Data Factory and pipeline actions below.
Diagram: High-level overview of how Nodinite monitors and manages Azure Data Factory and Pipelines; each Subscription holds zero or more Data Factories, and each Data Factory zero or more Pipelines.
| Application Management Team | IT Operations | Business |
|---|---|---|
| Empower your AM team to proactively manage Data Factory without disrupting IT operations | Maintain full control and visibility over all actions | Deliver actionable insights and self-service for business solutions built on Data Factory |
Management Features
For Resources in Role-based Monitor Views with Remote Actions privileges, you can:
| Category | Monitoring | Actions | Metrics/Statistics |
|---|---|---|---|
| Data Factory | ✅ Monitoring | Details | - |
| Pipelines | ✅ Monitoring | Details Edit History |
- |
As an Administrator with access to the Configuration of Monitoring Agents, you can:
| Monitoring | Remote Configuration |
|---|---|
| Monitoring Data Factory |
|
| Monitoring Data Factory Pipelines |
|
Example: Nodinite Monitor View with Data Factory resources.
Data Factory
The 'Data Factory' Category provides one Resource for each Data Factory found using the configuration, with the specified display name as the Resource name.
Example: Monitor View with a list of 'Data Factory'.
The Data Factory category provides Resources that display the evaluated monitored state according to built-in rules.
You can perform the following Remote Actions for the Data Factory Category:
Example: Remote Actions for Data Factory.
Data Factory Details
To view details about the selected Data Factory Resource, click the Action button and select Details from the 'Control Center' menu.
Use the 'Details' action to open the details modal for the selected Data Factory.
Next, select the option to present the modal.
Example: Details for the selected Data Factory.
Nodinite lists the Pipelines of the selected Data Factory in the Pipelines section at the bottom of the modal, with each pipeline's Description, Parameters Count, Activities Count, and Concurrency.
Pipelines
The 'Data Factory Pipeline' Category provides one Resource for each pipeline per Data Factory found using the configuration, with the deployed name as the Resource name.
Example: Monitor View with a list of 'Data Factory Pipelines'.
The Data Factory Pipeline category provides Resources that display the evaluated monitored state according to built-in rules. Review the Monitoring Data Factory user guide for details.
You can perform the following Remote Actions for the Data Factory Pipeline Category:
Example: Data Factory Pipeline Remote Actions.
Pipeline Details
To view details about the selected Data Factory Pipeline Resource, click the Action button and select Details from the 'Control Center' menu.
Use the 'Details' action to open the details modal for the selected Data Factory Pipeline.
Next, select the option to present the modal.
Example: Details about the selected Data Factory Pipeline, with Ignore errors before and the collapsed Parameters, Variables, Annotations, and Evaluation thresholds sections.
Edit Pipeline thresholds
Edit the monitoring thresholds by clicking the Action button and selecting the Edit menu item within the 'Control Center' section.
The Edit menu item opens the Edit modal with the pipeline's thresholds.
Next, select the option to present the modal.
Example of the Edit modal for the selected 'Data Factory Pipeline', with Use global thresholds checked. The note below Save says that after Save, monitoring this pipeline uses specific settings instead of the global ones.
You can manage the following monitoring properties:
- Description – The user-friendly description of this specific Data Factory Pipeline monitoring configuration
- Last clear date time (UTC) – Ignore reported monitoring errors before this point in time
- Use global thresholds – When checked, monitoring thresholds use the global settings from the Remote Configuration; uncheck it to set the thresholds below for this pipeline only
- Enable Run Duration Tolerance – Lets runs occasionally exceed the duration threshold without turning Warning or Error; the duration evaluation applies only when the number of runs over the threshold exceeds the configured acceptable limit
- Lookback period – How far back in time to look for problems. Monitoring evaluation uses this value if the current time is higher than the last clear date-time + the lookback period
- Min Execution count – Enter the monitoring thresholds for the count-based evaluation. -1 means that the check is disabled (default)
- ⚠️ Min execution count (Warning)
- ❌ Min execution count (Error)
- Max Execution count – Enter the monitoring thresholds for the count-based evaluation. -1 means that the check is disabled (default)
- ⚠️ Max execution count (Warning)
- ❌ Max execution count (Error)
- Duration (ms) – Set the threshold for the maximum duration
- ⚠️ Warning
- ❌ Error
Click the Save button to persist changes. After you save, Nodinite evaluates this pipeline with the specific settings instead of the global settings.
Save button.
Warning and Error states reach your team through Alarm Plugins. To override the evaluated state of a single pipeline, use the Expected State option of the Resource.
Pipeline History
To review the runs of the selected Data Factory Pipeline Resource, click the Action button and select History from the 'Control Center' menu. The History modal shows the pipeline's details and its Run history.
Example of the History modal with the time frame set to 4 weeks. This pipeline didn't run in that time frame, so the list is empty.
The Run history toolbar has:
- Time frame – From 5 minutes up to 4 weeks
- Sync – Reloads the runs from Azure
- State filter – OK, Error, or All
- Free-text filter – Narrows the list to matching runs
- Trigger new run – Starts a new run of the pipeline in Azure
The list shows each run's Run start, Run end, Duration, Invoked by, Run ID, Message, and Last updated. For a failed run, Message holds the error from Azure.
Warning
Trigger new run runs the real pipeline in Azure, with all its side effects. Give the Remote Actions privilege in the Monitor View only to Roles that may do that.
Next Steps
- Monitoring Data Factory
- Configuring the Nodinite Azure Monitoring agent
- Azure Logging and Monitoring Overview