- 7 minutes to read

Release Notes

Monitoring Content Files

Technical guide to content-based pattern detection monitoring—configure RegEx patterns, negative filtering, threshold evaluation, and alert states for text-based log files.

What is Content-Based Monitoring?

Content-based monitoring evaluates each log line independently using regular expression pattern matching. When a line matches the configured pattern (and does not match negative exclusion patterns), the Log File Parser Monitoring Agent records a detection event and evaluates threshold rules to determine resource state.

Pattern Detection Flow

flowchart TD Start([File Scanning<br/>Every 60 seconds]) --> FileFilter{Apply Time<br/>Options Filter} FileFilter -->|Modified/Created<br/>After Clear Date| IncludeFiles[Select Matching Files] FileFilter -->|Evaluate All| IncludeFiles IncludeFiles --> FileLoop{More Files<br/>to Scan?} FileLoop -->|Yes| ReadNew[Read New Log Lines<br/>from File] FileLoop -->|No| CheckState{Any Matches<br/>Found?} ReadNew --> LineTest{Match Positive<br/>Pattern?} LineTest -->|No Match| NextLine[Continue to Next Line] LineTest -->|Match| NegCheck{Match Negative<br/>Pattern?} NegCheck -->|Yes - Exclude| NextLine NegCheck -->|No - Include| ErrorState([State: Error<br/>Alert Triggered]) NextLine --> MoreLines{More Lines<br/>in File?} MoreLines -->|Yes| LineTest MoreLines -->|No| FileLoop ErrorState --> MoreLines CheckState -->|Yes| MaintainError([State: Error<br/>Remains]) CheckState -->|No| OKState([State: OK<br/>No Alert]) MaintainError --> Wait[Wait for Next Scan] OKState --> Wait Wait --> Start style Start fill:#e3f2fd style FileFilter fill:#e1bee7 style IncludeFiles fill:#fff9c4 style ReadNew fill:#fff9c4 style ErrorState fill:#f44336,color:#fff style MaintainError fill:#f44336,color:#fff style OKState fill:#4caf50,color:#fff style NegCheck fill:#e1bee7

Key mechanism: Files are filtered by time options before scanning. Each matched line in any file immediately triggers Error state—spans multiple log files, no counting, no thresholds.

When to Use This Monitoring Type

Use content-based monitoring for independent event detection scenarios:

  • Error detection - Match "ERROR", "EXCEPTION", "FATAL" keywords or specific error codes
  • Security events - Detect authentication failures, SQL injection attempts, privilege escalation
  • Performance patterns - Identify slow queries, timeout events, resource exhaustion warnings
  • Business events - Track specific transaction outcomes, user actions, compliance events

For transaction sequences requiring start/end correlation, use Monitoring Correlated Events instead.

Note

For benefits and use cases, see Log File Parser Monitoring Agent. This page focuses on technical monitoring behavior and configuration options.

Pattern Detection Mechanics

Evaluation Flow

  1. File scanning - Agent reads new log file content at configured interval (default: 60 seconds)
  2. Line-by-line matching - Each line tested against positive pattern RegEx
  3. Negative pattern filtering - Matching lines tested against negative pattern RegEx (if configured)
  4. Threshold evaluation - Detection count compared to warning/error thresholds
  5. State determination - Resource state updated (OK, Warning, Error) based on threshold rules
  6. Alert generation - State changes trigger notifications via Monitor Views and Alarm Plugins

Pattern Matching Rules

  • Positive patterns - RegEx expressions that identify events of interest
  • Negative patterns (optional) - RegEx expressions to exclude from positive matches (reduces false positives)
  • Case sensitivity - Configurable per pattern
  • Multi-line matching - Supported for stack traces and XML/JSON content

Example configuration with separate negative pattern:

Positive Pattern: ERROR|EXCEPTION|FATAL
Negative Pattern: ERROR: Retry attempt \d+/\d+ successful|Connection timeout retrying
Result: Alerts on errors but ignores expected retry messages

Alternative: Negative lookahead within positive pattern:

Positive Pattern: ERROR(?!.*(retry|retrying|successful retry|transient))
Negative Pattern: (leave empty)
Result: Matches "ERROR" only if the line does NOT contain retry-related keywords

Use negative lookahead when you want a single self-contained pattern, or use separate negative patterns for better maintainability with complex exclusion rules.

File Selection Criteria

  • Wildcard patterns - app*.log, error_\d{8}.log for rotating files
  • Time-based filtering - Control which files are included in monitoring based on file timestamps:
    • Modified/Created after Clear Date Time - Monitor only files modified or created after specified timestamp
    • Modified/Created after File time span - Monitor files within relative time window (e.g., last 24 hours)
    • Last Modified/Last Created, one file only - Monitor only the most recently modified or created file
    • Evaluate all - Monitor all matching files regardless of timestamp
  • Recursive scanning - Include subdirectories (optional)
  • Network paths - UNC paths (\\server\share\logs), SMB shares, local folders

Time option use cases:

  • Active logs only - Use "Modified after Clear Date Time" to ignore archived logs
  • Recent errors - Use "Modified after File time span" for rolling 24-hour monitoring
  • Single current file - Use "Last Modified, one file only" when log rotates hourly but you only care about active file

Threshold Configuration

Define when detected patterns trigger Warning vs. Error states:

  • Match count thresholds - Trigger Warning after X matches, Error after Y matches
  • Time window - Evaluate matches within sliding time window (e.g., 5 matches in 10 minutes)
  • State persistence - State remains until cleared via Clear Errors remote action

Content File Monitoring Overview
Example: Content-based log file configurations in a Monitor View

Monitor Configuration Options

Nodinite presents each Content File Configuration as a Resource, evaluated in real-time. You can:

  • Organize by purpose - Group configurations by Application (Web Server, Database, Payment Processing)
  • Categorize by type - All content configurations appear under 'Log File Event' Category
  • Scale monitoring - Deploy multiple agents across servers for distributed monitoring
  • Apply different rules - Use multiple configurations for the same files with different patterns

Content File Categories
Example: Content File Configurations organized by Category

Configuration Setup

  • Display name comes from your Content File Configuration settings
  • Application association links configurations to business applications
  • Pattern definitions specify what content triggers alerts
  • Time options control which files are included in monitoring

Resource State Evaluation

Each Content File Configuration (Resource) maintains one of these states based on threshold evaluation:

State Condition Description
Unavailable File access failed Agent cannot read files (network issue, permissions, path not found)
Error Threshold exceeded Pattern matches exceed error threshold (e.g., 10+ matches)
Warning Threshold exceeded Pattern matches exceed warning threshold (e.g., 3+ matches)
OK Below threshold Pattern matches below warning threshold or no matches detected

Resource States
Example: Content file resource showing current monitoring state

Override default state evaluation using the Expected State feature (e.g., expect Warning state during maintenance windows).

Remote Actions for Detected Patterns

When patterns are detected or monitoring requires adjustment, perform these Remote Actions from the Web Client:
Remote Actions
Example: Remote Actions available Content File Configurations.

  • Edit Thresholds - Adjust warning/error thresholds and pattern definitions without agent restart
  • Error Report - View matched log entries with file context and download detected content
  • Clear Errors - Reset resource state after investigating and resolving detected issues
  • Show Log Files - Display monitored files, download log files, manage file-level settings

All actions require appropriate Role permissions and are logged to the Audit Trail.

Tip

See Managing Content Files for complete action details, including confirmation dialogs, success criteria, and troubleshooting.

Performance Characteristics

Scan Interval Configuration

  • Default interval: 60 seconds
  • Configurable range: 10 seconds to 24 hours
  • Trade-off: Lower interval = faster detection, higher CPU usage

File Position Tracking

  • Persistent state - Agent remembers last read position per file
  • Restart resilience - Resumes from last position after agent restart
  • Rotation handling - Detects file rotation (size decrease) and starts from beginning of new file
  • Incremental reading - Only processes new content since last scan

Resource Usage

  • CPU: <5% during active monitoring (200 files)
  • Memory: ~100 MB per agent (200 files tracked)
  • Network: Minimal (read-only file access, no log transfer to Core Services)
  • Disk I/O: Sequential reads only, leverages OS file cache

Supported File Formats

Any text-based format where log entries are line-delimited or follow parsable structure:

  • Plain text - .log, .txt, custom extensions
  • Structured logs - CSV, TSV, tab-delimited (match specific columns)
  • JSON logs - One JSON object per line (newline-delimited JSON)
  • XML logs - Multi-line XML matching with RegEx
  • Web server - IIS W3C, Apache Combined/Common, Nginx, custom formats

Timestamp parsing - Configurable for custom date/time formats

Binary formats not supported - Windows .evtx, database proprietary formats require specialized agents

Next Steps

Configuration and setup:

Configuration - Content File - Complete setup guide with pattern syntax and threshold rules
Prerequisites - File permissions, network access, agent requirements
Install Agent - Agent installation and Core Services registration

Management and operations:

Managing Content Files - Remote actions for detected patterns (Edit, Error Report, Clear, Show Files)
Monitoring Correlated Events - Alternative monitoring type for transaction sequences
Monitoring Nodinite Diagostics - Monitor Nodinite diagnostics logs for issues

Implementation examples:

Monitor IIS Logs - HTTP errors and performance patterns
Monitor File Content - General pattern detection examples

Monitoring Correlated Events - Track transaction sequences and ACK/NAK patterns
Managing Content Files - Remote actions and alert management
Configuration - Content File - Complete setup guide
Troubleshooting Content Monitoring - Common issues and solutions