How to use the Search Field Wizard

The Search Field Wizard is Nodinite's context-aware, quick-setup tool for creating Search Fields directly from Log Views—without leaving the logs you're analyzing. This guide shows you how to use the wizard to extract business values from messages, test expressions with real data, and deploy new search capabilities in minutes.

What you'll achieve:

• Create Search Fields in minutes without switching to the Administration section
• Test expressions with real message data from your current Log View
• Auto-detect formats and suggest extraction methods (XML, JSON, CSV, etc.)
• Enable self-service for business users without administrator access
• Apply retroactively by re-indexing after wizard-based creation
• Configure Search Field Links to connect to external systems

When to use the Search Field Wizard? Use the wizard when analyzing logs and you discover you need a new search criteria immediately. For planned, multi-field setups or batch operations, use the full Add or manage Search Field administration interface instead.

Understanding the Search Field Wizard

The Search Field Wizard is designed for speed and context—it brings the power of Search Fields directly into your active troubleshooting or analysis workflow.

Wizard vs Administration Interface

Decision Guide:

• Use the Wizard: You're troubleshooting in a Log View and need to search by a value not currently extracted (e.g., "I need to find all messages with OrderID 12345")
• Use Administration: You're planning Search Field strategy across multiple Message Types, setting up Search Field Links, or managing dozens of fields

How the Search Field Wizard Works

The wizard follows a simple, guided workflow:

Diagram: Search Field Wizard workflow from log analysis to deployed Search Field.

Step 1: Open the Search Field Wizard

Start from an active Log View that shows the messages you want to make searchable.

Prerequisites:

• Access to a Log View containing the messages you want to extract values from
• Permission to create or edit Search Fields (configured in Roles and Permissions)
• At least one Log Event visible in the current Log View (needed for sample data)

To open the wizard:

1. Navigate to the Log Views section in the Web Client
2. Open the Log View that contains the messages you want to extract data from
3. Locate and click the Search Field Wizard button (typically in the toolbar or action menu)

Example: The Search Field Wizard button in a Log View toolbar.

No messages in your view? The wizard requires at least one visible log event to provide sample data for testing. Apply filters to show relevant messages before opening the wizard.

Step 2: Create or Edit a Search Field

Choose whether to create a new Search Field or modify an existing one.

The Search Field Wizard modal showing creation and configuration options.

Create New Search Field

1. Click Create New Search Field in the wizard modal
2. Enter a Name for the Search Field:
   • Use business-friendly terms (e.g., "Order Number", "Customer ID", "Invoice Date")
   • Must be unique across all Search Fields
   • Avoid technical jargon (users will search with this name)
3. Select the appropriate Data Type:
   • Text: For alphanumeric values (Order123, InvoiceABC)
   • Integer/Long Integer: For whole numbers (counts, quantities, IDs)
   • Real Number: For decimal values (prices, amounts)
   • GUID: For unique identifiers (system-generated IDs)
   • Date time with offset: For timestamps with timezone
   • Bitwise Integer: For flag values
4. Optionally add a Description to explain what the field represents

Data Type cannot be changed after creation. Choose carefully—using "Text" for numbers prevents range queries (>, <), while using "Integer" for GUIDs causes extraction failures.

Edit Existing Search Field

1. Click Edit Existing Search Field in the wizard modal
2. Select the Search Field you want to modify from the dropdown list
3. The wizard loads the current configuration (name, data type, expression)
4. You can update the Name, Description, and Expression—but not the Data Type

Step 3: Configure the Expression

Tell Nodinite how to extract the value from your messages by choosing an expression type and configuring the extraction rule.

Expression Types and Plugins

The wizard supports all Search Field Expression types:

How to choose:

1. Check message format in the sample data preview (wizard shows you a real message from the current Log View)
2. Look for structure indicators:
   • XML tags (<OrderID>12345</OrderID>) → Use XPath
   • JSON properties ("orderId": "12345") → Use JSON Path
   • Delimited columns (OrderID,CustomerID,Amount) → Use Flat File CSV
   • Repeating patterns in text (Order: 12345) → Use RegEx
3. Use the wizard's auto-detection if available (some versions suggest the best plugin based on message content)

Example: Extracting an Order Number from XML

Sample message:

<Order>
  <OrderID>ORD-2024-12345</OrderID>
  <Customer>ACME Corp</Customer>
  <Amount>599.99</Amount>
</Order>

Configuration:

• Expression Type: XPath
• Expression: /Order/OrderID
• Expected Result: ORD-2024-12345

Example: Extracting a Customer ID from JSON

Sample message:

{
  "order": {
    "orderId": "ORD-2024-12345",
    "customerId": "CUST-789",
    "amount": 599.99
  }
}

Configuration:

• Expression Type: JSON Path
• Expression: $.order.customerId
• Expected Result: CUST-789

Example: Extracting a Booking Number from Flat Text

Sample message:

Shipment confirmed. Booking: TRN-2024-67890. Carrier: FedEx. ETA: 2024-12-15.

Configuration:

• Expression Type: RegEx
• Expression: Booking:\s*([A-Z0-9-]+)
• Expected Result: TRN-2024-67890

Step 4: Test the Expression with Sample Data

The wizard's killer feature: test your expression against real messages from the current Log View before deploying.

Testing an XPath expression against a real message in the wizard.

How Testing Works

1. The wizard displays a Sample Message (the body or context of a Log Event from your current view)
2. Enter your expression in the Expression field
3. Click Test or Validate button
4. The wizard shows:
   • Extracted Value (if successful)
   • Error Message (if expression is invalid or no match found)
   • Warnings (if data type mismatch or unexpected result)

Testing Best Practices

Do:

• Test with multiple sample messages by selecting different log events (if wizard supports message navigation)
• Verify extracted value matches your expectations exactly
• Check for data type compatibility (e.g., don't extract "ABC123" into an Integer field)
• Test with edge cases (empty values, special characters, null data)

Avoid:

• Assuming first test success means all messages will work (test with variety)
• Ignoring warnings about data type mismatches
• Using overly complex expressions without validation (start simple, refine iteratively)

Common Testing Issues

Issue: "No value extracted"

• Cause: Expression doesn't match message structure (wrong path, typo, namespace issue for XML)
• Fix: Verify expression syntax, check sample message structure, try simplified expression

Issue: "Data type mismatch"

• Cause: Extracted value doesn't fit selected data type (e.g., extracting "ABC" into Integer field)
• Fix: Change Data Type to Text, or adjust expression to extract numeric portion only

Issue: "Expression error"

• Cause: Invalid syntax for the selected plugin (e.g., malformed XPath or RegEx)
• Fix: Consult plugin documentation (XPath, JSON Path, RegEx), validate syntax with online tools

Step 5: (Optional) Configure Message Type Filtering

By default, the Search Field applies to all Message Types in all Log Databases. Optionally, restrict the Search Field to specific Message Types for performance and relevance.

When to filter by Message Type:

• Performance: Extracting complex values from millions of irrelevant messages slows re-indexing
• Accuracy: Different Message Types may have conflicting structures (same field name, different meaning)
• Clarity: Users should only see Search Fields relevant to their Log View context

How to configure Message Type filtering:

1. In the wizard, locate the Message Types or Apply to Message Types section
2. Choose one of:
   • All Message Types (default): Search Field applies to every message in every Log Database
   • Specific Message Types: Select one or more Message Types from the dropdown list
3. The wizard may pre-select Message Types visible in your current Log View as a suggestion

Example:

• Search Field: "SAP Document Number"
• Message Types: Only apply to "SAP IDoc Inbound", "SAP IDoc Outbound"
• Rationale: Non-SAP messages don't contain SAP Document Numbers; filtering prevents false negatives and improves indexing speed

Step 6: (Optional) Configure Search Field Links

Transform extracted values into clickable links to external business systems. Search Field Links bridge Nodinite with CRM, ERP, logistics, and custom applications.

When to use Search Field Links:

• You want users to jump directly from integration logs into external systems (e.g., order management, CRM)
• The extracted value is a business identifier (Order ID, Customer ID, Booking Number) used in another system
• You want to eliminate manual copy-pasting during troubleshooting

How to configure Search Field Links in the wizard:

1. Locate the Web Site or Link Template field in the wizard modal
2. Enter a URL template with a placeholder for the extracted value
   • Use {0} as the placeholder (some versions may use {value} or {field})
   • Example: https://erp.company.com/orders?id={0}
3. When users click the extracted value in a Log View, the link opens with {0} replaced by the actual value

Example: Link to Logistics System

• Search Field: "Transport Booking Number"
• Extracted Value: TRN-2024-67890
• Link Template: https://logistics.company.com/shipments?booking={0}
• Result: Clicking "TRN-2024-67890" opens https://logistics.company.com/shipments?booking=TRN-2024-67890

Example: Link to CRM

• Search Field: "Customer ID"
• Extracted Value: CUST-789
• Link Template: https://crm.company.com/customers/{0}
• Result: Clicking "CUST-789" opens https://crm.company.com/customers/CUST-789

Security Note: Ensure the target system supports direct URL access with IDs. Some systems require authentication tokens or session cookies—test links before deploying to users.

Step 7: Save the Search Field

Once configuration and testing are complete, save the Search Field to make it available in Log Views.

What happens when you save:

1. The Search Field is created in the Search Fields database
2. New values are extracted from incoming messages automatically (based on configured Message Types)
3. Historical data is NOT automatically re-indexed (requires manual re-indexing, see Step 8)
4. The Search Field appears in the Search Field list in all Log Views
5. Users can immediately start searching new messages using the field

Validation before saving:

• Name is unique (no duplicate Search Fields)
• Expression syntax is valid (tested in Step 4)
• Data Type is appropriate for extracted values
• If validation fails, the wizard shows error messages and prevents saving

Step 8: Re-Index to Apply to Historical Data

Search Fields created with the wizard only extract values from NEW messages by default. To make historical messages searchable, trigger a Re-Index operation.

Understanding Re-Indexing

What is re-indexing?

• Re-indexing applies the Search Field's expression to existing Log Events already stored in Log Databases
• This allows you to search retroactively (e.g., find all messages from last month by a newly created Customer ID field)
• Re-indexing is a background process that can take minutes to hours depending on data volume

When to re-index:

• You created a new Search Field and need to search historical data
• You modified an existing Search Field's expression and need to update extracted values
• You changed Message Type filtering and need to re-apply extraction rules

How to trigger re-indexing:

1. After saving the Search Field in the wizard, look for a Re-Index button or link
2. Alternatively, navigate to Administration > Search Fields Overview > select the Search Field > click Re-Index
3. Choose the re-indexing scope:
   • All Log Databases: Re-index every database (slowest, most comprehensive)
   • Specific Log Database: Re-index one database (faster, targeted)
   • Date Range: Re-index only messages within a specific timeframe (fastest)
4. Confirm and monitor progress (some versions show progress bar or completion notification)

Re-Indexing Performance Tips

Optimize re-indexing:

• Schedule during off-peak hours (re-indexing consumes database resources)
• Filter by Message Type before re-indexing (reduces scope)
• Use date ranges if you only need recent historical data
• Monitor SQL Server performance during large re-indexing operations

Avoid:

• Re-indexing all databases for a Search Field that applies to one specific Message Type (wastes resources)
• Running multiple large re-indexing operations simultaneously (causes contention)
• Re-indexing without testing the expression first (may extract incorrect values for millions of messages)

Verification and Testing

After creating and re-indexing the Search Field, verify it works correctly:

Test in Log Views

1. Navigate to a Log View that shows messages containing the extracted value
2. Open the Search Field filter panel
3. Select your new Search Field from the dropdown
4. Enter a known value (from testing in Step 4) and click Search
5. Verify the correct messages are returned

Check Extracted Values

1. Open a Log Event that should contain the extracted value
2. Look for the Search Field in the Search Fields tab or details panel
3. Verify the extracted value matches your expectations
4. If the value is blank or incorrect, review the expression and re-test

Test Search Field Links (if configured)

1. Find a Log Event with an extracted value
2. Click the value (if it's a Search Field Link)
3. Verify the external system opens with the correct record
4. If the link is broken, check the URL template syntax

Validate with Business Users

1. Ask business users to search using the new Search Field
2. Confirm they can find the data they need
3. Gather feedback on naming, data types, and usability
4. Refine if necessary (edit Search Field and re-test)

Best Practices

Naming Conventions

• Use business-friendly names that non-technical users understand (e.g., "Order Number", not "OrderID_Field_01")
• Be consistent across related Search Fields (e.g., "Customer ID", "Customer Name", "Customer Email")
• Avoid technical abbreviations unless universally understood (e.g., "GUID" is fine, "MSH-10" is not)
• Include context when ambiguity exists (e.g., "SAP Document Number", "CRM Customer ID")

Expression Design

• Start simple and refine iteratively (test basic extraction before adding transformations)
• Test with variety (multiple messages, edge cases, empty values)
• Document complex expressions in the Description field (future admins will thank you)
• Use Message Type filtering to avoid extracting from irrelevant messages

Data Type Selection

• Choose the most specific data type (Integer for counts, GUID for system IDs, Date for timestamps)
• Never use Text for everything (prevents range queries and numeric operations)
• Consider user workflows (Text supports "Contains", Integer supports "<" and ">")

Re-Indexing Strategy

• Test before re-indexing (verify expression works on sample data first)
• Use date ranges for large databases (re-index last 90 days, not all historical data)
• Schedule during off-peak hours (minimize impact on production users)
• Monitor progress and database performance during re-indexing

Common Pitfalls

Pitfall: Creating duplicate Search Fields

• Problem: Multiple Search Fields with similar names or purposes confuse users
• Solution: Check existing Search Fields before creating new ones; edit existing instead of duplicating

Pitfall: Using overly complex expressions

• Problem: Advanced RegEx or Formula expressions are hard to maintain and debug
• Solution: Use simpler expression types when possible (XPath for XML, JSON Path for JSON)

Pitfall: Forgetting to re-index

• Problem: Users can't find historical data with new Search Field
• Solution: Always trigger re-indexing after creating or modifying Search Fields (document this step)

Pitfall: Ignoring Message Type filtering

• Problem: Extracting values from irrelevant messages slows performance and causes false positives
• Solution: Apply Message Type filters when creating Search Fields with specific purpose

Troubleshooting

Issue: "Search Field Wizard button is disabled or missing"

Possible Causes:

• Your user account doesn't have permission to create/edit Search Fields
• The current Log View has no visible Log Events (wizard requires sample data)
• The Nodinite version doesn't support the wizard feature

Solutions:

• Contact your administrator to grant Search Field edit permissions (check Roles and Permissions)
• Apply filters to show Log Events in the current Log View
• Update Nodinite to a version that supports the Search Field Wizard

Issue: "Expression test shows 'No value extracted'"

Possible Causes:

• Expression syntax is incorrect (typo, wrong path, malformed RegEx)
• Sample message structure doesn't match expression (wrong format, missing field)
• Namespace issues for XML (XPath requires namespace handling)

Solutions:

• Copy the sample message and validate expression in an external tool (Regex101 for RegEx, online XPath/JSON Path testers)
• Simplify the expression to test individual components (e.g., /Order instead of /Order/Details/Item[1]/Price)
• For XML with namespaces, use local-name() or configure namespace prefixes

Issue: "Extracted value is blank in Log Views after re-indexing"

Possible Causes:

• Re-indexing didn't complete (check background job status)
• Expression doesn't match actual message structure in Log Database (tested with wrong sample)
• Message Type filtering excludes the messages you're searching

Solutions:

• Check re-indexing job status in Administration > Background Jobs (if available)
• Open a Log Event and manually verify message structure matches expression
• Review Message Type filter settings and expand to include all relevant types

Issue: "Search Field Link doesn't work (404 or wrong page)"

Possible Causes:

• URL template syntax is incorrect ({0} placeholder missing or malformed)
• External system URL changed (link target moved or requires authentication)
• Extracted value contains special characters not URL-encoded

Solutions:

• Copy the generated link from browser address bar and compare to expected URL
• Test the URL template manually by replacing {0} with a known value
• Contact external system administrators to verify URL format and authentication requirements

Next Steps

Now that you've mastered the Search Field Wizard, explore related features:

• Add or manage Search Field - Use the full administration interface for advanced scenarios
• Search Field Expressions - Deep dive into expression types, plugins, and syntax
• Search Field Links - Configure external system integration for one-click navigation
• Re-Index - Learn advanced re-indexing strategies for large databases
• Log Views - Discover how to use Search Fields for powerful filtering and analysis

Related Topics

Getting Started

• Getting Started with Search Fields - Beginner's guide to Search Field concepts
• What is a Search Field? - Understand the value and use cases

Features and Configuration

• Search Field Expression - All expression types and plugins explained
• XPath - Extract values from XML messages
• JSON Path - Extract values from JSON messages
• RegEx - Pattern-based extraction for unstructured text
• Flat File CSV - Extract from delimited files
• Flat File Fixed Width - Extract from fixed-width files
• Message Context Key - Extract from message context properties
• Formula - Transform and combine extracted values

Advanced Topics

• Message Types - Configure Message Type filtering
• Log Databases - Understand log storage and re-indexing scope
• Roles and Permissions - Manage Search Field access control