What are Search Field Expressions?
A Search field expression for a Search Field is the syntax/format/code/logic used to extract one or more unique values on selected Message Types and is crucial to enable support for correlation spanning multiple platforms and time. For example, an Order Number may exist in numerous messages (of different Message Types), and to find this Order Number, one or more search field expressions can be used for the Search Field.
A tracking example of the Search Field Order Number to enable a true end-to-end logging experience.
- There can be any number of search field expressions for a given Search Field
- Each search field expression can be used on one or more message type
- Set the Global option to apply the expression on all message types. Use this feature with care as this causes additional processing.
- Set the Optional option to prevent the Log Event to end up in the Processed with Warnings state when the value is missing.
A Search Field Expression retrieves values from any part of the Log Event:
- Messages (message body/payload) (and/or depending on plugin)
- Context properties (Metadata stored in a Key/Value Collection for the Event)
- Custom Plugins: External source using data from message or Context
The extract operation performed by the Logging Service uses the statement from the Expression on the selected Message Type using the selected Search Field Expression Type. The latter is a .NET assembly implementing the appropriate Nodinite contracts. Put the assemblies in the 'Plugins' folder of the Logging Service. You must restart to load the assemblies.
NOTE: The contracts may change with different versions, you must have a plan to update your code whenever this happens.
Nodinite comes with many built-in Search Field Expression Type plugins for use in everyday scenarios. There are many different data formats, and the data of interest may be embedded within the payload or the context properties (key/value pair). You may also need to clean the data to get the result you seek (for example, you may only want to extract the filename, without the suffix from a full path). You can also create virtual search fields, for example, count the number of order rows or the length of a field. The latter is helpful for non-events monitoring scenarios.
The Different plugins sport different features on what data is possible to extract; please review each one and/or the Troubleshooting user guide for additional details.
Example of Expression Types, in this use case, the high-performance XPath is exemplified.
|Plugin Name||XML||EDIFACT/X12/FLATFILE||JSON||Context Property|
|Flat File CSV|
|Flat File Fixed Width|
|Message Context Key|
|RegEx on Message Context with capturing groups|
|RegEx on Message Context|
|RegEx with capturing groups|
|XPath on Message Context|
|XPath on Wrapped XPath|
|XPath with RegEx|
Nodinite is an extendable platform, you may write and add your own plugins. This is useful for special cases (lookup tables, API calls, ...) and encrypted messages.