- 12 minutes to read

Logging with the IBM Integration Bus Logging Agent

On this page you will learn how to achieve end-to-end Logging of the IBM Monitoring Events and Nodinite. You will also learn hos the Logging is being performed technically, and you will also learn about the rules for mapping the IBM Monitoring Event into a Nodinite Log Event.

Create the Monitoring Profile

IBM has the following user guides available for creating the required Monitoring Profile from which IBM Monitoring Events will be emitted:

IBM Integration Bus Version How to Create the Monitoring Profile Monitoring Event
12
11
10
9

You may also find this IBM blog post useful with a complete user guide for

Technical details for Logging IBM Monitoring Events

In your environment you may have multiple IBM Integration Bus workflows running on different servers. This means you may end up having tons of Monitoring Events to search. You may have different developers having enabled or tuned logging differently due to lack of governance. With Nodinite you get total control and a foundation for governance and an active documentation using our Repository Model. With Nodinite you can centralize logging and manage/share insights from IBM Monitoring events.

graph LR subgraph "IBM Integration Bus workflow" roIBMApplication(fal:fa-sitemap IBM Integration Bus workflow)--> roIBMTracing(fal:fa-bolt IBM Monitoring Events
Monitoring profile) roFolder(fal:fa-list IBM MQ Queue) roIBMTracing --> roFolder end subgraph "Nodinite Server" roPS(fal:fa-truck-pickup Nodinite IBM Integration Bus Logging Agent) roFolder --> |TCP| roPS end
Messages put on IBM MQ queues are consumed by :Nodinite:, you do not need to create custom cleaning tasks

Scenario: Basic Request/Response

In this scenario we have one or more Consumers of the IBM Integration Bus workflow.

sequenceDiagram participant Consumer participant IIB workflow Consumer->>IIB workflow: Request IIB workflow->>Consumer: Response

IBM Monitoring Events enabled

Enable logging by defining Monitoring profiles. With the built-in IBM Monitoring Events enabled the Request and the Response is caught by the policy and is copied to the IBM MQ destination queue.

sequenceDiagram participant Consumer participant IIB workflow participant Monitoring Event Consumer->>IIB workflow: Request IIB workflow ->> Monitoring Event: Log Request IIB workflow->>Consumer: Response IIB workflow ->> Monitoring Event: Log Response

Nodinite Nodinite Logging

The Nodinite IBM Integration Bus Logging Agent consumes the Monitoring Events on the IBM MQ queues. This means that Nodinite will NOT waste your disk space or other valuable resources over time and you do not have to setup any cleaning maintenance jobs.

sequenceDiagram participant Consumer participant IIB workflow participant Monitoring Event participant Nodinite Logging Consumer->>IIB workflow: Request IIB workflow ->> Monitoring Event: Log Request Monitoring Event ->> Nodinite Logging: Nodinite Log Event (Json) IIB workflow->>Consumer: Response IIB workflow ->> Monitoring Event: Log Response Monitoring Event ->> Nodinite Logging: Nodinite Log Event (Json)

Mapping from IBM Monitoring Event to Nodinite Log Event

This paragraph describes the how IBM Monitoring Events are mapped to Nodinite Log Events.

You must first configure IBM Integration Bus to emit Monitor Events to one or more queues, read more in the Configuring IBM Integration Bus Logging user guide.

NOTE: IBM changed the format with version 11 (same for version 12) compared to version 9 and 10.

Version 11 and 12 (ACE)

This section applies to IBM ACE

Sample IBM Monitoring Event (XML Message)

<?xml version="1.0" encoding="UTF-8"?>
<mon:event xmlns:mon="http://www.ibm.com/xmlns/monitoring/event/v2">
	<mon:eventPointData>
		<mon:eventData mon:productVersion="110010"
			mon:profileVersion="11" mon:eventSourceAddress="HTTP Input.transaction.Start">
			<mon:eventIdentity mon:eventName="HTTP Input.TransactionStart" />
			<mon:eventSequence mon:creationTime="2020-07-20T16:20:21.776758Z"
				mon:counter="1" />
			<mon:eventCorrelation mon:localTransactionId="40ddc412-f102-4840-a212-8b793d12b6bf-1"
				mon:parentTransactionId="" mon:globalTransactionId="" />
		</mon:eventData>
		<mon:messageFlowData>
			<mon:integrationServer mon:name="110010"
				mon:hostName="DESKTOP-7Q9JIP6" />
			<mon:application mon:name="Transformation_Map" />
			<mon:messageFlow mon:uniqueFlowName="110010.Transformation_Map.Transformation_Map"
				mon:name="Transformation_Map" mon:threadId="12244" />
			<mon:node mon:nodeLabel="HTTP Input" mon:nodeType="ComIbmWSInputNode" />
		</mon:messageFlowData>
	</mon:eventPointData>
	<mon:bitstreamData>
		<mon:bitstream mon:serializationEncoding="base64Binary"
			mon:dataCCSID="1208" mon:dataEncoding="546">UE9TVCBodHRwOi8vbG9jYWxob3N0Ojc4MDAvVHJhbnNmb3JtYXRpb25fTWFwIEhUVFAvMS4xDQpDb250ZW50LVR5cGU6IHRleHQvcGxhaW47IGNoYXJzZXQ9dXRmLTgNClVzZXItQWdlbnQ6IEphdmEvMS44LjBfMjUxDQpIb3N0OiBsb2NhbGhvc3Q6NzgwMA0KQWNjZXB0OiB0ZXh0L2h0bWwsIGltYWdlL2dpZiwgaW1hZ2UvanBlZywgKjsgcT0uMiwgKi8qOyBxPS4yDQpDb25uZWN0aW9uOiBrZWVwLWFsaXZlDQpDb250ZW50LUxlbmd0aDogNjgwDQpYLU9yaWdpbmFsLUhUVFAtQ29tbWFuZDogUE9TVCBodHRwOi8vbG9jYWxob3N0Ojc4MDAvVHJhbnNmb3JtYXRpb25fTWFwIEhUVFAvMS4xDQpYLVNlcnZlci1OYW1lOiBsb2NhbGhvc3QNClgtU2VydmVyLVBvcnQ6IDc4MDANClgtUmVtb3RlLUFkZHI6IDEyNy4wLjAuMQ0KWC1SZW1vdGUtSG9zdDogbG9jYWxob3N0DQpYLVNjaGVtZTogaHR0cA0KDQo8P3htbCB2ZXJzaW9uPSIxLjAiIGVuY29kaW5nPSJVVEYtOCI/Pg0KPFNhbGVFbnZlbG9wZT4NCgk8SGVhZGVyPg0KCQk8U2FsZUxpc3RDb3VudD4xPC9TYWxlTGlzdENvdW50Pg0KCQk8VHJhbnNmb3JtYXRpb25UeXBlPnhzbDwvVHJhbnNmb3JtYXRpb25UeXBlPg0KCTwvSGVhZGVyPg0KCTxTYWxlTGlzdD4NCgkJPEludm9pY2U+DQoJCQk8SW5pdGlhbD5UPC9Jbml0aWFsPg0KCQkJPEluaXRpYWw+SjwvSW5pdGlhbD4NCgkJCTxTdXJuYW1lPkR1bm53aW48L1N1cm5hbWU+DQoJCQk8SXRlbT4NCgkJCQk8Q29kZT4wNDwvQ29kZT4NCgkJCQk8Q29kZT4wNTwvQ29kZT4NCgkJCQk8Q29kZT4wMTwvQ29kZT4NCgkJCQk8RGVzY3JpcHRpb24+VGhlIE9yaWdpbiBvZiBTcGVjaWVzPC9EZXNjcmlwdGlvbj4NCgkJCQk8Q2F0ZWdvcnk+Qm9va3MgYW5kIE1lZGlhPC9DYXRlZ29yeT4NCgkJCQk8UHJpY2U+MjIuMzQ8L1ByaWNlPg0KCQkJCTxRdWFudGl0eT4wMjwvUXVhbnRpdHk+DQoJCQk8L0l0ZW0+DQoJCQk8QmFsYW5jZT44MS44NDwvQmFsYW5jZT4NCgkJCTxDdXJyZW5jeT5FdXJvczwvQ3VycmVuY3k+DQoJCTwvSW52b2ljZT4NCgk8L1NhbGVMaXN0Pg0KCTxUcmFpbGVyPg0KCQk8Q29tcGxldGlvblRpbWU+MTIuMDAuMDA8L0NvbXBsZXRpb25UaW1lPg0KCTwvVHJhaWxlcj4NCjwvU2FsZUVudmVsb3BlPg==</mon:bitstream>
	</mon:bitstreamData>
</mon:event>

Mapping table between Log Event and the IBM Monitoring Event

Mandatory Data Type Field Path Value
string LogAgentValueId N/A Configuration file
string EndPointName event/eventPointData/eventData/eventIdentity/@eventName HTTP Input.TransactionStart
string EndPointUri event/eventPointData/eventData/@eventSourceAddress HTTP Input.TransactionStart
NOTE: Make sure to create the Monitoring profile to listen on the "right" side of the IIB shape
number EndPointDirection event/eventPointData/eventData/@eventSourceAddress From Terminal shape .in=0, .out = 1, no match = -2
number EndPointTypeId N/A 41
string OriginalMessageTypeName "Monitoring Event" Hardcoded, unless the payload is XML; Then Nodinite extracts the Message Type Name, else, please use Context Options and send as simpleContent
string LogDateTime event/eventPointData/eventData/eventSequence/@creationTime 2020-07-20T16:20:21.776758Z
number EventDirection - Send as SimpleContent, see example below table:
  • DefaultProperties/1.0#EventDirection
string ProcessingUser N/A
number SequenceNo event/eventPointData/eventData/eventSequence/@counter 1
number EventNumber N/A
string LogText - Send as SimpleContent, see example below table:
  • DefaultProperties/1.0#LogText
string ApplicationInterchangeId TBD missing in ACE was previously event/eventPointData/messageFlowData/messageFlow/@UUID N/A
guid LocalInterchangeId event/eventPointData/eventData/eventCorrelation/@localTransactionId 40ddc412-f102-4840-a212-8b793d12b6bf-1
string LogStatus N/A '-1' (Unknown) if no match on EndPointDirection, else '0' (OK) and '-2' (catch)
This value may be overridden using a value for LogStatusCode as simpleContent
string ProcessName event/eventPointData/messageFlowData/messageFlow/@uniqueFlowName 110010.Transformation_Map.Transformation_Map
string ProcessingMachineName event/eventPointData/messageFlowData/integrationServer/@hostName ACESRV01.ACME.local
string ProcessingUser event/eventPointData/messageFlowData/integrationNode/@name XYZ.123Q
string ProcessingModuleName event/eventPointData/messageFlowData/node/@nodeLabel HTTP Input
string ProcessingModuleType event/eventPointData/messageFlowData/node/@nodeType ComIbmWSInputNode
guid ServiceInstanceActivityId event/eventPointData/messageFlowData/messageFlow/@threadId 4201
NOTE: If this is not a GUID, then the value is in the Context collection with the ThreadId as the key
string ProcessingTime N/A

TIP: Use the Nodinite Context Options to override default behaviour

Version 9 and 10

Sample IBM Monitoring Event (XML Message)

<?xml version="1.0" encoding="UTF-8"?>
<wmb:event xmlns:wmb="http://www.ibm.com/xmlns/prod/websphere/messagebroker/6.1.0/monitoring/event">
	<wmb:eventPointData>
		<wmb:eventData wmb:eventSourceAddress="MQInput1.terminal.in" wmb:eventSchemaVersion="6.1.0.3" wmb:productVersion="7000">
			<wmb:eventIdentity wmb:eventName="MQInput event"/>
			<wmb:eventSequence wmb:creationTime="2017-09-23T12:00:00+02:00" wmb:counter='2'/>
			<wmb:eventCorrelation wmb:localTransactionId="a13122ae-1c01-0000-0080-b1b02528c6cd" wmb:parentTransactionId="456" wmb:globalTransactionId="789"/>
		</wmb:eventData>
		<wmb:messageFlowData>
			<wmb:broker wmb:UUID="d53122ae-1c01-0000-0080-b1b02528c6bf" wmb:name="myBroker"/>
			<wmb:executionGroup wmb:UUID="d43122ae-1c01-0000-0080-b1b02528c6bf" wmb:name="default"/>
			<wmb:messageFlow wmb:UUID="e6d224ae-1c01-0000-0080-9100cd1a61f7" wmb:name="myMessageFlow" wmb:threadId="4201" wmb:uniqueFlowName="myBroker.default.myMessageFlow"/>
			<wmb:node wmb:nodeLabel="MQInput1" wmb:nodeType="ComIbmMqInputNode" wmb:terminal="in" wmb:detail="MYMESSAGEFLOW.IN"/>
		</wmb:messageFlowData>
	</wmb:eventPointData>
	<wmb:applicationData xmlns="">
		<wmb:simpleContent wmb:name="invoiceNo" wmb:targetNamespace="" wmb:dataType="string" wmb:value="567"/>
		<wmb:complexContent wmb:elementName="customerName" wmb:targetNamespace="">
			<customerName>
				<firstName>Steve</firstName>
				<lastName>Bloggs</lastName>
			</customerName>
		</wmb:complexContent>
	</wmb:applicationData>
	<wmb:bitstreamData>
		<wmb:bitstream wmb:encoding="base64Binary">TUQgIAIAAAAAAAAACAAAAP////8AAAAAIgIAALUBAAAgICAgICAgIAAAAAAAAAAAQU1RIFFNMSAgICAgICAgIHo640ggABsHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFFNMSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg</wmb:bitstream>
	</wmb:bitstreamData>
</wmb:event> 

Mapping table between Log Event and the IBM Monitoring Event

Mandatory Data Type Field Path Value
string LogAgentValueId N/A Configuration file
string EndPointName event/eventPointData/eventData/eventIdentity/@eventName MQInput event
string EndPointUri event/eventPointData/eventData/@eventSourceAddress MQInput1.terminal.in
NOTE: Make sure to create the Monitoring profile to listen on the "right" side of the IIB shape
number EndPointDirection event/eventPointData/eventData/@eventSourceAddress From Terminal shape .in=0, .out = 1, no match = -2
number EndPointTypeId N/A 41
string OriginalMessageTypeName "Monitoring Event" Hardcoded, unless the payload is XML; Then Nodinite extracts the Message Type Name, else, please use Context Options and send as simpleContent
string LogDateTime event/eventPointData/eventData/eventSequence/@creationTime 2017-09-23T12:00:00+02:00
number EventDirection - Send as SimpleContent, see example below table:
  • DefaultProperties/1.0#EventDirection
string ProcessingUser N/A
number SequenceNo event/eventPointData/eventData/eventSequence/@counter 2
number EventNumber N/A
string LogText - Send as SimpleContent, see example below table:
  • DefaultProperties/1.0#LogText
string ApplicationInterchangeId event/eventPointData/messageFlowData/messageFlow/@UUID e6d224ae-1c01-0000-0080-9100cd1a61f7
guid LocalInterchangeId event/eventPointData/eventData/eventCorrelation/@localTransactionId a13122ae-1c01-0000-0080-b1b02528c6cd
string LogStatus N/A '-1' (Unknown) if no match on EndPointDirection, else '0' (OK) and '-2' (catch)
This value may be overridden using a value for LogStatusCode as simpleContent
string ProcessName event/eventPointData/messageFlowData/messageFlow/@uniqueFlowName myBroker.default.myMessageFlow
string ProcessingMachineName event/eventPointData/messageFlowData/broker/@name myBroker
string ProcessingModuleName event/eventPointData/messageFlowData/node/@nodeLabel MQInput1
string ProcessingModuleType event/eventPointData/messageFlowData/node/@nodeType ComIbmMqInputNode
guid ServiceInstanceActivityId event/eventPointData/messageFlowData/messageFlow/@threadId 4201
NOTE: If this is not a GUID, then the value is in the Context collection with the ThreadId as the key
string ProcessingTime N/A

TIP: Use the Nodinite Context Options to override default behaviour

SimpleContent

The following simpleContent will be converted to Nodinite Context Properties (Key Value pairs):

the wmb prefix is just a prefix, it can be different in your instance/examples and you can use the Nodinite Context Options to override default behaviour

Additional field values

The following example converts simpleContent to Additional Field Values

From To
<p:simpleContent p:name="ToolOfTheDay" p:dataType="string"><p:valueQuery p:queryText="'Nodinite'"/></p:simpleContent> Key=ToolOfTheDay
Value: Nodinite
<p:simpleContent p:name="invoiceNo" p:dataType="int"><p:valueQuery p:queryText="'123'"/></p:simpleContent> Key: invoiceNo
Value: 123

Override options

From To
<p:simpleContent p:name="LogStatusCode" p:dataType="int"><p:valueQuery p:queryText="'-1337'"/></p:simpleContent> Optional override for LogStatus
<p:simpleContent p:name="ExtendedProperties/1.0#MessageTypeName" p:dataType="string"><p:valueQuery p:queryText="'https://www.acme.com/supplychain/1.0#invoice'"/></p:simpleContent> Optional override for MessageType
<p:simpleContent p:name="DefaultProperties/1.0#LogText" p:dataType="string"><p:valueQuery p:queryText="'Hello World'"/></p:simpleContent> Optional override for LogText
Value: Hello world
Event Direction

You can provide additional information about the direction of the logged event(EventDirection). Simply log the message using simpleContent using either the integer or string value as exemplified in the table below:

# string Value integer Value Comment
Default null Not set
1 ExternalIncoming 17 Incoming message before receive in a One-Way receive Port processing
2 ExternalIncomingRequest 21 Incoming request message before receive Request-Response Port processing
3 ExternalIncomingResponse 25 Outgoing response message after receive Request-Response Port processing
4 ExternalOutgoing 18 Outgoing message after send in a One-Way Port processing
5 ExternalOutgoingRequest 22 Outgoing request message after send (Request-Response Port processing
6 ExternalOutgoingResponse 26 Incoming response message after send Request-Response Port processing
7 InternalIncoming 33 Incoming message after receive in a One-Way receive Port processing
8 InternalOutgoing 34 Outgoing message before send in a One-Way send Port processing
9 ProcessIncoming 65 Incoming message to a function/workflow/orchestration
10 ProcessOutgoing 66 Outgoing message to a function/workflow/orchestration
<p:simpleContent p:name="DefaultProperties/1.0#EventDirection" p:dataType="integer"><p:valueQuery p:queryText="17"/></p:simpleContent>

Example sending EventDirection ExternalIncoming using integer value

<p:simpleContent p:name="DefaultProperties/1.0#EventDirection" p:dataType="string"><p:valueQuery p:queryText="ExternalIncoming"/></p:simpleContent>

Example sending EventDirection ExternalIncoming using string value

ComplexContent

complexContent will be converted to Nodinite Context Properties (Key Value pairs):

From To
<wmb:complexContent wmb:elementName="customerName" 
                            wmb:targetNamespace="">
    <customerName>
        <firstName>Steve</firstName>
        <lastName>Bloggs</lastName>
    </customerName>
</wmb:complexContent>
Key:

Environment.Monitoring.ComplexContent.customerName

Value:

<customerName><firstName>Steve</firstName><lastName>Bloggs</lastName></customerName>

TIP: Use the Formula plugin to extract values from ComplexContent context properties!

ExceptionList

If you have a Monitoring Profile that catches the error information as ExceptionList in the ComplexContent element then Nodinite will replace any existing body with this XML content. The LogStatus code if not overridden will be set to -2.

The name of the ComplexContent element does not matter when the content is ExceptionList xml data.

bitstreamData

The content of the <bitstreamData> element is the payload encoded as a base64string. Nodinite internally happen to use the same base64string encoding for the payload so this value is simply copied to the body property of the Log Event.

Transaction Id's

Some properties for correlation are put in the Additional Fields but they are also copied to the Context Properties.

<wmb:eventCorrelation wmb:localTransactionId="123" wmb:parentTransactionId="456" wmb:globalTransactionId="789"/>
Key Value
Environment.Monitoring.EventCorrelation.LocalTransactionId 123
Environment.Monitoring.EventCorrelation.ParentTransactionId 456
Environment.Monitoring.EventCorrelation.GlobalTransactionId 789

RFH2 Header

If your IBM MQ messages has RFH2 header then the IBM Integration Bus Logging Agent converts all these to Context Properties and thus be an integral part of the Nodinite Log Event.

Nodinite IBM Integration Bus Logging Agent takes care of the RFH2 header automatically if it exists on the IBM MQ queue


Next Step

Now Add or manage Log Views and use the Log Views to enjoy end to end tracking with Nodinite including IBM Monitoring Events.