Azure Event Hub Configuration Example
On this page you will learn how to configure the Nodinite Pickup Logging Service to fetch Nodinite JSON Log Event from an Azure Event Hub.
EventHubs
The EventHubs section is where you manage Azure EventHub related sources. The JSON Log Events on the event hub may come from logging using a Policy from your API management entry points, please review the 'Logging the Request/Response from the Azure API Management platform' user guide.
- An Azure Event Hub (target for Logging).
- You can use one or more Event Hub entities as the target.
- A storage container to keep track of the current checkpoint.
- Each Event Hub Entity must have its unique container to store the bookmark (checkpoint).
Important
You must have one unique storage container per unique Event Hub entity! Otherwise, you will overwrite the checkpoint, preventing the Logging feature from being operational.
The configuration file (Settings.json), is in JSON format, the default path is:
C:\Program Files\Nodinite\Logging Agent - Pickup Service\Settings.json
{
...
"EventHubs": [
{
"Enabled": true,
"LogAPI": "http://localhost/Nodinite/Dev/LogAPI/",
"UseConfigurationDatabase": false,
"ConfigurationDatabaseConnectionString": null,
"EventHubConnectionString": "Endpoint=sb://replaceme.servicebus.windows.net/;SharedAccessKeyName=somename;SharedAccessKey=replaceme-1337=;EntityPath=replaceme",
"EventHubName": null,
"StorageContainerName": "event-hub-replaceme",
"StorageAccountName": "replaceme",
"StorageAccountKey": "replaceme-42=",
"ReplaceRules": [
{
"Name": "Fix Endpoint real customer id to {customerId}",
"ReplaceName": false,
"ReplaceUri": true,
"Pattern": "/([0-9]{4,})$",
"Group1Value": "{customerId}"
}]
}]
...
}
EventHubs is an array of a EventHub configuration. Each configuration is valid for one source Event Hub Entity. This configuration tells the Log Agent where to look for log messages with the Nodinite JSON Log Events.
| Property | Description | Value Example | Comment |
|---|---|---|---|
| EventHubConnectionString | Connection string for EventHub | Endpoint=sb://replaceme.servicebus.windows.net/;SharedAccessKeyName=[KeyName];SharedAccessKey=[Key];EntityPath=[EntityPathName] |
|
| EnableAmqpLinkRedirect | A flag used to control the EnableAmqpLinkRedirect property | true (default) - currently not honoured. Default is True | Review more information here |
| EventHubName | Name of the EventHub | If EntityPath=[EntityPathName] is not provided in the connection string then you must provide the name using this entry |
|
| StorageAccountName | Storage Acccount Name in use to store the checkpoint (syncpoint) | Please review the Storage Account Name section below | |
| StorageAccountKey | The access key to the storage account | Please review the Storage Account Keys section below | |
| WebProxy | Proxy | 1.3.3.7:1337 | Sets the EventProcessorOptions.WebProxy Property. When in use, also set the TransportType = EventHubsTransportType.AmqpWebSockets |
| StorageContainerName | Name of the storage | Checkpoints are being used and these are written to the storage | |
| Enabled | Review the Shared Configuration section for additional information | ||
| LogAPI | Review the Shared Configuration section for additional information | ||
| UseConfigurationDatabase | Review the Shared Configuration section for additional information | ||
| ConfigurationDatabaseConnectionString | Review the Shared Configuration section for additional information | ||
| ReplaceRules | Review the shared Replace Rules section for additional information |
Important
You must restart the Nodinite Pickup Logging Service for the changes to the configuration file to be operational.
Note about EnableAmqpLinkRedirect
Clients that use AMQP connections over TCP require ports 5671 and 5672 to be opened in the local firewall. Along with these ports, it might be necessary to open additional ports if the EnableLinkRedirect feature is enabled. EnableLinkRedirect is a new messaging feature that helps skip one-hop while receiving messages, thus helping to boost throughput. The client would start communicating directly with the back-end service over port range 104XX.
A .NET client would fail with a SocketException ("An attempt was made to access a socket in a way forbidden by its access permissions") if these ports are blocked by the firewall. The feature can be disabled by setting EnableAmqpLinkRedirect=false in the connectiong string, which forces the clients to communicate with the remote service over port 5671.
Storage Account Name
The StorageAccountName can be found as pictured below in the Azure Portal:
Storage Account Keys
The StorageAccountKey can be found as pictured below in the Azure Portal:
Important
Use the value from a Key, Not the connection string.