- 4 minutes to read

Azure API Management Service Policy-based Logging

Info

This guide teaches how to apply a Nodinite-specific policy to enable Logging from the Azure API Management Service platform.

In this guide, there are three ways to create and send the Nodinite Log Event.

Policy Full payload (Body) HTTP Headers (Context) Monitoring
1. Any size, Blob Storage Policy (recommended) Azure Agent
Non Events Agent
2. <200KB, Event Hub Policy Azure Agent
Non Events Agent
3. Direct API call Log API Policy (supported, but not recommended) Web Services Agent
Non Events Agent
graph LR subgraph "Payload (Any size)" AA[Azure API Management
with Policy 1 - Blob Storage] --> |1. Log Event| roBS[Blob Storage] end subgraph "Payload < 200KB" A[Azure API Management
with Policy 2 Event Hub] A -->|2. Log Event| B(Event Hub) end subgraph "Consumption tier - API Call" AAA[Azure API Management
with Policy 3] end subgraph "Nodinite" AAA -.-> |"3. Log Event (NOT our recommendation)"|F B --> C{Pickup Service} C --> F[Log API] roBS --> C end

When the policy (Blob Storage Policy or Event Hub Policy) is in place and active, the code running in the Azure API Management platform creates a Nodinite-specific JSON Log Event and sends it to the intermediate storage.

The Nodinite Pickup Log Events Service Logging Agent consumes the Nodinite Log Events from the intermediate store (Event Hub or Blob Storage Container). This pattern is Asynchronous.

Tip

You can use the Nodinite Non-Events Monitoring Agent to get alerts if you have too many or too few events within a period.

Body

You can log the full payload. If you use the Blob Storage Policy, the size of the message does not matter. If it is <200KB, then you can use the Event Hub Policy. Having the payload in Nodinite has the advantage that you can use Search Fields in Log Views to create a rich user-experience on-demand for your business.

Important

When you log the payload, you must provide a name for the Message Type.

You can change the display of the body using Stylesheets. This is a cute trick to hide content. You can use the Role-based security to apply who can access what and who gets to see what.

Context

A Nodinite Log Event supports the use of a Context; A collection of key-value pairs. In addition with the existing HTTP Headers in the call, you can have additional properties in the Logging policy. You can later use these to create the set of Search Fields of interest for use in self-service enabled Log Views.

Request Headers Response Headers

Tip

You can add to the code in your policy exactly what properties are included in the Logging.

If you happen to log to much, you may want to tweak the following System Parameter:

Obviously since you have full control of the Logging in the Policy, you can determine up-front what to include in the Logging.

Troubleshooting

Contact our Support for additional guidance if you fail to implement the Logging.

Logging Options

In the diagram below you can find the many different options for Logging from Azure to Nodinite.

graph LR subgraph "Integration solution" C[Azure Function
with Serilog logging] ---|<1 MB payload|C1(Serilog Event Hub Sink) C --> |Any size| C2(Serilog Blob Storage Sink) A[Azure API Management Service API
with Policy] --- |<200 KB payload| A2(Policy2) A --- |Any size| A1(Policy1) B[Azure Logic App
with the proper diagnostic setting enabled] end subgraph "Event Hub" A2-->|1. Nodinite JSON Log Event| AA(EH1) C1-->|1. Nodinite JSON Log Event| AA C1 -.- |1. Nodinite JSON Log Event| CC(EH2) B-->|2. Azure Diagnostics log| BB(EH3) end subgraph "Azure Storage Account" A1-->|1. Nodinite JSON Log Event| Cont(Container1) end C2 --> |1. Nodinite JSON Log Event|Cont subgraph "Nodinite" Cont --> ro[Pickup Service] AA --> ro CC -.-> ro BB --> roLA(Logic Apps Logging and Monitoring Agent) end

If the target for logging from the API policy is an Event Hub entity; The named Event Hub entity CAN be re-used if the content logged, is a Nodinite JSON Log Event.

Important

EH1, EH2 and EH3 MUST have each own syncpoint (bookmark), i.e. one blob storage container for each event hub entity and client.

Using a shared Event Hub for a Nodinite JSON Log Event has the advantage of less administration (i.e. you must configure the Nodinite Pickup Log Events Service Logging Agent to fetch these events)
The disadvantage can be performance-related, or other factors should be considered like pricing (logging from multiple regions), scalability, retention settings, number of partitions, ...

Tip

Make sure to use separate event hub entities for different purposes and needs, i.e. multiple Logic App Logging, multiple API management Services, multiple APIs, multiple Azure functions, all according to your architecture.

Important

You MUST NOT use the same Event Hub Entity for Logic Apps Logging as for the Serilog and Policy based solutions. This is because the diagnostics from Logic Apps are from Microsoft and all the other creates a Nodinite JSON Log Event. These can not appear on the same Event Hub Entity, EVER.

Next Step

Pickup Log Events Service Logging Agent
Azure Application Access

JSON Log Event
Managing
Monitoring
Non Events Agent
Stylesheets

Interested in logging from other Azure Related Services?