What is a Log Database?
Nodinite log events and messages from your system integration brokers (Mule, BizTalk, ...) and even your custom solutions using Log Events. All this data is persisted for long term storage in SQL Server based databases in Nodinite known as Log Databases. Over time as stored volumes grow, there is normally a performance hit with storing large quantities of data in relational databases. Also, SQL Express does not allow yuo to store more than 10GB per database. Nodinite overcomes all these limitations by simply spawning new Log Databases as they grow too large or even too old. When you use Nodinite we keep the knowledge of multiple Log Databases stored in the Configuration Database and resolves access internally so you don't even know from which underlying Log Database presented data originated from.
During first start of the Logging Service while performing the installation of Nodinite, a SQL Database prefixed NodiniteLog_ is created. This database has a different name for different environments (like NodiniteLog_Prod_, NodiniteLog_Test_ etc.). The Log Database is part of the Core Services package which means that the Nodinite Update Tool provides a script that an administrator must manually execute to update one or more Log Databases. See Prerequisites for pre-installation related information.
The Configuration Database interact, and depends on the Log Databases. The connection with these, uses the concept of a Linked Server using the configured security settings. The table in the Core Services Overview provides links to all SQL, Windows and firewall settings required for the other components. The 'Prerequisites' page further details this for the Log Database. Having the Log Databases on the same SQL Instance as the Configuration Database means less administration regarding firewalls, distributed transactions and so on.
Overview of Nodinite Configuration Database interacting with the Log Databases using a linked server
Sensitive data within the Log Databases are protected using an obfuscator. This means that a SQL DBA will not be able to read the content or context properties of messages like patient journals and salary payments.
Data is partitioned over time into multiple Log Databases. This feature improves performance, and provides the ability to store virtually infinite amounts of data horizontally, and vertically. Your hardware is the limit, not Nodinite
Given the proper access rights; Automatic Log archiving is built-into Nodinite. The Log Databases split on Size and/or Time. This is governed by the run time values found in the System Parameters table in the Configuration Database.
- SizeToSplitDatabaseOn - The maximum size (in GB) the active Log Database is allowed to reach until a new Nodinite Log Database should be created by the system.
- DaysToSplitDatabaseOn - The number of days since creation of the active Log databases before a new Log Database is created by the system.
- LogDatabaseName - The default naming convention in use creating new Log Databases
Large amounts of data and customers with high transaction count needs to scale the hardware and software configuration for the environment running Nodinite. See the 'Prerequisites' page for additional information.
If Nodinite is not given elevated rights, to avoid performance related issues; Your organization, from time to time, need to create new Log Databases with other, more or less automated tasks. If you create the databases on your own, please honour the naming scheme which is detailed in the LogDatabaseName user guide.
Once created; The Log databases must have the appropriate SQL Rights set. The two following System Parameters are used by the Logging Service to automtically set the configured access rights on the new Log Databases:
- ImLogAccessRoles - Database role(s) to grant the service user(s) to allow access to new Nodinite Log database(s)
- ImLogServiceUsers - Service user(s) to grant for new Nodinite Log database(s)
Next Step
Related
Log API
Logging Service
Monitoring Service