PingFederate Database Logging for Performance

Many software applications implement logging as a core feature to assist with troubleshooting, as well as maintain audit records. Logging however comes at the expense of the systems overall performance, as the logging requires system resources. Managing log files can be a headache as well to ensure disk space is not consumed causing software application issues. The key questions we come across in our consulting is what strategies are best for the management of log files, how often to roll the log files, and how long to keep the log files around? We explore PingFederate logging in this article and how to improve performance.

Software logging does require system resources, in particular the actual writing of the log file does consume resources. With multiple application threads attempting to write to the same log file, access must be synchronized such that threads do not interfere with each other, essentially creating a single threaded logging process. PingFederate has been designed to support concurrent threads writing to log files such as the server.log and audit.log. As the PingFederate server processes an increasing number of transactions, contention to the log files also increases thus contributing to degradation of performance since each thread must wait to write to the log file.

One strategy to overcome the degradation in performance is to decrease the amount of logging. We find that this is not very practical for support as the log files are the source of truth with respect to the system, as well as the main source of information for troubleshooting. The best method is to log to a database rather than to the file system (e.g. PingFederate default configuration). Databases are optimized for concurrent access and can more easily support multiple threads/clients writing information concurrently. Logging to a database also has the benefit of implementing a single repository for all PingFederate runtime engines.

PingFederate provides a simple way to solve these problems in your datacenter. The Ping Identity Knowledge Center provides instructions for configuring your environment to log to a database. PingFederate supports MySQL, Oracle, and Microsoft SQL Server for database logging. You will need to perform these steps on each of your PingFederate engine nodes.

PingFederate logging to database

Getting back to some of the key questions we encounter with respect to logging.

  • What strategies are best for the management of log files? Logging to a database is always a good strategy as it centralizes log records and improves performance.
  • How often to roll the log files?  When logging to a database, this comes down to a database retention policy for information.
  • How long to keep the log files around? For audit log files, it is quite common to keep a minimum of 90 days of audit records and have the maximum defined by the amount of disk capacity. The answer for the maximum number of days for audit log file online retention is defined in the corporate data retention policy, where we have seen 2 to 3 years. For server log files, we have found anywhere between 3 and 10 days to be sufficient for supporting incident reports. Server log files tend to be very large and thus disk space and number of days retention is engineered as well as driven by the incident reporting timelines.

I hope you found this article helpful and utilize the recommendation to leverage a database for logging when additional performance is needed within your PingFederate deployment. If you have comments or questions, please contact us.

Rate this blog entry:
2