Preventing PingAccess Connection Timeouts

As enterprises continue to focus on securing their on-premise, cloud and mobile applications, the move to centralize access and access policies is driving demand for solutions like PingAccess from Ping Identity. PingAccess provides security controls to web applications and application programming interfaces (APIs) through identity-based access policies by proxying traffic between clients (typically web browsers) and back-end application or web servers without the use of agents. This lightweight approach to web access management makes it flexible, scalable, and transparent to web clients. During several recent deployments for PEGRight customers, I have developed a few configuration best practices and wanted to pass them along.

Inside PingAccess, access controls are applied to Virtual Hosts to protect multiple domain names in a single instance. When configuring a PingAcess Virtual Host to connect to services hosted by Apache httpd (Hypertext Transfer Protocol Daemon) web servers, you may experience “Internal Server” or “Service Unavailable” errors in the web browser. HTTP traces show that the response is an HTTP 500 error (which is a common general web server error) returned by the PingAccess JBOSS web service. The pingaccess.log file will contain something similar to the following:

com.pingidentity.pa.core.transport.http.HttpClient:114 - Server connection to  terminated before line was read. Line so far: 
[date] [time] DEBUG [tid] com.pingidentity.pa.core.transport.http.conn.ConnectionImpl:112 - Closing HTTP connection to :443 from local port xxxxx
[date] [time] DEBUG [tid] com.pingidentity.pa.core.interceptor.HTTPClientInterceptor:126 - Retrying HTTP call for proxy ' [] / /*:3000' after exception
[date] [time] DEBUG [tid] com.pingidentity.pa.core.interceptor.HTTPClientInterceptor:127 - Stack Trace java.io.IOException: Server connection terminated prematurely

This issue is likely caused by a mismatch in the site pool “Keep Alive Timeout (ms)” setting between the PingAccess Virtual Host and the backend web server. If the PingAccess Virtual Host "Keep Alive Timeout (ms)," used to protect the backend server is greater than the KeepAliveTimeout setting in Apache httpd, this error can occur. The default KeepAliveTimeout setting in Apache 2.2 or newer is 5 seconds (5000ms if milliseconds postfix is applied). For PingAccess, the default is 30000ms or 30 seconds, thus the mismatch.

In Apache, the Timeout directive specifies the amount of time Apache will wait for a GET, POST, or PUT request and subsequent ACKs. If the KeepAlive directive is set to On (default), the KeepAliveTimeout value is applied to keep channels open for the period of time specified in the KeepAliveTimeout option. Apache recommends that for better performance, turn on the KeepAlive directive to allow more than one request per connection, and to set the KeepAliveTimeout value low to reduce the number of resources utilized on the server. These settings are configured in the httpd.conf file, located in the directory /etc/httpd/conf by default.

To resolve this issue, it is best to configure the PingAccess timeout to be equal to or less than the KeepAliveTimeout setting in Apache. Alternatively, the KeepAliveTimeout can be adjusted to be greater than the setting configured in PingAccess.

The following sequence flow diagram illustrates how PingAccess uses proxy services to maintain connection timeout values.

Prevent PingAccess Connect Timeout

In this diagram, the client is assumed to be a web browser, PingAccess is acting as the proxy, and the Server is an Apache httpd web server, although this principle applies to any backend web server. By default, when PingAccess opens a connection to the Apache httpd server on behalf of the client, it reserves the connection for 30 seconds. However, on the backend application Apache server, the connection by default is opened for only 5 seconds. The initial connection occurs without error, but within 5 seconds, the connection is closed on the Apache server, but left open on the PingAccess server. When the client makes subsequent attempts to access the backend server after the Apache server closes the connection, the PingAccess server assumes that the connection is no longer available as it is no longer able to establish new connections over the same channel. As a result, clients may receive “Service Unavailable” errors.

Following these tips for configuring PingAccess connections will reduce the chance for errors. I’ll be writing about best practices and troubleshooting tips uncovered while working with clients who are deploying PingFederate and PingAccess, so stay tuned to this space. Contact us if you have any questions or are looking for help meeting your 2016 IT security goals..

Rate this blog entry:
4