Wednesday, November 21, 2012

SYN_SENT hitting firewall, 2 minutes timeout

I am trying to connect to a Oracle RAC instance protected by a firewall.

jdbc_url='jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=theDNSentryforRAC.acme.com)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=srv_osb)))'
username='soainfra'
password='soainfra1'
driver = "oracle.jdbc.OracleDriver"
from com.ziclix.python.sql import zxJDBC
conn = zxJDBC.connect(jdbc_url, username, password, driver)

this request times out after 2 minutes, with an error

DatabaseError: The Network Adapter could not establish the connection [SQLCode: 20], [SQLState: 61000]

in the meantime, I monitor the connection with:
netstat -an | grep 1521
tcp 0 1 ::ffff:10.56.5.185:58597 ::ffff:10.56.13.112:1521 SYN_SENT

the IP 10.56.13.112 corresponds to one of the 2 RAC instances, and the initial :ffff represents a IPv4 address translated into IPv6

As very welle explained here http://www.faqs.org/docs/iptables/tcpconnections.html, the SYN_SENT is the first phase of the handshake to establish a connection, and its default timeout is 2 minutes.

The interesting thing is that if I specify port 1552, on which NOTHING is listening, the error
DatabaseError: The Network Adapter could not establish the connection [SQLCode: 20], [SQLState: 61000]

is returned IMMEDIATELY, which makes me think that I can actually contact the DB, but for some reason the DB is unable to send me back the SYN-ACK. Still investigating.

The connection waiting for an SYN-ACK is called an "embrionic connection" http://en.wikipedia.org/wiki/TCP_half-open

http://www.ibm.com/developerworks/aix/library/au-aixnetworkproblem2/index.html : a status of SYN_SENT indicates that a three-way handshake has been initiated by your host, but as yet no acknowledgement has been received from the target host. This could mean that there's a route to the target but no route back for this type of traffic. In this situation, ask the network administrator whether any firewalls on the route back are blocking this type of traffic.

No comments: