Near the start of the year, we had a number of RedHat 6.5 servers apply a set of platform updates. These updates included an OpenSSL update.
We then began to see a number of cross-platform issues when an application on the unix servers interacted with a service hosted on a Windows server over an encrypted channel. On the Windows side, the primary two services involved were Active Directory (LDAPS) and IIS (HTTPS). The error message on the client side was usually “Operations error”. Network packet inspection showed some funny business, including the Windows server closing the LDAP connection without notification to the client.
In some cases, specific to the AD involved issue, some folks have successfully implemented a workaround of changing their TCP port to 3268, following *many* online posts which suggest this as a solution. That workaround has two potential problems:
- The GC port is a subset of all directory info. So you get some info, but not all of it. And you have no inkling that you are missing something. But if the GC has all the info you need, then no problem.
- The GC port is read-only. No writes. If you don’t need to write, then no problem.
It’s taken us quite a while to troubleshoot what’s going on here, especially since there is an OS platform divide. There’s also confusion aplenty in how TLS relates to SSL. 🙂
We initially believed that OpenSSL changed the default negotiation version to SSLv2. And that something about the Windows implementation of SSLv2 is incompatible with the OpenSSL implementation. Obvious workarounds based on that belief are forcing the unix client to choose SSLv3 and turning off SSLv2 on the Windows server, both good outcomes outside of this issue.
We’ve been deliberate in not jumping to turn off SSLv2 on the Windows servers because the impact isn’t clear. http://support.microsoft.com/kb/245030 talks about how to go about doing that.
For the unix client —HTTPS—> IIS case, we’ve successfully had the client choose SSLv3, and resolved those incidents.
For the unix client —LDAPS—> AD case, we had more problems, mostly because openldap adds some complexity.
After much hair-pulling, we finally figured out that the openldap client doesn’t handle LDAP referrals after a LDAP bind well–even if they are configured to not follow referrals. Invisible to us, our domain began issuing some new LDAP referrals after we moved the DNS zone for the domain to AD-integrated DNS. The openldap client configuration needed to be updated to use the ldap_set_rebind_proc() function to allow an immediate rebind after connecting and receiving referrals.
The OpenSSL changes were red-herrings that threw us off and SSLv2 had nothing to do with the AD issue.
Hope this helps someone else. Especially folks who are getting the most common advice (which isn’t good advice) on the web for this which is to switch to port 3268 (which doesn’t suffer the same problem because it doesn’t issue those referrals).