Only call SSL_shutdown if we have an SSL context.
This would otherwise cause trouble for listening sockets.