XML-RPC over SSL in Python

As part of one of the top-secret projects being developed at the Anchor Weapons Research Facility (AWRF, like a dog), I recently needed to find a secure way for some of the computers on our network to ask other computers to do things and tell me what happened. Helpfully Python offered a way to do "stuff" securely over a network, and it offered me a way to ask other computers to do stuff; alas it offered no way to harmoniously combine these goals. Where there's a will there's a way though, and I'm going to show you how I did it.

First of all, just in case I left anyone behind in the title, XML-RPC is a Remote Procedure Call system that happens to talk in eXtended Markup Language over the network. XML-RPC however is not a secure protocol, and we really need security. SSL stands for Secure Sockets Layer, it's a library that lets you turn your old, insecure network code into secure network code without doing any of the work.

Before we go any further though, here's the code. If you just need to setup a secure XML-RPC server in Python, this will let you do it and it's only the SecureXMLRPCServer class you need to look at, along with changing KEY_FILE and CERT_FILE to point to SSL certificates for your site.

#!/usr/bin/python

'''
Hacked up implementation of XMLRPC over an SSL transport.
'''

from OpenSSL import SSL
import SocketServer, socket, SimpleXMLRPCServer, pprint, time, sys, config

KEY_FILE = config.SSL_KEY_FILE
CERT_FILE = config.SSL_CERT_FILE

class ConnWrapper :
    '''
    Base class for implementing the rest of the wrappers in this module.
    Operates by taking a connection argument which is used when 'self' doesn't
    provide the functionality being requested.
    '''
    def __init__(self, connection) :
        self.connection = connection

    def __getattr__(self, function) :
        return getattr(self.connection, function)

class SSLConn2File(ConnWrapper):
    '''
    Wrapper for SSL.Connection that makes it look like a file object for use
    with xmlrpclib.
    '''
    def __init__(self, connection):
        ConnWrapper.__init__(self, connection)
        self.buf = ''
        self.closed = False

    def flush (self) :
        pass

    def close(self) :
        self.closed = True

    def readline(self, length=None) :
        # inspect the internal buffer, if there's a newline there, use that.
        # else, we need to fill the buffer up and do the same thing.
        def bufManip(location) :
            '''
            Update self.buf as if there were a newline at the offset @location.
            Then return the string up to the newline.
            '''
            result = self.buf[0 : location + 1]
            self.buf = self.buf[location + 1:]
            return result

        start_time = time.time()

        nl_loc = self.buf.find('\n')
        if nl_loc != -1 :
            return bufManip(nl_loc)

        segments = []
        while self.buf.find('\n') == -1 :
            newdata = self.connection.recv(4096)
            if len(newdata) :
                segments.append(newdata)
                if newdata.find('\n') != -1 :
                    self.buf += ''.join(segments)
                    nl_loc = self.buf.find('\n')
                    return bufManip(nl_loc)
            else :
                # if we get to here, it means we weren't able to fetch any
                # more data with newlines (probably EOF). Just return the
                # contents of the internal buffer.
                self.buf += ''.join(segments)
                return self.buf

class SSLConnWrapper(ConnWrapper):
    '''
    Proxy class to provide makefile function on SSL Connection objects.
    '''
    def __init__(self, connection) :
        ConnWrapper.__init__(self, connection)

    def makefile(self, mode, bufsize = 0) :
        (_, _) = (mode, bufsize)
        return SSLConn2File(self.connection)

    def shutdown(self, _) :
        return self.connection.shutdown()

class SSLWrapper(ConnWrapper):
    '''
    Proxy class to inject the accept method to generate the *next* proxy class.
    '''
    def __init__(self, connection) :
        ConnWrapper.__init__(self, connection)

    def accept(self) :
        (conn, whatsit) = self.connection.accept()
        return (SSLConnWrapper(conn), whatsit)

class SSLSocketServer(SocketServer.ThreadingMixIn, SocketServer.BaseServer) :
    '''
    Hack to provide SSL over the existing TCPServer class.
    '''
    address_family = socket.AF_INET
    socket_type = socket.SOCK_STREAM
    request_queue_size = 5
    allow_reuse_address = True

    def __init__(self, server_address, RequestHandlerClass) :
        '''Constructor. We're overloading it, even though TCPServer's
        constructor says not to.'''
        SocketServer.BaseServer.__init__(self, server_address, RequestHandlerClass)

        # this function should do the authentication checking to see that
        # the client is who they say they are.
        def verify_cb(conn, cert, errnum, depth, ok) :
            return ok

        # setup an SSL context.
        context = SSL.Context(SSL.SSLv23_METHOD)
        context.set_verify(SSL.VERIFY_PEER, verify_cb)

        # load up certificate stuff.
        context.use_privatekey_file(KEY_FILE)
        context.use_certificate_file(CERT_FILE)

        # make the socket
        real_sock = socket.socket(self.address_family, self.socket_type)
        self.socket = SSLWrapper(SSL.Connection(context, real_sock))

        self.server_bind()
        self.server_activate()

    # functions after here are copied directly from TCPServer, just to avoid
    # a pychecker error.
    def server_bind(self) :
        if self.allow_reuse_address:
            self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.socket.bind(self.server_address)

    def server_activate(self) :
        self.socket.listen(self.request_queue_size)

    def server_close(self) :
        self.socket.close()

    def fileno(self) :
        return self.socket.fileno()

    def get_request(self) :
        return self.socket.accept()

    def close_request(self, request) :
        request.close()

class SecureXMLRPCServer(SSLSocketServer, SimpleXMLRPCServer.SimpleXMLRPCDispatcher) :
    '''
    Hacked up SSL-aware XMLRPC server.
    '''
    def __init__(self, addr,
            requestHandler=SimpleXMLRPCServer.SimpleXMLRPCRequestHandler,
            logRequests=1):
        self.logRequests = logRequests

        SimpleXMLRPCServer.SimpleXMLRPCDispatcher.__init__(self)
        SSLSocketServer.__init__(self, addr, requestHandler)

How does it work?

There actually isn't a lot of magic in this code though it does look complicated. The key problem is that Python's sockets don't behave in quite the same way that SSL connections do. Specifically, sockets have some convenience methods for reading and writing data which are extensions over the standard C API. To get around this problem I've used the __getattr__ method to intercept calls to those methods and have implemented versions of them that use only the standard SSL APIs.

The more observant of your will probably realise that ordinarily you could just use inheritance to achieve my nefarious ends. The reason for intercepting calls with __getattr__ is that the Python wrappers for the SSL are not real objects; so they can't be overloaded using the standard techniques.

If you need to use this method, there isn't much to know. The __getattr__ method will be called on an object, if the standard method resolution mechanisms fail to find an attribute with the given name. In that case, the job of finding the relevant attribute is handed off to the __getattr__ method, which can do whatever it wants to find an implementation of the attribute with the given name. You can even generate the attributes on demand. It makes implementation of proxy classes really simple. The 4 line ConnWrapper class is honestly all you need.

Got some interesting uses for class proxying or some questions about it? Why not post about in our Online forum?