Fix pickling of httpclient.HTTPError subclasses and web.HTTPError

[andersk]

The args member variable is set by BaseException.__new__ and used by BaseException.__reduce__ for pickling. To avoid interfering with it, we need to avoid calling BaseException.__init__ from classes that have subclasses with incompatible constructors, and rename our own tornado.web.HTTPError.args member.

>>> pickle.loads(pickle.dumps(tornado.simple_httpclient.HTTPTimeoutError("message")))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: HTTPTimeoutError.__init__() takes 2 positional arguments but 4 were given
>>> str(pickle.loads(pickle.dumps(tornado.web.HTTPError(500, "%s", "foo"))))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/anders/python/tornado/tornado/web.py", line 2488, in __str__
    return message + " (" + (self.log_message % self.args) + ")"
                             ~~~~~~~~~~~~~~~~~^~~~~~~~~~~
TypeError: not enough arguments for format string

[bdarnell]

args is part of the de facto public interface of this class. We just jumped through some hoops to preserve backwards compatibility in this class in #3465. What exactly are the rules of pickling as applied to exceptions? (I've never knowingly used pickle and I'm surprised to learn that exceptions are pickleable)

[andersk]

args isn’t our interface to define; it was, and still is, part of the public interface of BaseException since before Tornado existed. It’s specified to be the tuple of arguments given to the exception constructor. Overriding it with any other meaning is a bug, and breaks pickling.

One common situation where exception pickling comes up is multiprocessing:

import multiprocessing, tornado

def f():
    raise tornado.simple_httpclient.HTTPTimeoutError("message")

with multiprocessing.Pool() as pool:
    pool.apply(f)

This not only fails to propagate the raised error, but in fact wedges the whole process while trying.

[bdarnell]

Right. It was a mistake to use self.args in this way, but it's a fifteen-year-old mistake now (I appreciate the link to the Python 2.5 docs since that was the current release back when the project that became Tornado was started). The question is what to do about it now. Pickling is the first known instance of this misuse of self.args actually causing problems, so is there a pickling-specific solution? Can we define __reduce__ and/or __setstate__ somehow to make this work, for example? Or do we have to make a breaking change to avoid this problem?