curl_httpclient: Future.cancel() closes connection

[nicholascorey]

The user just needs to cancel the Future passed by the fetch() function, and it'll properly close the connection. This works on curl_httpclient. Simple_httpclient hasn't been implemented.

[bdarnell]

Thanks for the contribution, and I apologize for being so slow to respond. However, I think there are issues with performance that need to be addressed before this capability can be merged.

Also, new functionality like this needs a test, and I'd really like to have it implemented for both HTTP clients instead of being curl-only.

[bdarnell]

Looping over every pending request each time any request finishes makes the entire thing quadratic, which isn't good. In order to support cancellation properly we need to make it event-driven: add a callback to the future that fires when it is cancelled, instead of periodically calling cancelled() to check the status. (I think there might be more idiomatic ways to implement this for simple_httpclient which is already coroutine-based)

[nicholascorey]

All great feedback, thanks! Was desperate for a solution and had just learned about co-routines hours before writing this. I noticed some checks failed. Any insight on that? I did a somewhat deep dive, but was having trouble understanding what the errors even meant.

[ploxiln]

Looks like the main test failure is just:

ERROR: test_error_after_cancel (tornado.test.curl_httpclient_test.CurlHTTPClientCommonTestCase)
...
Exception: did not get expected log message

That test is pretty straightforward:

    def test_error_after_cancel(self):
        fut = self.http_client.fetch(self.get_url("/404"))
        self.assertTrue(fut.cancel())
        with ExpectLog(app_log, "Exception after Future was cancelled") as el:

The other failing bits are formatting checks from flake8 and black