Skip to content

Comments

Fix parse error at EOF#207

Merged
seliopou merged 7 commits intoinhabitedtype:masterfrom
dhouse-js:surface-parse-errors-at-eof
Jun 18, 2021
Merged

Fix parse error at EOF#207
seliopou merged 7 commits intoinhabitedtype:masterfrom
dhouse-js:surface-parse-errors-at-eof

Conversation

@dhouse-js
Copy link
Contributor

@dhouse-js dhouse-js commented Apr 29, 2021

Currently, if there is a parse error after read_eof, the runtime has no
chance to discover it. The error is not surfaced immediately, and if
the runtime calls next_read_operation, it just gets `Close.
This is because Reader.next first checks to see if it is closed, and
if so, always returns `Close.

This feature fixes this by inverting the logic in that function.

Unfortunately this leads to another problem: the parser effectively
expects an unending stream of requests, because after finishing
parsing one request it immediately expects the next one. This means
that read_eof is guaranteed to lead to a parse error. Thankfully this
is easily fixed by treating an empty input as a valid as well.

dhouse-js and others added 3 commits June 2, 2021 17:58
Currently, if there is a parse error after [read_eof], the runtime has no
chance to discover it. The error is not surfaced immediately, and if
the runtime calls [next_read_operation], it just gets [`Close].
This is because [Reader.next] first checks to see if it is closed, and
if so, always returns [`Close].

This feature fixes this by inverting the logic in that function.

Unfortunately this leads to another problem: the parser effectively
expects an unending stream of requests, because after finishing
parsing one request it immediately expects the next one. This means
that read_eof is guaranteed to lead to a parse error. Thankfully this
is easily fixed by treating an empty input as a valid as well.
Instead of adding complexity to the parser, it seems better to me to
simply not invoke `start` when we have no bytes to feed to the parser.
The tests still pass with this.
@dpatti dpatti force-pushed the surface-parse-errors-at-eof branch from d1f7836 to 92a68e5 Compare June 2, 2021 22:52
@dpatti
Copy link
Collaborator

dpatti commented Jun 2, 2021

I reverted the parser change and instead changed it so that we don't even try to start parsing if we're at a request boundary and are fed 0 bytes. I think this is equivalent but maybe a bit cleaner since empty inputs shouldn't really be valid requests.

That said, I expanded the test and found that this is yet another case of writing after close. This happens even without the parser change I made. Seeing this for the third time in PRs makes me wonder if this is already prevalent and we just don't know it.

@seliopou
Copy link
Member

seliopou commented Jun 2, 2021

Build's busted.

@dpatti
Copy link
Collaborator

dpatti commented Jun 3, 2021

Yeah, that's intentional

dhouse-js and others added 4 commits June 3, 2021 07:47
For now let's just accept that this is possible, since I believe it is
in other situations already.
The error handler in the latest test no longer raises that the writer is
closed. There are a few things going on in this fix:

- In `Reader.read_with_more`, we closed the reader when something was
  fed when `more = Complete`. This isn't entirely accurate, though -- it
  meant that a single `read_eof` with multiple requests would enter this
  branch multiple times because `Server_connection.read_with_more` was
  recursive and might call into the reader with the same buffer multiple
  times. Now we only close the reader once we've actually exhausted the
  bytes and hit eof.

- In `_next_read_operation`, the first thing we do is check if we have
  no request and the reader is closed. In that case, we should be safe
  to shutdown the connection. One minor note is that this was really
  equivalent to `shutdown_writer` since we are basically testing that
  the reader is shut down. But the more complicated thing is that just
  because the `request_queue` is empty doesn't mean there isn't pending
  handler work -- specifically, when the `error_handler` is invoked
  without a `?request`. This happens during connection and parse errors,
  so we should make sure to keep the writer open until that handler is
  resolved.

Minor changes:

- `is_waiting` was unused
- Added more tracing that I found helpful while debugging tests
@dpatti
Copy link
Collaborator

dpatti commented Jun 6, 2021

Okay, I revisited this because I ran into some related issues in the upgrade PR. I made some more extensive changes, which I described in the commit message, pasted below. I think this should be the next thing we merge. I think I would split things up a bit more, but I think the PR overhead is getting to be a lot, especially with the constant test merge conflicts.


The error handler in the latest test no longer raises that the writer is
closed. There are a few things going on in this fix:

  • In Reader.read_with_more, we closed the reader when something was
    fed when more = Complete. This isn't entirely accurate, though -- it
    meant that a single read_eof with multiple requests would enter this
    branch multiple times because Server_connection.read_with_more was
    recursive and might call into the reader with the same buffer multiple
    times. Now we only close the reader once we've actually exhausted the
    bytes and hit eof.

  • In _next_read_operation, the first thing we do is check if we have
    no request and the reader is closed. In that case, we should be safe
    to shutdown the connection. One minor note is that this was really
    equivalent to shutdown_writer since we are basically testing that
    the reader is shut down. But the more complicated thing is that just
    because the request_queue is empty doesn't mean there isn't pending
    handler work -- specifically, when the error_handler is invoked
    without a ?request. This happens during connection and parse errors,
    so we should make sure to keep the writer open until that handler is
    resolved.

Minor changes:

  • is_waiting was unused
  • Added more tracing that I found helpful while debugging tests

@dpatti dpatti requested a review from seliopou June 6, 2021 18:41
@dhouse-js
Copy link
Contributor Author

@dpatti are you saying you want to merge this before my PR which changes the type of Body.flush to indicate when things have been closed? (That PR doesn't exist yet, but I'm working on it.) I thought you had previously been concerned that merging this PR would increase the chance of the user hitting the "cannot write to closed writer" error.

@dpatti
Copy link
Collaborator

dpatti commented Jun 8, 2021

@dhouse-js That's right. In my last commit I fixed it so that we would never run into that scenario, at least in this PR

@seliopou seliopou merged commit 6d47c52 into inhabitedtype:master Jun 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants