Update HttpSys shutdown logic

[BrennanConroy]

Updated HttpSys's shutdown logic to be more like Kestrels.
IServer.Dispose calls IServer.StopAsync with an already canceled token.
StopAsync now waits for the accept loops to complete, before it only waited for active requests to finish which left open a race where we dispose the request queue handle while the accept loops were still active.

In the ungraceful close case, we still wait for the accept loops to complete, but don't wait for any blocked request processing to complete (imagine app logic taking a long time to finish). The only case where this would be slightly problematic is when using UnsafePreferInlineScheduling and app code was holding onto request processing, then the accept loop wouldn't complete and shutdown would hang until the app code completed.

This should hopefully reduce/eliminate problems we've seen in test code with handles being invalid when they shouldn't be.

[halter73]

Do we really want to wait for the accept loops in the abortive new CancellationToken(canceled: true) case?

[BrennanConroy]

I think so. In normal cases (not using inline scheduling) they should be very quick to complete as they just call HttpReceiveHttpRequest which either returns with a request immediately which we dispatch off the accept loop, goes async which gets off the accept loop, or errors due to us closing the request queue.

Edit: I also think Kestrel does this too, waits for its accept loops to finish.

[Copilot]

Pull request overview

This PR refactors HttpSys's shutdown logic to align with Kestrel's approach. The key improvement is ensuring that StopAsync waits for accept loops to complete before disposing resources, eliminating a race condition where the request queue handle could be disposed while accept loops were still active.

Changes:

• Modified MessagePump.StopAsync to track and wait for accept loops to complete before disposing the listener, preventing invalid handle usage
• Added RequestQueue.StopProcessingRequests method that calls HttpShutdownRequestQueue to gracefully cancel pending accept operations without closing the handle
• Changed MessagePump.Dispose to call StopAsync with an already-cancelled token for ungraceful shutdown
• Updated test utilities and test expectations to accommodate the new shutdown behavior where cancelled shutdowns abort in-flight requests

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/Servers/HttpSys/src/MessagePump.cs	Implements accept loop tracking with counters and completion sources, refactors StopAsync to wait for accept loops, and updates Dispose to use StopAsync
src/Servers/HttpSys/src/NativeInterop/RequestQueue.cs	Adds StopProcessingRequests method calling HttpShutdownRequestQueue, updates Dispose to use SetHandleAsInvalid instead of Dispose
src/Servers/HttpSys/src/NativeMethods.txt	Adds HttpShutdownRequestQueue native method declaration
src/Servers/HttpSys/test/FunctionalTests/Utilities.cs	Adds configureOptions parameter to CreateHttpServerReturnRoot for test flexibility
src/Servers/HttpSys/test/FunctionalTests/ServerTests.cs	Updates test expectations from successful responses to expecting exceptions when shutdown is cancelled
src/Servers/HttpSys/test/FunctionalTests/RequestTests.cs	Updates all test calls to pass configureOptions parameter, simplifies one test by removing try-catch, adds optimization comment for MaxAccepts setting

[Copilot]

The StopProcessingRequests method should check if the request queue is already disposed before calling HttpShutdownRequestQueue. Without this check, calling this method on a disposed instance could result in operating on an invalid handle. Consider adding CheckDisposed() at the beginning of this method, similar to how SetLengthLimit and SetRejectionVerbosity handle disposed state.

Suggested change

	{
	{
	CheckDisposed();

[DeagleGross]

why are we not interested in reporting error here anymore?

[BrennanConroy]

Before, the Assert.Equal(string.Empty, response) check would truncate the error so all you'd see is something like "XUnitAssertException". Now SendRequestAsync will throw with the XUnit exception which will let us see the actual error.