HttpClient provides a convenient way to make web requests in .NET. But if you want to cancel all pending requests using CancelPendingRequests or use the Timeout functionality, you must be aware of the gotchas.
I recently encountered some strange behaviour where some requests were not cancelled when CancelPendingRequests was called and some did not time out after the Timeout had elapsed. I couldn't find any documentation about this behaviour, so I explored it myself and discovered some strange behaviour.
Gotcha #1: CancelPendingRequests does not cancel all requests
I expect CancelPendingRequests to cancel all current requests on an HttpClient. But it does not. Its behaviour depends on which HttpCompletionOption you use.
If you run the following code on a url containing a large file, it will print Starting Download, and then after 3 seconds print Cancel followed by Cancelled and the download will stop. This is the expected behaviour of CancelPendingRequests.
using(var client = new HttpClient())
{
Task.Delay(3000).ContinueWith(t => {
Console.Out.WriteLine("Cancel");
client.CancelPendingRequests();
});
try
{
Console.Out.WriteLine("Starting Download");
using(var result = await client.GetAsync(url, HttpCompletionOption.ResponseContentRead))
{
return await result.Content.ReadAsByteArrayAsync();
}
}
catch
{
Console.Out.WriteLine("Cancelled");
return new byte[0];
}
}
But if you change the HttpCompletionOption to HttpCompletionOption.ResponseHeadersRead and run the code, it will print Starting Download, and then after 3 seconds print Cancel, but it will not print Cancelled and the download will continue. This is unexpected behaviour for CancelPendingRequests.
Gotcha #2: Timeout does not apply to all requests
I expect that a request will be cancelled once the Timeout value has elapsed. But it does not. Like CancelPendingRequests, its behaviour depends on which HttpCompletionOption you use.
If we remove the code that cancels pending requests:
Task.Delay(3000).ContinueWith(t => {
Console.Out.WriteLine("Cancel");
client.CancelPendingRequests();
});
And replace it with code to set a timeout:
client.Timeout = TimeSpan.FromSeconds(3);
We get the same results as we did with CancelPendingRequests. The Timeout works as expected when HttpCompletionOption.ResponseContentRead is used and gives the unexpected behaviour when HttpCompletionOption.ResponseHeadersRead is used.
Analysis of behaviour
When you use HttpCompletionOption.ResponseHeadersRead instead of HttpCompletionOption.ResponseContentRead, you're saying "I want more control".
You might use this extra control to stream directly to disk, to abort large downloads early based on the response headers or the response content, or to implement a sliding time out.
As Timeout is global to all requests on an HttpClient, it makes sense to be able to override it on some requests. The requests where you might want to override it are the requests where you take more control, so to some extent it is understandable that this is linked to HttpCompletionOption.ResponseHeadersRead. However, I would expect to see this documented on HttpCompletionOption and Timeout and ideally would like it to be a separate self-documenting parameter to GetAsync for those times when you want more control, but still want a time out.
I cannot think of any reason to opt-out of CancelPendingRequests on some requests. However, I expect both CancelPendingRequests and Timeout may have been difficult to implement or non-performant when you take low-level control over the request using HttpCompletionOption.ResponseHeadersRead. Even so, I feel this behaviour should still have been documented.
Solution: Implement cancellation yourself
Using a CancellationTokenSource you can implement cancellation yourself, but you'll need to read the stream manually too. The following code demonstrates this. When run, it prints Starting Download, and then after 3 seconds prints Cancel followed by Cancelled and the download stops.
using(var client = new HttpClient())
using(var cts = new CancellationTokenSource())
{
Task.Delay(3000).ContinueWith(t =>
{
Console.Out.WriteLine("Cancel");
cts.Cancel();
});
try
{
Console.Out.WriteLine("Starting Download");
using(var result = await client.GetAsync(url, HttpCompletionOption.ResponseHeadersRead, cts.Token))
using(var output = new MemoryStream())
using(var stream = await result.Content.ReadAsStreamAsync())
using(cts.Token.Register(() => stream.Close()))
{
byte[] buffer = new byte[80000];
int bytesRead;
while((bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length)) != 0)
{
output.Write(buffer, 0, bytesRead);
}
return output.ToArray();
}
}
catch
{
Console.Out.WriteLine("Cancelled");
return new byte[0];
}
}
To duplicate the functionality of CancelPendingRequests, create a CancellationTokenSource with the same lifetime as HttpClient and call Cancel on the CancellationTokenSource when you would call CancelPendingRequests.
To duplicate the functionality of Timeout, create a CancellationTokenSource for each request using the constructor that takes a TimeSpan to define the cancellation delay.
To do both, combine the approaches by creating a linked token source using CancellationTokenSource.CreateLinkedTokenSource. When doing this, remember to dispose the linked token source in addition to the underlying token sources. The linked token source creates a registration with the underlying token sources and therefore failure to dispose the linked token source will create a memory leak if one of the underlying token sources is long lived.
Unfortunately, it is necessary to forcefully close the stream via the token registration, because although you can pass the CancellationToken to stream.ReadAsync, this has no effect. stream.ReadAsync only checks for cancellation when first called and then delegates to lower-level code that does not support cancellation. If this lower-level code is waiting for bytes that never arrive, cancellation may never occur. This lower-level implementation may change in the future, in which case passing the CancellationToken to stream.ReadAsync will be preferable.
