The request was aborted: A connection that was expected to be kept alive was closed by the server

it all started with this

The operation has timed out
at System.Net.HttpWebRequest.GetRequestStream()

first i thought this might be because of firewall, proxy, something local for our network...then we turned our attention to the counterparty - they chased their logs and couldn't find anything related to timeouts or connection drops for our endpoints...
then crazy testing began - the crazy thing was that all seemed to work on the local machined and in production, then it stopped working in production, and later it stopped working in dev also...nothing miraculous here - turned our we tested wrongly in dev, so all failed everywhere.
this code is a part of a nightly process so it quickly became a pain to wait and see if the nightly process worked and if not (most of the times) run the processes manually..pain pain pain...

so before this weekend i finally sat and decided to dedicate all my time to solving this fundamental problem

i had already set KeepAlive to false and this 'seemed' to work for some time...before coming back with fierce force

so the long chase on the net began...

the first step in the correct direction was to add .Net network tracing to the app, through this app.config setting:

<system.diagnostics>
<sources>
<source name="System.Net" tracemode="includehex" maxdatasize="1024">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Sockets">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
<source name="System.Net.Cache">
<listeners>
<add name="System.Net"/>
</listeners>
</source>
</sources>
<switches>
<add name="System.Net" value="Verbose"/>
<add name="System.Net.Sockets" value="Verbose"/>
<add name="System.Net.Cache" value="Verbose"/>
</switches>
<sharedListeners>
<add name="System.Net"
type="System.Diagnostics.TextWriterTraceListener"
initializeData="network.log"/>
</sharedListeners>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="file" type="System.Diagnostics.TextWriterTraceListener" initializeData="networktrace.log"/>
</listeners>
</trace>
</system.diagnostics>

this pinpointed the problem and helped develop a reproduce-case: the time-out reoccured every third time HttpWebRequest.GetRequestStream() was called - awesome, so now what?

well the trace at least gave me hints no to what too look for in google - the timeout occured after these lines

System.Net Information: 0 : [1696] Associating HttpWebRequest#38557668 with ServicePoint#62013521
System.Net Information: 0 : [1696] Associating Connection#12798926 with HttpWebRequest#38557668

which meant, there was a problem creating the socket?
whatever - the case continued...

many people were experiencing thi serror:
The underlying connection was closed: A connection that was expected to be kept alive was closed by the server

and offered as solution setting keepalive=false, which i already had in place.
there was also a link here:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;915599
which gave another hint:
playing with ServicePointManager.MaxServicePointIdleTime property

there was also a hint to set
webRequest.ProtocolVersion=HttpVersion.Version10
which i don't to play with right now

the last and seemingly, best solution was given by Darren Neimke
http://markitup.com/Posts/Post.aspx?postId=c98db0ca-f3a8-4f49-bb96-641574274c1a:

WebRequest req = HttpWebRequest.Create("http://foo");
req.ConnectionGroupName = Guid.NewGuid().ToString();

using (Stream stream = req.GetRequestStream()) {
// do stuff here
}

a saw the ConnectionGroupName approach also here:
http://blog.josh420.com/archives/2008/02/underlying-connection-closed-fix-pesky-aspnet-webexception.aspx

still testing....will this help me not see this log below anymore?

System.Net Verbose: 0 : [3828] HttpWebRequest#48833621::Abort(The operation has timed out)
System.Net.Sockets Verbose: 0 : [3828] Socket#3798230::Dispose()
System.Net Error: 0 : [1696] Exception in the HttpWebRequest#38557668:: - The request was aborted: A connection that was expected to be kept alive was closed by the server.
System.Net Verbose: 0 : [3828] Exiting HttpWebRequest#48833621::Abort()