ZF-10878: Zend_Http_Client: compressed streams unusable

Description

When fetching a file from a server supporting gzip or compress and sending it to a stream, the data is compressed and there is no handy way to uncompress/inflate it.

$client = new Zend_Http_Client( "http://some_url_on_a_server_with_gzip_support/some_text_file.txt" , array( "adapter" => "Zend_Http_Client_Adapter_Curl" , "storeresponse" => false , "timeout" => 60 ) );

// This works around the problem, but it's far from a solution! //$client->setHeaders("Accept-Encoding", "compress;q=0 gzip;q=0");

$reply = $client->setStream("target_file.txt")->request();

...

bash> file target_file target_file.txt: gzip compressed data

Comments

The response header:

object(Zend_Http_Response_Stream)#37 (8) { ["stream":protected] => resource(33) of type (stream) ["stream_name":protected] => string(20) "target_file.txt" ["_cleanup":protected] => NULL ["version":protected] => string(3) "1.1" ["code":protected] => int(200) ["message":protected] => string(2) "OK" ["headers":protected] => array(9) { ["Date"] => string(29) "Tue, 28 Dec 2010 22:05:57 GMT" ["Server"] => string(3) "off" ["Last-modified"] => string(29) "Tue, 28 Dec 2010 05:30:01 GMT" ["Etag"] => string(30) ""2c8171a-2346ff-49871be796f43"" ["Accept-ranges"] => string(5) "bytes" ["Vary"] => string(15) "Accept-Encoding" ["Content-encoding"] => string(4) "gzip" ["Connection"] => string(5) "close" ["Content-type"] => string(10) "text/plain" } ["body":protected] => string(0) "" }

It is possible that adding a gzip stream filter on the stream will help.

Sure, but that's kind of a hack...

My point is, that the file created by the Zend_Http_Client is not usable without ugly extra-code (check the response headers if compression was used, apply a stream filter, etc.)

The expected result should be a usable file without having to care about HTTP details.

The underlying problem seems to be that Zend_Http_Client is responsible for adding the Accept-Encoding headers, but Zend_Http_Response is responsible for dealing with a compressed response. This works out fine for normal usage, since the response body is accessed through Zend_Http_Response, but when setStream is used, the response data is streamed directly to the destination, without Zend_Http_Response being involved at all.

Using the zlib stream filters would let you deal with this, but it seems like something the framework would need to do for you; you can't know whether you need to inflate the output until you look at the headers, and as a user of Zend_Http_Client, you can't see the headers until the response has already been {{stream_copy_to_stream}}'d to its destination.

Of course, the reporter's workaround (or {{$client->setHeaders('Accept-Encoding', 'identity');}}) is the simplest workaround, but it'd be ideal to have both HTTP compression and streaming.