Zend Framework

Serializer doesn't support cyclic reference

Details

  • Type: Bug Bug
  • Status: Resolved Resolved
  • Priority: Blocker Blocker
  • Resolution: Fixed
  • Affects Version/s: 1.7.7
  • Fix Version/s: 1.8.4
  • Component/s: Zend_Amf
  • Labels:
    None

Description

The class : Zend_Amf_Parse_Amf3_Serializer doesn't support cyclic references. (I didn't test the Zend_Amf_Parse_Amf0_Serializer).

Problem sample :
class ServiceProvider
{
/**

  • @return TestObject
    */
    public function testGetRecursiveObject() { $o = new TestObject(); $o->recursive = new TestObject(); $o->recursive->recursive = $o; return $o; }

    }

When you call testGetRecursiveObject() with an AMF request, the server doesn't respond. What is worse is that : on Win Vista the apache server stops running.
This problem is critical : no error is returned by the server.

The solution to solve this problem, is to use the "reference system" described in the AMF3 specifications.

I've tried to solve the problem myself but unfortunately without success : I think i didn't write the good code for writing the object reference on the output stream.

Here is the patch :

Index: Serializer.php
===================================================================
— Serializer.php (revision 14591)
+++ Serializer.php (working copy)
@@ -35,6 +35,8 @@
*/
class Zend_Amf_Parse_Amf3_Serializer extends Zend_Amf_Parse_Serializer

{ + private $_references = array(); + /** * Serialize PHP types to AMF3 and write to stream * @@ -250,6 +252,24 @@ return $this; }

+ public function writeReference($object)
+ { + $this->_stream->writeByte(0x07); + $ref = & $object; + $ref = $ref << 1; + $this->writeInteger($ref); + }
+
+ private function referenceExist($object)
+ {
+ if(in_array($object, $this->_references, true)) { + return true; + } else { + $this->_references[] = $object; + return false; + }
+ }
+
/**

  • Write object to ouput stream
    *
    @@ -258,6 +278,11 @@
    */
    public function writeObject($object)
    {
    + if($this->referenceExist($object)) { + $this->writeReference($object); + return $this; + }
    +
    $encoding = Zend_Amf_Constants::ET_PROPLIST;
    $className = '';
  1. 009-Incubator-Zend_Amf_Parse_Amf0_Serializer.patch
    09/Jun/09 12:57 AM
    6 kB
    Stefan Klug
  2. main.mxml
    20/Apr/09 1:28 AM
    1 kB
    Mikaël LABRUT
  3. patch_serializer_v2.patch
    27/Apr/09 1:20 AM
    1 kB
    Mikaël LABRUT
  4. patch_serializer.patch
    17/Apr/09 9:42 AM
    1 kB
    Mikaël LABRUT

Activity

Hide
Mikaël LABRUT added a comment -

This patch correct the problem

Show
Mikaël LABRUT added a comment - This patch correct the problem
Hide
Mikaël LABRUT added a comment -

I think that I solve the problem.

You can try my patch in attachment.

Show
Mikaël LABRUT added a comment - I think that I solve the problem. You can try my patch in attachment.
Hide
Wade Arnold added a comment -

Confirmed that the patch does not break any of the old unit tests. Creating unit tests that expose the additional functionality.

Show
Wade Arnold added a comment - Confirmed that the patch does not break any of the old unit tests. Creating unit tests that expose the additional functionality.
Hide
Wade Arnold added a comment -

First I have confirmed that cyclic references do not work inside of ZendAmf and crash the serialize. However after testing your patch with the below service I no longer received any PHP errors but I never received a response from or a valid AMF byte-stream in flex. When trying to debug with Charles Proxy it actually takes down the application. This usually happens with malformed AMF responses. Where you able to make it work?

Thanks!

Recursive.php
<?php 
class Recursive {
    
    public function testGetRecursiveObject() { 
        $o = new TestObject(); 
        $o->recursive = new TestObject(); 
        $o->recursive->recursive = $o; 
        return $o; 
    }
}

class TestObject {
    public $recursive;
}
Show
Wade Arnold added a comment - First I have confirmed that cyclic references do not work inside of ZendAmf and crash the serialize. However after testing your patch with the below service I no longer received any PHP errors but I never received a response from or a valid AMF byte-stream in flex. When trying to debug with Charles Proxy it actually takes down the application. This usually happens with malformed AMF responses. Where you able to make it work? Thanks!
Recursive.php
<?php 
class Recursive {
    
    public function testGetRecursiveObject() { 
        $o = new TestObject(); 
        $o->recursive = new TestObject(); 
        $o->recursive->recursive = $o; 
        return $o; 
    }
}

class TestObject {
    public $recursive;
}
Hide
Mikaël LABRUT added a comment -

I test my patch with Flex 3 and ServiceCapture and occured no problem. (In attachment my minimal MXML file)

I don't understand where is your problem ?

Show
Mikaël LABRUT added a comment - I test my patch with Flex 3 and ServiceCapture and occured no problem. (In attachment my minimal MXML file) I don't understand where is your problem ?
Hide
Mikaël LABRUT added a comment -

This patch fix an other problem relative to arrays.

Explanation (from amfphp library)
//Circular referencing is disabled in arrays
//Because if the array contains only primitive values,
//Then === will say that the two arrays are strictly equal
//if they contain the same values, even if they are really distinct
//if(($key = patched_array_search($d, $this->storedObjects, TRUE)) === FALSE )

So we just need to save the index information into _references table when an array is serialized.

Show
Mikaël LABRUT added a comment - This patch fix an other problem relative to arrays. Explanation (from amfphp library) //Circular referencing is disabled in arrays //Because if the array contains only primitive values, //Then === will say that the two arrays are strictly equal //if they contain the same values, even if they are really distinct //if(($key = patched_array_search($d, $this->storedObjects, TRUE)) === FALSE ) So we just need to save the index information into _references table when an array is serialized.
Hide
Penny Leach added a comment -

Hi,

I'm having the same problem here too and the patch didn't fix the problem.

To be clear -

We're sending data from Flash with circular references, and they get decoded into PHP fine. The problem exists in the opposite direction - sending the data back from PHP to Flash.

It seems odd to me that the bug is asymmetric.

More than happy to help with testing patches.

Show
Penny Leach added a comment - Hi, I'm having the same problem here too and the patch didn't fix the problem. To be clear - We're sending data from Flash with circular references, and they get decoded into PHP fine. The problem exists in the opposite direction - sending the data back from PHP to Flash. It seems odd to me that the bug is asymmetric. More than happy to help with testing patches.
Hide
Penny Leach added a comment -

I've been looking into this further (we really need it to work so I'm trying to hack together a fix), and I'm really confused by how this works. We're sending amf3 requests from Flash, and they are certainly arriving as part of the request as amf 3 (error_log("client version $clientVersion"); ) in the readMessage method in Zend_Amf_Request definitely says 3. However, the $clientVersion variable is never promoted to a member variable - I see there's a $_clientType member, but that doesn't seem to get set, and defaults to 0.

Worse, the Amf3 Serializer never even gets included - putting an error_log call at the top of the file (outside the class even) , never shows up.

Looking through the code, I can't find any connection between getting an Amf3 request, and sending an Amf3 Response. The _handle method of the Server certainly wants to send back amf0, based on objectEncoding it gets from the request (which surely should be 3, since that's what $clientVersion is!?), and the only way I can see the Amf3 Serializer being loaded is in the Amf0 Serializer's writeAmf3TypeMarker method, which has a comment which says, "Once AMF3 is encountered it will not return to AMf0", yet I cannot find any code-evidence that is the case, and this method isn't being called anyway.

I'm sure I am just being dense and there's something silly we're doing wrong as nobody else seems to have a problem with amf3 Is there any obvious way to force amf3, since setting the objectEncoding to 3 in Flash doesn't seem to work for us?

Show
Penny Leach added a comment - I've been looking into this further (we really need it to work so I'm trying to hack together a fix), and I'm really confused by how this works. We're sending amf3 requests from Flash, and they are certainly arriving as part of the request as amf 3 (error_log("client version $clientVersion"); ) in the readMessage method in Zend_Amf_Request definitely says 3. However, the $clientVersion variable is never promoted to a member variable - I see there's a $_clientType member, but that doesn't seem to get set, and defaults to 0. Worse, the Amf3 Serializer never even gets included - putting an error_log call at the top of the file (outside the class even) , never shows up. Looking through the code, I can't find any connection between getting an Amf3 request, and sending an Amf3 Response. The _handle method of the Server certainly wants to send back amf0, based on objectEncoding it gets from the request (which surely should be 3, since that's what $clientVersion is!?), and the only way I can see the Amf3 Serializer being loaded is in the Amf0 Serializer's writeAmf3TypeMarker method, which has a comment which says, "Once AMF3 is encountered it will not return to AMf0", yet I cannot find any code-evidence that is the case, and this method isn't being called anyway. I'm sure I am just being dense and there's something silly we're doing wrong as nobody else seems to have a problem with amf3 Is there any obvious way to force amf3, since setting the objectEncoding to 3 in Flash doesn't seem to work for us?
Hide
Penny Leach added a comment -

Manually applying the patch changes to the Amf0 Serializer just gives me client.data.underflow error, so that doesn't seem like a viable workaround

Show
Penny Leach added a comment - Manually applying the patch changes to the Amf0 Serializer just gives me client.data.underflow error, so that doesn't seem like a viable workaround
Hide
Wade Arnold added a comment -

@penny the $clientVersion does not explicitly meant hat the data is going to be AMF3. In most cases it just means it is coming from a flash player 9 or greater player which supports AMF3. The deserialization process starts in the amf0 system and until it encounters and AMF3 type it stays AMF0. If you are using netconnection you will almost always stay inside of amf0. RemoteObject is wrapped in a RemoteMessage that is an AM3 type so if you use remoteobject it will always be amf3. Hope that helps.

There are two patches that we committed yesterday for resource handling and authentication and this is next on the list!

Show
Wade Arnold added a comment - @penny the $clientVersion does not explicitly meant hat the data is going to be AMF3. In most cases it just means it is coming from a flash player 9 or greater player which supports AMF3. The deserialization process starts in the amf0 system and until it encounters and AMF3 type it stays AMF0. If you are using netconnection you will almost always stay inside of amf0. RemoteObject is wrapped in a RemoteMessage that is an AM3 type so if you use remoteobject it will always be amf3. Hope that helps. There are two patches that we committed yesterday for resource handling and authentication and this is next on the list!
Hide
Penny Leach added a comment -

Hi Wade, thanks for replying. I'm happy to help with debugging or testing if that helps develop a fix for this circular reference problem.

I can be reached on irc (Mjollnir` on freenode/oftc) or skype (mjollnir__) and I will try patches.

Cheers!

Show
Penny Leach added a comment - Hi Wade, thanks for replying. I'm happy to help with debugging or testing if that helps develop a fix for this circular reference problem. I can be reached on irc (Mjollnir` on freenode/oftc) or skype (mjollnir__) and I will try patches. Cheers!
Hide
Stefan Klug added a comment -

Just a note. This bug is mostly a duplicate of ZF-6641.
The patch provided here will not work in all cases as it doesn't reference XML data, Date objects etc.
In ZF-6641 I provided a patch which also fixes this problem.

Stefan

Show
Stefan Klug added a comment - Just a note. This bug is mostly a duplicate of ZF-6641. The patch provided here will not work in all cases as it doesn't reference XML data, Date objects etc. In ZF-6641 I provided a patch which also fixes this problem. Stefan
Hide
Stefan Klug added a comment -

Hi Penny, are you really running Amf0 only? If you could provide a testcase for your issue it would be easier to fix.
I attached a patch for the Amf0 Serializer which adds support for references. This patch is a quick shot and not tested at all (as I already said a testcase is needed).

Regards Stefan

Show
Stefan Klug added a comment - Hi Penny, are you really running Amf0 only? If you could provide a testcase for your issue it would be easier to fix. I attached a patch for the Amf0 Serializer which adds support for references. This patch is a quick shot and not tested at all (as I already said a testcase is needed). Regards Stefan
Hide
Penny Leach added a comment -

argh, I can't figure out how to making it do Amf3 - Here's a simple test case:

http://paste.dollyfish.net.nz/6bcd3e

and the flash side, we're using the code here:

http://git.penny.liip.ch/?p=amftester.git;a=summary

When I uncomment the second $return, I get Object in the flash view. When I comment it out, I get ... nothing.. anger.

Show
Penny Leach added a comment - argh, I can't figure out how to making it do Amf3 - Here's a simple test case: http://paste.dollyfish.net.nz/6bcd3e and the flash side, we're using the code here: http://git.penny.liip.ch/?p=amftester.git;a=summary When I uncomment the second $return, I get Object in the flash view. When I comment it out, I get ... nothing.. anger.
Hide
Penny Leach added a comment -

Sorry, should have tried the patch before speaking.

At initial testing (with test data, not the real data since my flash guy is away this week), it seems to completely fix the problem! You are my hero!

Still no idea why it's not switching to Amf3, but if this solves the problem I'll be really happy. I'll test again with real data and get back to you.

Thanks

Show
Penny Leach added a comment - Sorry, should have tried the patch before speaking. At initial testing (with test data, not the real data since my flash guy is away this week), it seems to completely fix the problem! You are my hero! Still no idea why it's not switching to Amf3, but if this solves the problem I'll be really happy. I'll test again with real data and get back to you. Thanks
Hide
Penny Leach added a comment -

Confirm this patch completely fixed the problem. Stefan, thanks so much!

Show
Penny Leach added a comment - Confirm this patch completely fixed the problem. Stefan, thanks so much!
Hide
Wade Arnold added a comment -

Issues has been resolved in the standard trunk. Patch and unit test have been committed. Thanks everyone for the patch and for the test cases!

Show
Wade Arnold added a comment - Issues has been resolved in the standard trunk. Patch and unit test have been committed. Thanks everyone for the patch and for the test cases!

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: