librelist archives

« back to archive

Can JAYL parse an array of nameless dicts?

Can JAYL parse an array of nameless dicts?

From:
Can Özmen
Date:
2010-09-16 @ 13:33
Hi All,

I'm using YAJL via MGTwitterEngine and I got a problem with parsing
the "entities" part of a Twitter status response. The problem part is
this:

"entities":  {
        "urls":
        [
            {
                "url":"http://bit.ly/dCueLT",
                "indices":
                [
                    117,
                    137
                ]
            }
        ],
        "hashtags":
        [
            {
                "indices":
                [
                    62,
                    71
                ],
                "text":"titanium"
            }
        ],
        "user_mentions":
        [
            {
                "indices":
                [
                    3,
                    16
                ],
                "screen_name":"squarefactor",
                "name":"squareFACTOR",
                "id":15893275
            },
            {
                "indices":
                [
                    48,
                    61
                ],
                "screen_name":"appcelerator",
                "name":"Appcelerator",
                "id":9662352
            },
            {
                "indices":
                [
                    88,
                    97
                ],
                "screen_name":"intridea",
                "name":"Intridea,
                  Inc.",
                "id":14455548
            }
        ]
    }

Stepping through the code, my first idea is the parser is somehow
confused by the anonymous dictionaries as array members. Specifically
this:
"urls": [ { ... } ]

The parser starts an array, then correctly starts a dictionary but
gives it the name of array on the stack (i.e. "urls"); thus losing the
array. Like this:
"urls": { ...  }

I'd be grateful for any pointers on how to continue investigating this issue.

Thanks,
Can.

--
-end-
Can Ozmen
________________________

http://www.kralcan.org
http://posts.kralcan.org

Re: [yajl] Can JAYL parse an array of nameless dicts?

From:
Hatem Nassrat
Date:
2010-09-16 @ 14:01
On Thu, Sep 16, 2010 at 10:33 AM, Can Özmen <canozmen@gmail.com> wrote:

> The parser starts an array, then correctly starts a dictionary but
> gives it the name of array on the stack (i.e. "urls"); thus losing the
> array. Like this:
> "urls": { ...  }
>
> I'd be grateful for any pointers on how to continue investigating this
> issue.
>

Its definitely a bug in the way you are parsing. Yajl is SAX like parsing
for JSON,  So essentially, you will get a dict key callback for "urls" then
a start array callback then a start dict callback ... so its the code that
is inside the callback that doesn't properly add the dict to the open
(started) array.

There are general caveats with using SAX like parsing, due to its low level
nature you have to be aware of what is going on to be able to write
an efficient parser for your needs. Maybe if you post your code someone can
point out the error.


-- 
Hatem Nassrat

Re: [yajl] Can JAYL parse an array of nameless dicts?

From:
Can Özmen
Date:
2010-09-20 @ 13:42
Thanks for the tip Hatem,

It really is the callbacks in MGTwitterEngine that's causing the headache.
It looks like it was written assuming all dicts and arrays have names (start
array and dict callbacks are passed the last key encountered), which in this
case they do not.

Regards,
Can.

On Thu, Sep 16, 2010 at 17:01, Hatem Nassrat <hnassrat@gmail.com> wrote:

>
>
> On Thu, Sep 16, 2010 at 10:33 AM, Can Özmen <canozmen@gmail.com> wrote:
>
>> The parser starts an array, then correctly starts a dictionary but
>> gives it the name of array on the stack (i.e. "urls"); thus losing the
>> array. Like this:
>> "urls": { ...  }
>>
>> I'd be grateful for any pointers on how to continue investigating this
>> issue.
>>
>
> Its definitely a bug in the way you are parsing. Yajl is SAX like parsing
> for JSON,  So essentially, you will get a dict key callback for "urls" then
> a start array callback then a start dict callback ... so its the code that
> is inside the callback that doesn't properly add the dict to the open
> (started) array.
>
> There are general caveats with using SAX like parsing, due to its low level
> nature you have to be aware of what is going on to be able to write
> an efficient parser for your needs. Maybe if you post your code someone can
> point out the error.
>
>
> --
> Hatem Nassrat
>



-- 
-end-
Can Ozmen
________________________

http://www.kralcan.org
http://posts.kralcan.org