Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

org.openx.data.jsonserde.json.JSONArray cannot be cast to org.openx.data.jsonserde.json.JSONObject #112

Closed
furmangg opened this issue Aug 14, 2015 · 1 comment

Comments

@furmangg
Copy link

My JSON is usually this format:

            {
                "value": {
                    "US": 89,
                    "KR": 1
                },
                "end_time": "2015-07-01T07:00:00+0000"
            }

But sometimes Facebook makes value an empty array not a map:

            {
                "value": [],
                "end_time": "2015-07-01T07:00:00+0000"
            }

If I declare my table as a map as follows:

CREATE EXTERNAL TABLE IF NOT EXISTS mytable (
  value:map<string,int>,
  end_time:string
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS TEXTFILE 
LOCATION 'my location';

Then I get the error:
org.openx.data.jsonserde.json.JSONArray cannot be cast to org.openx.data.jsonserde.json.JSONObject

I would prefer you provide a way to flag this issue as ignorable. I'd prefer not to ignore the entire json file, but just rather null out that particular value. And ignore.malformed.json=true doesn't skip this error anyway.

Another possible fix would be to implement a uniontype complex type so I could declare that it could be an array or a map.

@rcongiu
Copy link
Owner

rcongiu commented Aug 14, 2015

Mm...making it ignore an empty array should not be hard to do. I want to code support for union types too since several people have been asking for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants