CollectionSerializer only processes first entry

[ghost]

Hello together,
I am currently struggling with deserializing a List using a custom deserializer for the generic type of the list. I think that I am missing something obvious but I can't get it to work and I didn't find any issue related to mine.
My simplified implementation:

@JsonbTypeDeserializer(ProxyDeserializer.class)
public interface User {
    // only getters (following Java naming convention)
}

public class ProxyDeserializer implements JsonbDeserializer<Object> {
    public Object deserialize(final JsonParser parser, final DeserializationContext ctx, final Type rtType) {
        final JsonObject value = ctx.deserialize(JsonObject.class, parser);
        final Proxy proxy = ... // Create a Java dynamic proxy for rtType (User.class) and the JsonObject
        return proxy;
    }
}

In my test I am executing a GET as followed:

ClientBuilder.newClient().target(url)
    .buildGet()
    .invoke(new GenericType<List<User>>() {
    });

But I only get a List with one entry. My deserializer is not called a second time.
I am using Yasson 1.0.3 in a Wildfly 14 container.

I appreciate any hint!

[notarmara]

I have the same problem, have you solved it?

[bravehorsie]

What is the JSON input for this part:
final JsonObject value = ctx.deserialize(JsonObject.class, parser); ?

[notarmara]

I can only talk about my case, that is a little bit different from the one that @cpoels reported.
I think that to the custom deserializer is arriving the entire json, that is to say the entire array, not only the contained objects one a time. For this reason the deserializer is called just once. Probably we are wrong in configuring the jsonb object.
this is my example, I have the following deserializer:

public class HistoryDeserializer implements JsonbDeserializer<History>{
@Override
public History deserialize(JsonParser parser, DeserializationContext ctx, Type rtType) {
           JsonObject node = parser.getObject();
           History doc = new History();
           if (node.get(ID) != null) {
           doc.setUniqueId(node.getString(ID));
} 
...

in which History is a simple class, without generics or anything else.

I have extended the Jersey's AbstractMessageReaderWriterProvider

@Override
public Object readFrom(Class<Object> type, Type genericType, Annotation[] annotations, MediaType mediaType, MultivaluedMap<String, String> httpHeaders, InputStream entityStream) throws IOException {
               JsonbConfig config = new JsonbConfig().setProperty(JSONB_FAIL_ON_UNKNOWN_PROPERTIES, 
               true).withNullValues(true);
		Jsonb jsonb = JsonbBuilder.create(config);
		try {
			return jsonb.fromJson(entityStream, genericType);
		} catch (JsonbException e) {
			throw new ProcessingException(LocalizationMessages.ERROR_JSONB_DESERIALIZATION(), e);
		}
	}

in which the genericType is History[].class, I have checked it.
this is my test rest call:

@POST
	@Path("/jsonArray")
	@Consumes(MediaType.APPLICATION_JSON)
	@Produces(MediaType.APPLICATION_JSON)
	public History[] jsonArray(History[] array) {
		return array;
	}

[notarmara]

I just want to add that without a custom deserializer, I have no problems with arrays.

[bravehorsie]

@notarmara In your custom deserializer you are handling jsonp yourself, so it looks to be either: a bug on jsonp, or an incorrect json parsing. Without a sample JSON text (the input for JsonParser) it is hard to tell. Looking at @Path("/jsonArray") it looks that JSON text contains an array instead of an object as its root, shouldn't your deserializer call JsonParser#getArray() instead of JsonParser#getObject()?

As a side note, you are creating Jsonb inside AbstractMessageReaderWriterProvider#readFrom on every call, which will cause redundant reflection class scanning, slowing down request processing considerably. Jsonb instance should be cached and reused.

[notarmara]

yes can be a bug of jsonp, I'm trying to understand where is the problem for this reason I involved you.
However the rest call I have reported is a sample, the deserializer is also called with a rest service that receive a single object, for this reason I cannot simply change from JsonParser#getObject() to JsonParser#getArray() .
Moreover the code of the AbstractMessageReaderWriterProvider#readFrom is a sample too. I have omitted all the implementation reporting only the things that in my mind are useful to solve the problem, so in my full implementation the jsonb is a singleton.

[bravehorsie]

It would be best to narrow down the problem to Yasson outside of Jersey context. Do you have a HTTP request log? Can you save the JSON feeded from parser and log it?

[notarmara]

yes let's will try to go down in deep.
this is the rest call I'm doing

this is the operations' sequence performed:
we receiving the START_ARRAY event into JsonbDeserializer#build():

Now my custom deserializer is loaded:

this is the result of parser.getObject():

I have also tryied to perform parser.getArray() but received an error becouse the START_OBJECT event was called.

After that the parser.hasNext() is called but the call returns and an array of one element is created by the
ObjectArrayDeserializer:

I hope I'm clear now. please tell me if you need more information.
Just one question: must I define in some way that the deserializer is valid both for array and for a single object?

[bravehorsie]

I don't see any problem in provided screens, you have an JSON with an json array as root and one json object inside this array. You get it deserialized once as expected. What is missing?

Am I correct to say it would be the same as calling:
jsonb.fromJson("[{...}]", History[].class); ?

In your case
public class HistoryDeserializer implements JsonbDeserializer<History>
is defined for History type. If there are more than one object entry in "history" json array it should be called multiple times (You need to feed this serializer with JsonbConfig#withSerializers(...).

If you want a deserializer for whole array instead of its member type you would define it as:

public class HistoryDeserializer implements JsonbDeserializer<List<<History>>
public class HistoryDeserializer implements JsonbDeserializer<History[]>

[bravehorsie]

Just tried following code on master branch and it works as expected:

@Test
public void testDeserializeArray() {
    String json = "[{\"stringProperty\":\"Property 1 value\"},{\"stringProperty\":\"Property 2 value\"}]";      
   Jsonb jsonb = JsonbBuilder.create(new JsonbConfig().withDeserializers(new 
   SimplePojoDeserializer()));
   SimplePojo[] result = jsonb.fromJson(json, SimplePojo[].class);
   Assert.assertEquals(2, result.length);
}

public class SimplePojoDeserializer implements JsonbDeserializer<SimplePojo> {
    @Override
    public SimplePojo deserialize(JsonParser parser, DeserializationContext ctx, Type rtType) {
        SimplePojo simplePojo = new SimplePojo();
        parser.next();
        parser.next();
        simplePojo.setStringProperty(parser.getString());
        return simplePojo;
    }
}

public class SimplePojo {
    private String stringProperty;

    public String getStringProperty() {
        return stringProperty;
    }

    public void setStringProperty(String stringProperty) {
        this.stringProperty = stringProperty;
    }
}

[notarmara]

I have tried your test and the deserializer is called 2 times, so I have rewritten your test using my deserializer and my pojo. the result of the test is the same, an array of 3 object and deserializer called just once.
I finally figure out the problem. the problem is related to the following instruction:
JsonObject node = parser.getObject();
infact if you rewrite your deserializer in this way you will have the same problem:

@Test
	public void testDeserializeArray() {
		String json = "[{\"stringProperty\":\"Property 1 value\"},{\"stringProperty\":\"Property 2 value\"}]";
		Jsonb jsonb = JsonbBuilder.create(new JsonbConfig().withDeserializers(new SimplePojoDeserializer()));
		SimplePojo[] result = jsonb.fromJson(json, SimplePojo[].class);
		Assert.assertEquals(2, result.length);
	}
	
//	@Test
//	public void testDeserializeArray() {
//		String json = "[{\"type\":\"WORK_CONTRACT\",\"uniqueId\":\"2840\",\"title\":\"Prime\",\"addDate\":\"2018-12-11T15:15:40.862Z\",\"additionalInfo\":{},\"additionalIds\":{}},{\"type\":\"WORK_CONTRACT\",\"uniqueId\":\"2840\",\"title\":\"2840, Test_07122018_145404.370\",\"addDate\":\"2018-12-07T15:15:40.862Z\",\"additionalInfo\":{},\"additionalIds\":{}}, {\"type\":\"WORK_CONTRACT\",\"uniqueId\":\"2840\",\"title\":\"2840, Test_07122018\",\"addDate\":\"2018-12-07T15:17:40.862Z\",\"additionalInfo\":{},\"additionalIds\":{}}]";
//		Jsonb jsonb = JsonbBuilder.create(new JsonbConfig().withDeserializers(new HistoryDeserializer()));
//		History[] result = jsonb.fromJson(json, History[].class);
//		Assert.assertEquals(3, result.length);
//	}

	public class SimplePojoDeserializer implements JsonbDeserializer<SimplePojo> {
		@Override
		public SimplePojo deserialize(JsonParser parser, DeserializationContext ctx, Type rtType) {
			JsonObject json = parser.getObject();
			SimplePojo simplePojo = new SimplePojo();
//			parser.next();
//			parser.next();
//			simplePojo.setStringProperty(parser.getString());
			simplePojo.setStringProperty(json.getString("stringProperty"));
			return simplePojo;
		}
	}

	public class SimplePojo {
		private String stringProperty;

		public String getStringProperty() {
			return stringProperty;
		}

		public void setStringProperty(String stringProperty) {
			this.stringProperty = stringProperty;
		}
	} 

Is there an alternative way to access to object data? Do I need to use parser.getString() as you have done in your example?

[notarmara]

After debugging, I think that the problem is related to glassfish implementation of the JsonParser, so it isn't a yasson issue. Probably the @cpoels original issue is different from this, I have misunderstood becouse the effect was the same.

[bravehorsie]

This https://github.com/eclipse-ee4j/jsonp is glassfish JsonParser impl (the only one). I will take a look at your example later.

[notarmara]

yeah, what I want to say is that I think the problem is not inside the yasson but in jakarta.json lib as you suggested me previously. Ok, I will waiting for your feedback. Thank you

[notarmara]

as you probably already know, the problem reported by @cpoels should be the same because the JsonObjectDeserializer calls parser.getObject()

[bravehorsie]

Thank you @notarmara, I now see the problem. Its not because of a bug in jsonp. JsonParser#getObject advances parser to the JsonParser.Event.END_OBJECT, which Yasson doesn't expect. I will try to fix this in Yasson.

You can also do:

SimplePojo sp = ctx.deserialize(SimplePojo.class, parser);
//instead of
JsonObject obj = parser.getObject();

[aguibert]

Closing this issue since @bravehorsie delivered the fix in #271