JsonDecodingException when Cookie.expires is null

[tidbeck]

Version

5.14.0-1299070

What happened?

Getting an exception when decoding a message with a Cookie that has expires set to null like the one below. I think the actual issue is that the Chrome DevTools Protocol, not setting expires as an optional. I have reported the issue, but maybe there is something that can be done in the Kotlin library?

Network.responseReceivedExtraInfo {"blockedCookies":[{"blockedReasons":["SameSiteUnspecifiedTreatedAsLax"],"cookie":{"domain":".twitter.com","expires":null,"httpOnly":true,"name":"fm","path":"/","priority":"Medium","sameParty":false,"secure":true,"session":false,"size":3,"sourcePort":443,"sourceScheme":"Secure","value":"0"},"cookieLine":"fm=0; Max-Age=0; Expires=Thu, 06 Feb 2025 14:43:26 GMT; Path=/; Domain=.twitter.com; Secure; HTTPOnly"},{"blockedReasons":["SameSiteUnspecifiedTreatedAsLax"],"cookie":{"domain":".twitter.com","expires":-1,"httpOnly":true,"name":"_twitter_sess","path":"/","priority":"Medium","sameParty":false,"secure":true,"session":true,"size":302,"sourcePort":443,"sourceScheme":"Secure","value":"BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%!A(MISSING)SGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCE4%!B(MISSING)t9uUAToMY3NyZl9p%!A(MISSING)ZCIlMzgzNTQ3NjYxMWM2ZmNlOGMzYzgwYTMyMGVjYWNiZmU6B2lkIiVhMmM3%!A(MISSING)MjgwZjJjODUxY2U3MDZhZTY2NzY0YWFkZDg5MQ%!D(MISSING)%!D(MISSING)--df362bcea8801c3a5475933d327faff6383e6c76"},"cookieLine":"_twitter_sess=BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%!A(MISSING)SGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCE4%!B(MISSING)t9uUAToMY3NyZl9p%!A(MISSING)ZCIlMzgzNTQ3NjYxMWM2ZmNlOGMzYzgwYTMyMGVjYWNiZmU6B2lkIiVhMmM3%!A(MISSING)MjgwZjJjODUxY2U3MDZhZTY2NzY0YWFkZDg5MQ%!D(MISSING)%!D(MISSING)--df362bcea8801c3a5475933d327faff6383e6c76; Path=/; Domain=.twitter.com; Secure; HTTPOnly"}],"cookiePartitionKey":"https://x.com","cookiePartitionKeyOpaque":false,"exemptedCookies":[],"headers":{"cache-control":"no-cache, no-store, must-revalidate, pre-check=0, post-check=0","content-encoding":"gzip","content-length":"2559","content-type":"text/javascript; charset=utf-8","date":"Thu, 06 Feb 2025 14:43:26 GMT","expires":"Tue, 31 Mar 1981 05:00:00 GMT","last-modified":"Thu, 06 Feb 2025 14:43:26 GMT","perf":"7402827104","pragma":"no-cache","server":"tsa_o","set-cookie":"fm=0; Max-Age=0; Expires=Thu, 06 Feb 2025 14:43:26 GMT; Path=/; Domain=.twitter.com; Secure; HTTPOnly\n_twitter_sess=BAh7CSIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%!A(MISSING)SGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCE4%!B(MISSING)t9uUAToMY3NyZl9p%!A(MISSING)ZCIlMzgzNTQ3NjYxMWM2ZmNlOGMzYzgwYTMyMGVjYWNiZmU6B2lkIiVhMmM3%!A(MISSING)MjgwZjJjODUxY2U3MDZhZTY2NzY0YWFkZDg5MQ%!D(MISSING)%!D(MISSING)--df362bcea8801c3a5475933d327faff6383e6c76; Path=/; Domain=.twitter.com; Secure; HTTPOnly","status":"200 OK","strict-transport-security":"max-age=631138519","x-connection-hash":"c510778eb3e5a956caf1a4b77adc8c0a6d78d2dbda0756d39431f43055f803e9","x-content-type-options":"nosniff","x-frame-options":"","x-response-time":"103","x-transaction":"72cd9d52f918cbd2","x-transaction-id":"72cd9d52f918cbd2","x-twitter-response-tags":"BouncerExempt\nBouncerCompliant","x-xss-protection":"0"},"requestId":"112.127","resourceIPAddressSpace":"Public","statusCode":200}

Exception:

kotlinx.serialization.json.internal.JsonDecodingException: Failed to parse literal 'null' as a double value at element: $.expires
JSON input: .....,"sameParty":false,"sourceScheme":"Secure","sourcePort":443}
	at kotlinx.serialization.json.internal.JsonExceptionsKt.JsonDecodingException(JsonExceptions.kt:24)
	at kotlinx.serialization.json.internal.JsonExceptionsKt.JsonDecodingException(JsonExceptions.kt:32)
	at kotlinx.serialization.json.internal.AbstractJsonTreeDecoder.unparsedPrimitive(TreeJsonDecoder.kt:99)
	at kotlinx.serialization.json.internal.AbstractJsonTreeDecoder.decodeTaggedDouble(TreeJsonDecoder.kt:551)
	at kotlinx.serialization.json.internal.AbstractJsonTreeDecoder.decodeTaggedDouble(TreeJsonDecoder.kt:37)
	at kotlinx.serialization.internal.TaggedDecoder.decodeDoubleElement(Tagged.kt:260)
	at org.hildan.chrome.devtools.domains.network.Cookie$$serializer.deserialize(NetworkTypes.kt:1058)
	at org.hildan.chrome.devtools.domains.network.Cookie$$serializer.deserialize(NetworkTypes.kt:1058)
	at kotlinx.serialization.json.internal.AbstractJsonTreeDecoder.decodeSerializableValue(TreeJsonDecoder.kt:337)
	at kotlinx.serialization.internal.TaggedDecoder.decodeSerializableValue(Tagged.kt:207)
	at kotlinx.serialization.internal.TaggedDecoder.decodeNullableSerializableElement$lambda$3(Tagged.kt:288)
	at kotlinx.serialization.internal.TaggedDecoder.tagBlock(Tagged.kt:294)
	at kotlinx.serialization.internal.TaggedDecoder.decodeNullableSerializableElement(Tagged.kt:286)
	at org.hildan.chrome.devtools.domains.network.BlockedSetCookieWithReason$$serializer.deserialize(NetworkTypes.kt:1265)
	at org.hildan.chrome.devtools.domains.network.BlockedSetCookieWithReason$$serializer.deserialize(NetworkTypes.kt:1265)
	at kotlinx.serialization.json.internal.AbstractJsonTreeDecoder.decodeSerializableValue(TreeJsonDecoder.kt:337)
	at kotlinx.serialization.internal.TaggedDecoder.decodeSerializableValue(Tagged.kt:207)
	at kotlinx.serialization.internal.TaggedDecoder.decodeSerializableElement$lambda$1(Tagged.kt:279)
	at kotlinx.serialization.internal.TaggedDecoder.tagBlock(Tagged.kt:294)
	at kotlinx.serialization.internal.TaggedDecoder.decodeSerializableElement(Tagged.kt:279)
	at kotlinx.serialization.encoding.CompositeDecoder$DefaultImpls.decodeSerializableElement$default(Decoding.kt:538)
	at kotlinx.serialization.internal.CollectionLikeSerializer.readElement(CollectionSerializers.kt:80)
	at kotlinx.serialization.internal.AbstractCollectionSerializer.readElement$default(CollectionSerializers.kt:51)
	at kotlinx.serialization.internal.AbstractCollectionSerializer.merge(CollectionSerializers.kt:36)
	at kotlinx.serialization.internal.AbstractCollectionSerializer.deserialize(CollectionSerializers.kt:43)
	at kotlinx.serialization.json.internal.AbstractJsonTreeDecoder.decodeSerializableValue(TreeJsonDecoder.kt:337)
	at kotlinx.serialization.internal.TaggedDecoder.decodeSerializableValue(Tagged.kt:207)
	at kotlinx.serialization.internal.TaggedDecoder.decodeSerializableElement$lambda$1(Tagged.kt:279)
	at kotlinx.serialization.internal.TaggedDecoder.tagBlock(Tagged.kt:294)
	at kotlinx.serialization.internal.TaggedDecoder.decodeSerializableElement(Tagged.kt:279)
	at org.hildan.chrome.devtools.domains.network.events.NetworkEvent$ResponseReceivedExtraInfo$$serializer.deserialize(NetworkEvents.kt:642)
	at org.hildan.chrome.devtools.domains.network.events.NetworkEvent$ResponseReceivedExtraInfo$$serializer.deserialize(NetworkEvents.kt:642)
	at kotlinx.serialization.json.internal.AbstractJsonTreeDecoder.decodeSerializableValue(TreeJsonDecoder.kt:337)
	at kotlinx.serialization.json.internal.TreeJsonDecoderKt.readJson(TreeJsonDecoder.kt:26)
	at kotlinx.serialization.json.Json.decodeFromJsonElement(Json.kt:186)
	at org.hildan.chrome.devtools.protocol.SessionSerializationKt.decodePayload(SessionSerialization.kt:57)
	at org.hildan.chrome.devtools.protocol.SessionSerializationKt.access$decodePayload(SessionSerialization.kt:1)
	at org.hildan.chrome.devtools.protocol.SessionSerializationKt$typedEvents$$inlined$mapNotNull$1$2.emit(Emitters.kt:220)
	at kotlinx.coroutines.flow.FlowKt__TransformKt$onEach$$inlined$unsafeTransform$1$2.emit(Emitters.kt:51)
	at org.hildan.chrome.devtools.protocol.ChromeDPSession$events$$inlined$filter$1$2.emit(Emitters.kt:219)
	at org.hildan.chrome.devtools.protocol.ChromeDPConnection$events$$inlined$filterIsInstance$1$2.emit(Emitters.kt:219)

Reproduction and additional details

Steps to reproduce:

• Navigate to https://x.com
• Click the "Sign in" button

Kotlin target platforms

macOS, Linux

[joffrey-bion]

Thank you for opening this issue, and even more for reporting this to the protocol definition project.

I'll see if I can somehow add some opt-in lenience for this. Not sure if it can be done, but if I can add a workaround I definitely will.

[joffrey-bion]

Interestingly, the error I'm getting is this one instead:

kotlinx.serialization.json.internal.JsonDecodingException: Expected JsonObject, but had JsonLiteral as the serialized body of org.hildan.chrome.devtools.domains.network.CookiePartitionKey at element: $.cookiePartitionKey
JSON input: "https://x.com"

Maybe another bug in the protocol definitions? Weird.

[joffrey-bion]

Looks like this is just a recent change in the protocol. This field used to be a string, and now it's defined as an object, but I guess I'm not testing against the most recent version of chrome (the zenika/alpine-chrome docker image might be lagging behind slightly).

[joffrey-bion]

I experimented with Browserless and their Docker image instead of zenika/alpine-chrome, and it seems they have a more recent browser version that doesn't suffer from either our issues: the Cookie.expires is present, and the cookiePartitionKey is an object (not a string).

That said, since your initial request is valid, I am adding some pre-processing to make Cookie.expires optional nonetheless.

I will also need to change a bit the entrypoint of the library to support directly providing web socket URLs in a more convenient way, so people can work with Browserless more easily.