Kotlin Serialization - polymorphic deserialization of a sealed hierarchy with JsonContentPolymorphicSerializer as an interceptor

Question

I have a need to deserialize JSON models into a strict hierarchy of classes with a strong forwards compatibility requirement.
This is done with closed polymorphism with sealed classes according to documentation.
The models I'm working with look like this:

@Serializable
sealed class BaseChatMessageDto {
    abstract val id: Int
    abstract val localId: Int
    abstract val fromId: Int
    abstract val replyTo: Int?
    abstract val remoteStatus: RemoteStatus?
    abstract val createdAt: Int
    abstract val type: ChatMessageType
}

ChatMessageType is an enum of message types the client is aware of. A custom serializer ensures that any unknown type is deserialized as ChatMessageType.UNKNOWN:

object ChatMessageTypeFallback :
    ForwardCompatibleEnumSerializer<ChatMessageType>(ChatMessageType::class)
@Serializable(ChatMessageTypeFallback::class)
enum class ChatMessageType {
    @SerialName("text")
    TEXT,
    @SerialName("file")
    FILE,
    @SerialName("photo")
    PHOTO,
    @SerialName("sticker")
    STICKER,
    @SerialName("gif")
    GIF,
    @SerialName("task")
    TASK,
    @SerialName("service")
    SERVICE,
    @ForwardsCompatible
    @SerialName("unknown")
    UNKNOWN
}

So, the type in BaseChatMessageDto serves as an implicit type discriminator for the kotlinx.serialization compiler plugin. Each of the message types must be deserialized into an object of its own class, for example:

@Serializable
@SerialName("sticker")
data class StickerChatMessageDto(
    override val id: Int,
    @SerialName("local_id")
    override val localId: Int,
    // Override the rest of BaseChatMessageDto

    @SerialName("sticker_id")
    val stickerId: Int?,
    // Other fields specific for "sticker" type

    // Override ChatMessageWithReactionsDto field
    override val reactions: List<Reaction>,
): BaseChatMessageDto(), ChatMessageWithReactionsDto

Some types like FILE and PHOTO are considered rich messages, and have their own superclass:

@Serializable
sealed class RichChatMessageDto: BaseChatMessageDto(), ChatMessageWithReactionsDto {
    abstract override val id: Int
    @SerialName("local_id")
    abstract override val localId: Int
    abstract override val type: ChatMessageType
    // Override the rest of BaseChatMessageDto

    abstract val text: String
    // Other abstract members of RichChatMessageDto

    // Override ChatMessageWithReactionsDto field
    abstract override val reactions: List<Reaction>?
}

@Serializable
@SerialName("file")
data class FileChatMessageDto(
    override val type: ChatMessageType = ChatMessageType.FILE,
    // Override everything else from RichChatMessageDto
): RichChatMessageDto()

UnknownChatMessageDto provides forwards compatibility by being deserialized into when an unknown ChatMessageType is encountered:

@Serializable
@SerialName("unknown")
data class UnknownChatMessageDto(
    // Override everything.
): BaseChatMessageDto()

But there is a very special case in this logic - deleted messages. Server-side they are discriminated not by type, but by status. If status is DELETED, then the base model is returned regardless of its type. So I need to create a deserialization interceptor of sorts, that will look at status at first, use DeletedMessageDto.serializer() if necessary, otherwise proceed with default deserialization pipeline. As far as I'm aware, JsonContentPolymorphicSerializer is designed specifically for this, so:

@Serializable
data class DeletedChatMessageDto(
    // Override everything
): BaseChatMessageDto()

@Serializable(
    with = DeletedMessageInterceptorSerializer::class
)
sealed class BaseChatMessageDto {
    // ...
}

object DeletedMessageInterceptorSerializer: JsonContentPolymorphicSerializer<BaseChatMessageDto>(
    BaseChatMessageDto::class
) {
    override fun selectDeserializer(element: JsonElement): DeserializationStrategy<BaseChatMessageDto> {
        val status = try {
            val primitive = element.jsonObject["status"]?.jsonPrimitive
            // Parse status into enum
        } catch (e: Exception) {
            e.printStackTrace()
            RemoteStatus.UNKNOWN
        }
        return when (status) {
            RemoteStatus.DELETED -> DeletedChatMessageDto.serializer()
            else -> PolymorphicSerializer(BaseChatMessageDto::class)
        }
    }
}

But it doesn't work as expected. DeletedChatMessageDto seem to be deserialized fine, but for other types it fails:

kotlinx.serialization.json.internal.JsonDecodingException: Polymorphic serializer was not found for class discriminator 'text'

If I provide a SerializersModule to my Json like this:

serializersModule += SerializersModule {
    polymorphic(BaseChatMessageDto::class) {
        defaultDeserializer {
            UnknownChatMessageDto.serializer()
        }
    }
}

Then DeletedChatMessageDtos are deserialized fine, but all other types are unsurprisingly deserialized as UnknownChatMessageDto.
If I register subclasses explicitly:

serializersModule += SerializersModule {
    polymorphic(BaseChatMessageDto::class) {
        subclass(TextChatMessageDto::class)
        subclass(FileChatMessageDto::class)
        subclass(PhotoChatMessageDto::class)
    }
}

Then another exception is thrown:

Caused by: java.lang.IllegalArgumentException: Polymorphic serializer for class <package>.FileChatMessageDto (Kotlin reflection is not available) has property 'type' that conflicts with JSON class discriminator. You can either change class discriminator in JsonConfiguration, rename property with @SerialName annotation or fall back to array polymorphism

If I instead try to use a suggestion from @simon-jacobs about using a typealias like this:

typealias BaseChatMessageDto =
    @Serializable(DeletedMessageInterceptorSerializer::class)
    BaseChatMessageDtoPlain

@Serializable
sealed class BaseChatMessageDtoPlain {
    // ...
}

// I tried both BaseChatMessageDtoPlain and BaseChatMessageDto here
object DeletedMessageInterceptorSerializer: JsonContentPolymorphicSerializer<BaseChatMessageDtoPlain>(
    BaseChatMessageDtoPlain::class
) {
    override fun selectDeserializer(element: JsonElement): DeserializationStrategy<BaseChatMessageDtoPlain> {
        val status = // ...
        return when (status) {
            RemoteStatus.DELETED -> DeletedChatMessageDto.serializer()
            else -> BaseChatMessageDtoPlain.serializer()
        }
    }

}

Then the serializer is called when BaseChatMessageDto is a property of another serializable type, but not called for a Retrofit call that returns a List<BaseChatMessageDto>, which results in exceptions for deleted messages:

kotlinx.serialization.MissingFieldException: Field 'text' is required for type with serial name 'text', but it was missing at path: $[1]

I can't quite figure out why DeletedMessageInterceptorSerializer is not applied through a typealias for this particular call.

I am aware that I could use JsonContentPolymorphicSerializer to select a serializer based on type by hand, but I want to offload as much logic to the compiler plugin for type safety, code clarity and less places a mistake could be made. What am I doing wrong here? Is PolymorphicSerializer(BaseChatMessageDto::class) not the way to proceed with closed polymorphic deserialization pipeline?

it's really weird that you have an enum inside a sealed hierarchy, the whole point of a sealed hierarchy is that it is the enumeration itself. it may "work", but it's not how things where designed to be. — Bruno Medeiros
– Bruno Medeiros, Commented Sep 6, 2024 at 22:22

Simon Jacobs · Accepted Answer · 2024-03-08 17:23:14Z

0

The `type` discriminator

type is used as the default JSON key for polymorphic serialization. As far as I am aware it might be possible to get polymorphic serialization to work with that property name occupied by an enum, but you are making life difficult for yourself by doing so.

I would suggest not using an enum class at all, or at least don't use it the type property for it. It would appear to have no real functionality in your server-side code anyhow, since you have a corresponding class hierarchy.

Instead, leave type blank in BaseChatMessageDto. The serializer can pick the type key up in the JSON itself and use it to choose the correct subclass provided those subclasses have serial names corresponding to the values type can be (something you look to have already taken care of).

Using `PolymorphicSerializer` and alternatives

Is PolymorphicSerializer(BaseChatMessageDto::class) not the way to proceed with closed polymorphic deserialization pipeline?

No, it's not. This serializer is associated with defining a SerializersModule, as alluded to by the documentation¹. To use this you would need to register each of the subclasses (in conjunction with removing the type property as described above).

You can still get it to work without a module though by having a different serializer specified as the default one for BaseChatMessageDto to the one you use to deserialize an incoming JSON object.

To do this, leave BaseChatMessageDto with its plain @Serializable annotation and refer to this in DeletedMessageInterceptorSerializer:

return when (status) {
    RemoteStatus.DELETED -> DeletedChatMessageDto.serializer()
    else -> BaseChatMessageDto.serializer()
}

And use your inteceptor serializer when you are actually decoding a message²:

Json.decodeFromString(DeletedMessageInterceptorSerializer, incomingMessage)

¹The sealed class counterpart to PolymorphicSerializer is SealedClassSerializer, which you could instantiate by hand. However I think the approach suggested above would be more typical rather than using this internal API.

²Or if your message is decoded as part of a larger class hierarchy, you might instead specify DeletedMessageInterceptorSerializer by annotating the appropriate enclosing class property.

answered Mar 8, 2024 at 17:23

Simon Jacobs

7,0302 gold badges14 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

Calamity Over a year ago

You seem to have assumed I control the server. I apologize for not clarifying this in my question, but I do not, in fact control server-side schemas. I'm working entirely with an Android client.

Calamity Over a year ago

I'm not having any problems with using an enum as a type discriminator, since it comes in a form of a string and gets deserialized into an enum after the lib runtime choses a concrete class serializer from a list of BaseChatMessageDto descendants. Moreover, I need it in my local data source to discriminate between types of messages after I query for them, so I need it exactly where it is.

Calamity Over a year ago

I'm not actually calling Json.decodeFromString by hand - Retrofit calls the decoder for me, so I can't control what serializer it uses in all places. For example, a @GET request that returns a List<BaseChatMessageDto>. Even if I can, it would be error-prone to annotate every usage of BaseChatMessageDto as a property of another serializable type with @Serializable(DeletedMessageInterceptorSerializer::class).

Calamity Over a year ago

For now I went with selecting a serializer based on type in DeletedMessageInterceptorSerializer to unblock myself, but I would love to make it work as I intended.

Simon Jacobs Over a year ago

(1) I haven't assumed you control the server: I was aware you might not. (2) I suggest you get it to work without using type as an enum first and you can experiment with changing it back afterwards. It would be a shame to refuse to try this when it appears to be an issue. Further, from what I have read, you don't appear to need the enum because the class of the enclosing object tells you the same information. (3) If you can't annotate the relevant usages of the BaseChatMessageDto serializer then you can use SealedClassSerializer which I described.

|

Collectives™ on Stack Overflow

Kotlin Serialization - polymorphic deserialization of a sealed hierarchy with JsonContentPolymorphicSerializer as an interceptor

1 Answer 1

The `type` discriminator

Using `PolymorphicSerializer` and alternatives

9 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

The type discriminator

Using PolymorphicSerializer and alternatives

9 Comments

Your Answer

Sign up or log in

Post as a guest

Related

The `type` discriminator

Using `PolymorphicSerializer` and alternatives