How kotlinx.serialization generates code: a compiler plugin deep dive

Kotlinx.serialization has become the standard serialization library for Kotlin, offering compile time code generation, multiplatform support, and format agnostic design. Most developers know the surface level usage: annotate a class with @Serializable, and the library handles the rest. But the deeper question remains: how does the compiler plugin actually generate serializers? What happens between your @Serializable annotation and the working KSerializer<T> implementation?

In this article, you’ll dive deep into the internal mechanisms of the kotlinx.serialization compiler plugin, exploring how the two pass IR generation pipeline works, how the plugin generates the $serializer nested class with its descriptor, serialize, and deserialize methods, and the clever optimizations like golden mask validation that make deserialization both safe and performant. Also, you’ll explore with the real use cases of kotlinx.serialization from the RevenueCat SDK.

The fundamental problem: Reflection is expensive and platform dependent

Consider a simple data class that you want to serialize to JSON:

data class User(
    val id: Long,
    val name: String,
    val email: String?,
    val createdAt: Date
)

Without code generation, serialization libraries must use reflection to discover properties, their types, and their values at runtime. This approach has significant drawbacks:

\/\/ Reflection-based serialization (simplified)
fun serializeWithReflection(obj: Any): Map<String, Any?> {
    val result = mutableMapOf<String, Any?>()
    obj::class.memberProperties.forEach { prop ->
        prop.isAccessible = true
        result[prop.name] = prop.get(obj)
    }
    return result
}

The problems with this approach are substantial. First, reflection is slow. Discovering properties, checking accessibility, and invoking getters at runtime adds overhead to every serialization operation. Second, reflection is platform dependent. Kotlin/Native and Kotlin/JS have limited or different reflection capabilities than the JVM. Third, there is no compile time safety. Type mismatches and missing properties are only discovered at runtime.

The kotlinx.serialization plugin solves these problems by generating serialization code at compile time. The generated code knows exactly which properties exist, their types, and how to read and write them, with no reflection required.

The compiler plugin architecture

The kotlinx.serialization compiler plugin operates as an extension to the Kotlin compiler, hooking into multiple stages of the compilation pipeline. The plugin is organized into several components that work together:

1. Frontend (K1 or K2): Detects @Serializable annotations and generates synthetic declarations (the $serializer class, companion methods).
2. IR Generation: Transforms the intermediate representation in two passes. Pass 1 creates function stubs, Pass 2 generates method bodies.
3. Backend: Emits platform specific code (JVM bytecode, JavaScript, or Native binaries).

The plugin registers itself through a CompilerPluginRegistrar that hooks into these extension points. The critical registration looks conceptually like this:

class SerializationComponentRegistrar : CompilerPluginRegistrar() {
    override fun ExtensionStorage.registerExtensions(configuration: CompilerConfiguration) {
        \/\/ Register resolve extension for synthetic declarations
        SyntheticResolveExtension.registerExtension(SerializationResolveExtension())

        \/\/ Register IR generation extension for code generation
        IrGenerationExtension.registerExtension(SerializationLoweringExtension())

        \/\/ Register K2\/FIR extensions for new compiler frontend
        FirExtensionRegistrarAdapter.registerExtension(FirSerializationExtensionRegistrar())
    }
}

The key observation here is that the plugin operates at multiple levels. It first generates synthetic declarations during the resolve phase, making the compiler aware of generated classes and methods. Then it generates the actual implementation bodies during the IR lowering phase.

Two pass IR generation: Stubs before bodies

One of the most interesting aspects of the kotlinx.serialization plugin is its two pass IR generation strategy. Understanding why this is necessary requires understanding how generated code references itself.

Consider what the plugin needs to generate for a serializable class:

@Serializable
data class User(val name: String, val age: Int)

\/\/ The plugin generates:
\/\/ 1. A nested $serializer class implementing KSerializer<User>
\/\/ 2. A companion object with serializer() method
\/\/ 3. Methods that reference each other

The generated serialize() method might call a helper method like write$Self(). The deserialize() method needs to call the class constructor. These methods reference each other and reference other generated elements. If the plugin tried to generate everything in a single pass, it would encounter undefined references.

The solution is a two pass approach:

class SerializationLoweringExtension : IrGenerationExtension {
    override fun generate(moduleFragment: IrModuleFragment, pluginContext: IrPluginContext) {
        \/\/ Pass 1: Create all declarations without bodies
        val preGenerator = SerializerClassPreLowering(pluginContext)
        moduleFragment.files.forEach { file ->
            preGenerator.runOnFileInOrder(file)
        }

        \/\/ Pass 2: Generate implementation bodies
        val generator = SerializerClassLowering(pluginContext)
        moduleFragment.files.forEach { file ->
            generator.runOnFileInOrder(file)
        }
    }
}

In the first pass, the plugin creates function declarations with empty bodies. This establishes all the symbols that generated code might reference. In the second pass, the plugin fills in the actual implementation bodies, now able to reference any symbol created in the first pass.

This is elegant. The two pass strategy mirrors how compilers handle forward declarations in languages like C, but applied to code generation within a single compilation unit.

The generated serializer structure

When you annotate a class with @Serializable, the plugin generates a nested class named $serializer that implements GeneratedSerializer<T>. Let’s trace through what gets generated for a simple class:

@Serializable
data class Person(
    val name: String,
    val age: Int,
    val email: String? = null
)

The plugin generates a structure that conceptually looks like this:

data class Person(val name: String, val age: Int, val email: String? = null) {

    \/\/ Generated nested serializer class
    internal object `$serializer` : GeneratedSerializer<Person> {

        override val descriptor: SerialDescriptor = buildClassSerialDescriptor("Person") {
            element<String>("name")
            element<Int>("age")
            element<String?>("email", isOptional = true)
        }

        override fun serialize(encoder: Encoder, value: Person) {
            val composite = encoder.beginStructure(descriptor)
            composite.encodeStringElement(descriptor, 0, value.name)
            composite.encodeIntElement(descriptor, 1, value.age)
            if (value.email != null || composite.shouldEncodeElementDefault(descriptor, 2)) {
                composite.encodeNullableSerializableElement(
                    descriptor, 2, String.serializer(), value.email
                )
            }
            composite.endStructure(descriptor)
        }

        override fun deserialize(decoder: Decoder): Person {
            val composite = decoder.beginStructure(descriptor)
            var name: String? = null
            var age: Int? = null
            var email: String? = null
            var seen = 0

            while (true) {
                when (val index = composite.decodeElementIndex(descriptor)) {
                    0 -> { name = composite.decodeStringElement(descriptor, 0); seen = seen or 1 }
                    1 -> { age = composite.decodeIntElement(descriptor, 1); seen = seen or 2 }
                    2 -> { email = composite.decodeNullableSerializableElement(
                               descriptor, 2, String.serializer()); seen = seen or 4 }
                    CompositeDecoder.DECODE_DONE -> break
                    else -> throw SerializationException("Unknown index $index")
                }
            }
            composite.endStructure(descriptor)

            \/\/ Validate required fields
            if (seen and 3 != 3) {
                throwMissingFieldException(seen, 3, descriptor)
            }

            return Person(
                name = name as String,
                age = age as Int,
                email = email
            )
        }

        override fun childSerializers(): Array<KSerializer<*>> = arrayOf(
            String.serializer(),
            Int.serializer(),
            String.serializer().nullable
        )
    }

    companion object {
        fun serializer(): KSerializer<Person> = `$serializer`
    }
}

Notice the structure. The $serializer class is an object (singleton) that holds the immutable descriptor and implements the serialization logic. The companion object provides a convenient serializer() accessor.

Descriptor generation: Metadata for format agnostic serialization

The SerialDescriptor is a critical piece of the generated code. It describes the structure of the serializable class in a format independent way, enabling different serialization formats (JSON, Protobuf, CBOR) to use the same serializer implementation.

The plugin generates descriptor initialization code that builds the complete metadata:

override val descriptor: SerialDescriptor =
    PluginGeneratedSerialDescriptor("com.example.Person", this, 3).apply {
        addElement("name", isOptional = false)
        addElement("age", isOptional = false)
        addElement("email", isOptional = true)
        \/\/ Annotations are also copied to the descriptor
    }

The descriptor includes the serial name of the class (which might differ from the class name if @SerialName is used), the number of elements, each element’s name and whether it’s optional, type information for nested serializers, and any @SerialInfo annotations applied to the class or its properties.

This design is clever. By separating the structural metadata from the serialization logic, formats can make intelligent decisions about encoding. A JSON encoder might use the element names directly as keys. A Protobuf encoder might use element indices. The serializer implementation remains the same.

Golden mask optimization: Efficient required field validation

One of the most important optimizations in the generated code is the golden mask pattern for validating required fields during deserialization. When deserializing, the plugin needs to ensure all required (non optional) fields are present in the input.

The naive approach would be to check each field individually:

\/\/ Naive approach
if (name == null) throw MissingFieldException("name")
if (age == null) throw MissingFieldException("age")
\/\/ ... for each required field

Instead, the plugin generates bitmask based validation:

var seen = 0

\/\/ During deserialization, set bits as fields are encountered
when (index) {
    0 -> { name = ...; seen = seen or 0b001 }  \/\/ bit 0
    1 -> { age = ...;  seen = seen or 0b010 }  \/\/ bit 1
    2 -> { email = ...; seen = seen or 0b100 } \/\/ bit 2 (optional)
}

\/\/ Golden mask contains bits for all REQUIRED fields
val goldenMask = 0b011  \/\/ name (bit 0) and age (bit 1) are required

\/\/ Single comparison validates all required fields
if (seen and goldenMask != goldenMask) {
    throwMissingFieldException(seen, goldenMask, descriptor)
}

This is a gold. A single bitwise AND operation validates all required fields simultaneously. The golden mask is computed at compile time based on which properties have default values (optional) and which don’t (required).

For classes with more than 32 properties, the plugin generates multiple mask integers:

var seen0 = 0  \/\/ Properties 0-31
var seen1 = 0  \/\/ Properties 32-63

val goldenMask0 = 0b...
val goldenMask1 = 0b...

if (seen0 and goldenMask0 != goldenMask0 ||
    seen1 and goldenMask1 != goldenMask1) {
    throwArrayMissingFieldException(intArrayOf(seen0, seen1),
                                     intArrayOf(goldenMask0, goldenMask1),
                                     descriptor)
}

The error reporting is also smart. The throwMissingFieldException function uses the seen mask and golden mask to determine exactly which fields are missing, providing a clear error message without requiring additional bookkeeping.

Serializer resolution: Finding the right serializer for each type

When generating serialization code for a property, the plugin must determine which serializer to use. This resolution follows a priority order:

fun findSerializerForType(type: IrType, property: IrProperty): IrExpression {
    \/\/ 1. Explicit serializer annotation
    property.getSerializableWith()?.let { return instantiate(it) }

    \/\/ 2. Contextual serializer
    if (property.hasContextualAnnotation()) {
        return getContextualSerializer(type)
    }

    \/\/ 3. Primitive types have built-in serializers
    if (type.isPrimitiveType()) {
        return getBuiltInSerializer(type)  \/\/ Int.serializer(), String.serializer(), etc.
    }

    \/\/ 4. Collections and maps
    if (type.isCollection()) {
        val elementSerializer = findSerializerForType(type.elementType)
        return ListSerializer(elementSerializer)
    }

    \/\/ 5. Enum classes
    if (type.isEnum()) {
        return getEnumSerializer(type)
    }

    \/\/ 6. Other @Serializable classes
    if (type.hasSerializableAnnotation()) {
        return type.classSerializer()  \/\/ Type.$serializer
    }

    \/\/ 7. Polymorphic fallback for interfaces
    if (type.isInterface()) {
        return PolymorphicSerializer(type)
    }

    throw SerializationException("No serializer found for $type")
}

This resolution happens at compile time, and the result is baked into the generated code. The generated childSerializers() method returns an array of all serializers needed for the class’s properties, enabling format implementations to introspect the complete serialization structure.

Real world application: Custom serializers in production SDKs

Understanding the generated serializer structure helps when building custom serializers. Production SDKs like RevenueCat’s Android SDK leverage this knowledge to build robust serialization for complex API responses.

For example, when dealing with backend responses that might include unknown enum values or polymorphic types, a custom deserializer with defaults becomes necessary:

internal abstract class SealedDeserializerWithDefault<T : Any>(
    private val serialName: String,
    private val serializerByType: Map<String, () -> KSerializer<out T>>,
    private val defaultValue: (type: String) -> T,
    private val typeDiscriminator: String = "type",
) : KSerializer<T> {

    override val descriptor: SerialDescriptor = buildClassSerialDescriptor(serialName) {
        element(typeDiscriminator, String.serializer().descriptor)
    }

    override fun deserialize(decoder: Decoder): T {
        val jsonDecoder = decoder as? JsonDecoder
            ?: throw SerializationException("Can only deserialize from JSON")
        val jsonObject = jsonDecoder.decodeJsonElement().jsonObject
        val type = jsonObject[typeDiscriminator]?.jsonPrimitive?.content

        return serializerByType[type]?.let { serializerFactory ->
            jsonDecoder.json.decodeFromJsonElement(serializerFactory(), jsonObject)
        } ?: defaultValue(type ?: "null")
    }

    override fun serialize(encoder: Encoder, value: T) {
        throw NotImplementedError("Serialization not needed for API responses")
    }
}

This pattern mirrors how the plugin generates deserializers but adds fallback behavior for unknown types. It reads the type discriminator, looks up the appropriate serializer, and falls back to a default when the type is unknown. This is essential for backward compatibility when servers add new types that older clients don’t recognize.

Similarly, enum deserialization with defaults handles unknown values gracefully:

internal abstract class EnumDeserializerWithDefault<T : Enum<T>>(
    private val valuesByType: Map<String, T>,
    private val defaultValue: T,
) : KSerializer<T> {

    override val descriptor: SerialDescriptor =
        PrimitiveSerialDescriptor(defaultValue.javaClass.simpleName, PrimitiveKind.STRING)

    override fun deserialize(decoder: Decoder): T {
        val key = decoder.decodeString()
        return valuesByType[key] ?: defaultValue
    }

    override fun serialize(encoder: Encoder, value: T) {
        throw NotImplementedError("Serialization not needed")
    }
}

These patterns work because they follow the same interface contract that the generated serializers implement. Understanding the generated structure makes building compatible custom serializers straightforward.

K1 vs. K2: Supporting both compiler frontends

The Kotlin compiler is undergoing a major transition from the K1 frontend to the K2 frontend (based on FIR, Frontend Intermediate Representation). The kotlinx.serialization plugin must support both during this transition.

The K1 support uses descriptor based APIs:

class SerializationResolveExtension : SyntheticResolveExtension {
    override fun getSyntheticNestedClassNames(thisDescriptor: ClassDescriptor): List<Name> {
        if (thisDescriptor.hasSerializableAnnotation()) {
            return listOf(Name.identifier("\$serializer"))
        }
        return emptyList()
    }

    override fun generateSyntheticClasses(
        thisDescriptor: ClassDescriptor,
        result: MutableCollection<DeclarationDescriptor>
    ) {
        \/\/ Generate synthetic $serializer class descriptor
    }
}

The K2 support uses the new FIR based APIs:

class SerializationFirResolveExtension : FirDeclarationGenerationExtension {
    override fun generateNestedClassLikeDeclaration(
        owner: FirClassSymbol<*>,
        name: Name
    ): FirClassLikeSymbol<*>? {
        if (name.identifier == "\$serializer" && owner.hasSerializableAnnotation()) {
            return generateSerializerClass(owner)
        }
        return null
    }
}

The key observation is that both frontends ultimately feed into the same IR generation phase. The IR lowering code that generates method bodies is shared between K1 and K2. This separation of concerns allows the plugin to support both frontends while maintaining a single implementation of the actual code generation logic.

Conclusion

In this article, you’ve explored the internal mechanisms of the kotlinx.serialization compiler plugin, from its two pass IR generation strategy to the golden mask optimization for required field validation. The plugin transforms simple @Serializable annotations into efficient, type safe serializers without requiring reflection or runtime code generation.

Of course, you don’t have any issues without understanding these internals mechanisms, but it will definitely help you make informed decisions when building custom serializers, debugging serialization issues, or evaluating kotlinx.serialization for your projects in any ways. The design choices, like compile time code generation, format agnostic descriptors, and bitmask validation, reflect careful engineering for both performance and flexibility.

Whether you’re building a multiplatform application that needs consistent serialization across JVM, JS, and Native, implementing custom serializers for complex API responses, or simply curious about how your @Serializable classes become working serializers, this knowledge provides the foundation for working effectively with one of Kotlin’s most important libraries.