Painless JSON with Kotlin and jackson

KotlinJSONJacksonRESTSpringBoot

It seems that many backends that provide a REST API end up being glorified proxies that move JSON from one place to another. This is especially true if you are trying to keep those backends as simple as possible (Microservices anyone?). Having good tools to parse and produce JSON can thus make a big impact in keeping the code tidy and compact. I want to talk about my experience using Kotlin and Jackson for this.

I remember that dealing with JSON in Java used to be pretty painful back in the day, as you had to write a ton of code to map objects. That is in fact what originally led me to use Ruby. Things have changed a lot (for the good!) since then. Nowadays using Kotlin and Jackson you can deal with JSON with minimal effort. Jackson is a very powerful library, but you can get lost easily. If have a bunch of examples of classes with different requirements being parsed with it, plus some code to integrate it in your workflow. In this case, I will be using SpringBoot.

Serialize/Deserialize

We will be using data classes to represent the entities that will get converted to a from JSON. They are the equivalent of using the @Value annotation in Lombok, with first class support from the language. They are immutable (yay!), and have convenience methods like equals and toString out of the box.

You can use an ObjectMapper to do the parsing, although this can be configured in SpringBoot to be done mostly automatically, which I will show later. I have a User entity with two fields that I want to convert to JSON and back.

data class User(val id: String, val age: Int)

fun User.toJson(): String = ObjectMapper().writeValueAsString(this)
fun String.toUser(): User = ObjectMapper().readValue(this)

for simple cases, just defining the data class is enough, as long as you have the right module. There are a bunch of extra configurations that you can do on top of it, though. Many of them can be controlled with annotations, which can be used to make the code a lot more compact. They can also be used to make your code an unmaintainable mess, so don’t abuse them.

Nullability

If some of the fields are optional, you can either provide default values in case they aren’t there

data class User(
  val id: String = ""
)

or allow them to be null

data class User(
  val id: String?
)

not doing anything will make the parsing fail with an exception, which I actually find a good thing.

Aliasing

If you are parsing your object from a different source that uses different attribute names, but still want to keep a canonical representation, @JsonAlias is your friend.

data class User(
  @JsonAlias("userId")
  val id: String
)

this will correctly parse something like

{
  "userId": "123"
}

Ignore properties

Maybe you are parsing an object with a ton of fields that you don’t need. If you are not going to use it in your code you really should avoid adding them, as that makes it harder to understand what is needed and what is not. @JsonIgnoreProperties can be used for this.

@JsonIgnoreProperties(ignoreUnknown = true)
data class User(val id: String)

Different representations

If your backend is acting as a proxy, you will be reading your data from somewhere and passing it to your client. In this case you might want to skip some fields in the serialization in order to give your client exactly the fields they need. You can accomplish this by customizing the access.

data class User(
  val id: String = "",
  @JsonProperty(access = JsonProperty.Access.WRITE_ONLY)
  val age: Int
)

The serialization of this object won’t contain the age, but it is available in our code. This approach does not scale that well, however. If you find that you have two different representations of the same entity and are adding a ton of annotations to only use one class, it’s probably better to split it in two different classes and provide a method one into the other.

This underscores an important point. You don’t need to use annotations and implicit conversions for everything, maybe in some places having dedicated converters is just more readable, more so if you want to attach some logic to that process.

And if you want more…

This is just a small part of what is possible to do. You can really control every aspect of the serialization/deserialization process. Have a look at this post if you want to know about other options.


Getting away from untyped strings

In JSON you tend to use strings to represent many entities. Any id type like a user id, or something like a language code, for example. I really prefer mapping them to dedicated classes in my code. I’ve seen many bugs where the wrong entity is used when that could be prevented directly by the compiler. Taking a UserId as an example, I like to model it as follows:

  • It should be an immutable data class
  • It should not force a change in the structure of the JSON (i.e. no nesting)
  • Serialize/Deserialize should work out of the box
data class UserId(private val value: String) {
    companion object {
        @JvmStatic
        @JsonCreator
        fun create(value: String) = UserId(value.toLowerCase())
    }

    @JsonValue
    override fun toString() = value
}

by using a data class, we get an immutable object that represents this entity. We can do relatively little with it. In this case we don’t even want access to the internal fields. We are going to compare instances directly, and if we need to get a string representation we’ll do that through the toString method.

The serialization happens through the @JsonValue annotation, where we use the value directly. If we modify our User class that we have been using before, it would look like this.

data class User(val id: UserId, val age: Int)

That class will be serialized to this JSON

{
  "id": "123",
  "age": 20
}

This matches how most clients (specially a frontend) would expect this structure to look like, without sacrificing any safety in the backend.

The deserialization happens automatically. However, I like to define a static constructor (using the @JvmStatic and @JsonCreator annotations) so that I can do things like sanitizing the input before generating my instance. This helps making sure our models are in a consistent state.

Since Kotlin 1.3, a new concept called inline classes has been introduced, which might match better with this use case. Jackson has some trouble deserializing it properly in nested objects as of 16/06/19, so I could not really replace my data classes with it so far. There is an open issue in Github to follow.


SpringBoot integration

This is the last piece of the puzzle. We can manually use an ObjectMapper and convert things explicitly. It is much easier if that happens on its own. The good news is that there is not really much to do here other than adding the jackson-module-kotlin as a dependency:

implementation("com.fasterxml.jackson.module:jackson-module-kotlin:${jacksonVersion}")

If you are using the latest versions of everything, that will be enough. If the spring magic does not happen on its own (spring does a lot of magic that I don’t really understand), you can do it manually. You can use a @Configuration so that your controllers can map to and from JSON automatically.

@Configuration
class JacksonConfiguration {
    @Bean
    fun mappingJackson2HttpMessageConverter(): MappingJackson2HttpMessageConverter {
        return MappingJackson2HttpMessageConverter().apply {
            this.objectMapper = ObjectMapper().apply {
                registerModule(KotlinModule())
            }
        }
    }
}

if you are making REST request to another service, you can build a custom RestTemplate doing the same:

open class DefaultRestTemplate(baseUrl: String) :
        RestTemplate(HttpComponentsClientHttpRequestFactory(
                HttpClientBuilder.create().useSystemProperties().build())) {
    init {
        uriTemplateHandler = DefaultUriBuilderFactory(baseUrl)
        messageConverters = jacksonConverter(messageConverters)
    }

    private fun jacksonConverter(converters: MutableList<HttpMessageConverter<*>>): List<HttpMessageConverter<*>> {
        val jsonConverter = MappingJackson2HttpMessageConverter()
        jsonConverter.objectMapper = jacksonObjectMapper().apply {
            registerModule(KotlinModule())
        }
        converters.add(jsonConverter)
        return converters
    }
}

again, all this should happen by just adding the library to the classpath. Use this as a fallback in case that does not work for some reason. Also, this template can be extended to use a base url, receive environment variables (to include the keystore for instance), or automatically add certain headers to your requests.

PathVariables are not JSON

Now that we are waist deep in automated JSON mapping, I’m getting ambitious. As mentioned above, we are no longer using plain strings but proper domain classes. Let’s say you have a route like GET /users/:userId. The controller would look like this:

@RestController
@RequestMapping("/users", produces = [MediaType.APPLICATION_JSON_VALUE])
class HelloController {
    @GetMapping("{userId}")
    fun user(@PathVariable("userId") userId: UserId): ResponseEntity<User>
}

If you make a request to this route, the userId will get parsed automatically, but our custom create method won’t get called, because this is a URL, not JSON. We didn’t come this far to start parsing strings manually again. Let’s fix this by using a custom converter.

@Configuration
class ConverterConfiguration : WebMvcConfigurer {
    override fun addFormatters(registry: FormatterRegistry) {
        registry.addConverter(userId())
    }

    private fun userId(): Converter<String, UserId> {
        return Converter<String, UserId> { source -> UserId.create(source) }
    }
}

That’s it. Now we can be sure that those pesky strings are not floating through our app at any point of the flow of a request.

Summary

Jackson is a very powerful library, and to be honest you can overdo it with all the annotations. If you use them judiciously, though, working with JSON becomes very easy, while keeping a good amount of type safety in the process. For testing, this goes really well with recorded APIs using WireMock.