Painless JSON with Kotlin and jackson

KotlinJSONJacksonRESTSpringBoot

It seems that many backends that provide a REST API end up being glorified proxies that move JSON from one place to another. It is especially true if you are trying to keep those backends as simple as possible (Microservices anyone?). Having the right tools to parse and produce JSON can thus make a big impact in keeping the code tidy and compact. I want to talk about my experience using Kotlin and Jackson for this.

I remember that dealing with JSON in Java used to be pretty painful back in the day, as you had to write a ton of code to map objects. That is what initially led me to use Ruby. Things have changed a lot (for the better!) since then. Nowadays, using Kotlin and Jackson you can deal with JSON with minimal effort. Jackson is a mighty library, but you can get lost easily. I have a bunch of examples showing how to parse different classes, plus some code to integrate it into your workflow. In this case, I will be using SpringBoot.

Serialize/Deserialize

We will be using data classes to represent the entities that will get converted to a from JSON. They are the equivalent of using the @Value annotation in Lombok, with first-class support from the language. They are immutable (yay!) and have convenience methods like equals and toString out of the box.

You can use an ObjectMapper to do the parsing, although you can configure SpringBoot to do it mostly automatically, which I will show later. I have a User entity with two fields that I want to convert to JSON and back.

data class User(val id: String, val age: Int)

fun User.toJson(): String = ObjectMapper().writeValueAsString(this)
fun String.toUser(): User = ObjectMapper().readValue(this)

For simple cases, just defining the data class is enough, as long as you have the right module. There are a bunch of extra configurations that you can do on top of it, though. Many of them can be controlled with annotations, which make the code a lot more compact. Abusing them will turn your code into an unmaintainable mess, though.

Nullability

If some of the fields are optional, you provide default values.

data class User(
  val id: String = ""
)

You can also allow them to be null.

data class User(
  val id: String?
)

not doing anything will make the parsing fail with an exception, which I find a good thing.

Aliasing

If you are parsing your object from a different source that uses different attribute names, but still want to keep a canonical representation, @JsonAlias is your friend.

data class User(
  @JsonAlias("userId")
  val id: String
)

this will correctly parse something like

{
  "userId": "123"
}

Ignore properties

Maybe you are parsing an object with a ton of fields that you don’t need. If you are not going to use it in your code, you really should avoid adding them, as that makes it harder to understand what is needed and what is not. @JsonIgnoreProperties can be used for this.

@JsonIgnoreProperties(ignoreUnknown = true)
data class User(val id: String)

Different representations

If your backend is acting as a proxy, you will be reading your data from somewhere and passing it to your client. In this case, you might want to skip some fields in the serialization to give your client precisely the fields it needs. You can accomplish this by customizing the access property.

data class User(
  val id: String = "",
  @JsonProperty(access = JsonProperty.Access.WRITE_ONLY)
  val age: Int
)

The serialization of this object won’t contain the age, but it is available in our code. This approach does not scale that well, however. If you find that you have two different representations of the same entity and are adding a ton of annotations to use one class, it’s probably better to split it into two distinct classes and provide a method one into the other.

This underscores an important point. You don’t need to use annotations and implicit conversions for everything. In some places having dedicated converters is just more readable, more so if you want to attach some logic to that process.

And if you want more…

This article shows just a small part of what is possible to do. You can control every aspect of the serialization/deserialization process. Have a look at this post if you want to know about other options.

Getting away from untyped strings

In JSON you tend to use strings to represent many entities. Any id type like a user id, or something like a language code, for example. I prefer mapping them to dedicated classes in my code. I’ve seen many bugs where the wrong entity is used when that could be prevented directly by the compiler. Taking a UserId as an example, I like to model it as follows:

  • It should be an immutable data class
  • It should not force a change in the structure of the JSON (i.e., no nesting)
  • Serialize/Deserialize should work out of the box
data class UserId(private val value: String) {
    companion object {
        @JvmStatic
        @JsonCreator
        fun create(value: String) = UserId(value.toLowerCase())
    }

    @JsonValue
    override fun toString() = value
}

By using a data class, we get an immutable object that represents this entity. We can do relatively little with it. In fact, we don’t even want access to the internal fields. We are going to compare instances directly, and if we need to get a string representation, we’ll do that through the toString method.

The serialization happens through the @JsonValue annotation, where we use the value directly. If we modify our User class that we have been using before, it will look like this.

data class User(val id: UserId, val age: Int)

That class serializes to this JSON

{
  "id": "123",
  "age": 20
}

That representation matches how most clients (especially a frontend) would expect this structure to look like, without sacrificing any safety in the backend.

The deserialization happens automatically. However, I like to define a static constructor (using the @JvmStatic and @JsonCreator annotations) so that I can do things like sanitizing the input before generating my instance. This helps to make sure our models are in a consistent state.

Since Kotlin 1.3, a new concept called inline classes has been introduced, which might match better with this use case. Jackson has some trouble deserializing it properly in nested objects as of 16/06/19, so I could not replace my data classes with it so far. There is an open issue in Github to follow.

SpringBoot integration

Here we get the last piece of the puzzle. We can manually use an ObjectMapper and convert things explicitly. It is much easier if that happens on its own. The good news is that there is not much to do here other than adding the jackson-module-kotlin as a dependency:

implementation("com.fasterxml.jackson.module:jackson-module-kotlin:${jacksonVersion}")

If you are using the latest versions of everything, that will be enough. If the spring magic does not happen on its own (spring does a lot of magic that I don’t quite understand), you can do it manually. You can use a @Configuration so that your controllers can map to and from JSON automatically.

@Configuration
class JacksonConfiguration {
    @Bean
    fun mappingJackson2HttpMessageConverter(): MappingJackson2HttpMessageConverter {
        return MappingJackson2HttpMessageConverter().apply {
            this.objectMapper = ObjectMapper().apply {
                registerModule(KotlinModule())
            }
        }
    }
}

if you are making REST request to another service, you can build a custom RestTemplate doing the same:

open class DefaultRestTemplate(baseUrl: String) :
        RestTemplate(HttpComponentsClientHttpRequestFactory(
                HttpClientBuilder.create().useSystemProperties().build())) {
    init {
        uriTemplateHandler = DefaultUriBuilderFactory(baseUrl)
        messageConverters = jacksonConverter(messageConverters)
    }

    private fun jacksonConverter(converters: MutableList<HttpMessageConverter<*>>): List<HttpMessageConverter<*>> {
        val jsonConverter = MappingJackson2HttpMessageConverter()
        jsonConverter.objectMapper = jacksonObjectMapper().apply {
            registerModule(KotlinModule())
        }
        converters.add(jsonConverter)
        return converters
    }
}

Again, all this should happen by just adding the library to the classpath. Use this as a fallback in case that does not work for some reason. Also, this template can be extended to use a base URL, receive environment variables (to include the keystore for instance), or automatically add certain headers to your requests.

PathVariables are not JSON

Now that we are waist-deep in automated JSON mapping, I’m getting ambitious. As mentioned above, we are no longer using plain strings but proper domain classes. Let’s say you have a route like GET /users/:userId. The controller would look like this:

@RestController
@RequestMapping("/users", produces = [MediaType.APPLICATION_JSON_VALUE])
class HelloController {
    @GetMapping("{userId}")
    fun user(@PathVariable("userId") userId: UserId): ResponseEntity<User>
}

If you send a request to this route, the userId will get parsed automatically, but our custom create method won’t get called, because this is a URL, not JSON. We didn’t come this far to start parsing strings manually again. Let’s fix this by using a custom converter.

@Configuration
class ConverterConfiguration : WebMvcConfigurer {
    override fun addFormatters(registry: FormatterRegistry) {
        registry.addConverter(userId())
    }

    private fun userId(): Converter<String, UserId> {
        return Converter<String, UserId> { source -> UserId.create(source) }
    }
}

That’s it. Now we can be sure that those pesky strings are not floating through our app at any point in the flow of a request.

Summary

Jackson is a very powerful library, and to be honest, you can overdo it with all the annotations. If you use them judiciously, though, working with JSON becomes very easy while keeping a good amount of type safety in the process. For testing, this goes well with recorded APIs using WireMock.

EDIT 25/12/2019: Grammar review

Related Posts

Keep Reading