JSON Encoding

Let’s move on to something a bit more exciting and look at how to encode native Go objects (like maps, structs and slices) to JSON.

At a high-level, Go’s encoding/json package provides two options for encoding things to JSON. You can either call the json.Marshal() function, or you can declare and use a json.Encoder type.

We’ll explain how both approaches work in this chapter, but — for the purpose of sending JSON in a HTTP response — using json.Marshal() is generally the better choice. So let’s start with that.

The way that json.Marshal() works is conceptually quite simple — you pass a native Go object to it as a parameter, and it returns a JSON representation of that object in a []byte slice. The function signature looks like this:

func Marshal(v any) ([]byte, error)

Let’s jump in and update our healthcheckHandler so that it uses json.Marshal() to generate a JSON response directly from a Go map — instead of using a fixed-format string like we were before. Like so:

package main

import (
    "encoding/json" // New import
    "net/http"
)

func (app *application) healthcheckHandler(w http.ResponseWriter, r *http.Request) {
    // Create a map which holds the information that we want to send in the response.
    data := map[string]string{
        "status":      "available",
        "environment": app.config.env,
        "version":     version,
    }

    // Pass the map to the json.Marshal() function. This returns a []byte slice 
    // containing the encoded JSON. If there was an error, we log it and send the client
    // a generic error message.
    js, err := json.Marshal(data)
    if err != nil {
        app.logger.Error(err.Error())
        http.Error(w, "The server encountered a problem and could not process your request", http.StatusInternalServerError)
        return
    }

    // Append a newline to the JSON. This is just a small nicety to make it easier to 
    // view in terminal applications.
    js = append(js, '\n')

    // At this point we know that encoding the data worked without any problems, so we
    // can safely set any necessary HTTP headers for a successful response.
    w.Header().Set("Content-Type", "application/json")

    // Use w.Write() to send the []byte slice containing the JSON as the response body.
    w.Write(js)
}

If you restart the API and visit localhost:4000/v1/healthcheck in your browser, you should now get a response similar to this:

That’s looking good — we can see that the map has automatically been encoded to a JSON object for us, with the key/value pairs in the map appearing as alphabetically sorted key/value pairs in the JSON object.

Creating a writeJSON helper method

As our API grows we’re going to be sending a lot of JSON responses, so it makes sense to move some of this logic into a reusable writeJSON() helper method.

As well as creating and sending the JSON, we want to design this helper so that we can include arbitrary headers in successful responses later, such as a Location header after creating a new movie in our system.

If you’re coding-along, open the cmd/api/helpers.go file again and create the following writeJSON() method:

package main

import (
    "encoding/json" // New import
    "errors"
    "net/http"
    "strconv"

    "github.com/julienschmidt/httprouter"
)

...

// Define a writeJSON() helper for sending responses. This takes the destination
// http.ResponseWriter, the HTTP status code to send, the data to encode to JSON, and a 
// header map containing any additional HTTP headers we want to include in the response.
func (app *application) writeJSON(w http.ResponseWriter, status int, data any, headers http.Header) error {
    // Encode the data to JSON, returning the error if there was one.
    js, err := json.Marshal(data)
    if err != nil {
        return err
    }

    // Append a newline to make it easier to view in terminal applications.
    js = append(js, '\n')

    // At this point, we know that we won't encounter any more errors before writing the
    // response, so it's safe to add any headers that we want to include. We loop
    // through the header map and add each header to the http.ResponseWriter header map.
    // Note that it's OK if the provided header map is nil. Go doesn't throw an error
    // if you try to range over (or generally, read from) a nil map.
    for key, value := range headers {
        w.Header()[key] = value
    }

    // Add the "Content-Type: application/json" header, then write the status code and
    // JSON response.
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(status)
    w.Write(js)

    return nil
}

Now that the writeJSON() helper is in place, we can significantly simplify the code in healthcheckHandler, like so

package main

import (
    "net/http"
)

func (app *application) healthcheckHandler(w http.ResponseWriter, r *http.Request) {
    data := map[string]string{
        "status":      "available",
        "environment": app.config.env,
        "version":     version,
    }

    err := app.writeJSON(w, http.StatusOK, data, nil)
    if err != nil {
        app.logger.Error(err.Error())
        http.Error(w, "The server encountered a problem and could not process your request", http.StatusInternalServerError)
    }
}

If you run the application again now, everything will compile correctly and a request to the GET /v1/healthcheck endpoint should result in the same HTTP response as before.

Additional Information

How different Go types are encoded

In this chapter we’ve been encoding a map[string]string type to JSON, which resulted in a JSON object with JSON strings as the values in the key/value pairs. But Go supports encoding many other native types too.

The following table summarizes how different Go types are mapped to JSON data types during encoding:

The last two of these are special cases which deserve a bit more explanation:

• Go time.Time values (which are actually a struct behind the scenes) will be encoded as a JSON string in RFC 3339 format like "2020-11-08T06:27:59+01:00", rather than as a JSON object.

• A []byte slice will be encoded as a base64-encoded JSON string, rather than as a JSON array. So, for example, a byte slice of []byte{'h','e','l','l','o'} would appear as "aGVsbG8=" in the JSON output. The base64 encoding uses padding and the standard character set.

A few other important things to mention:

• Encoding of nested objects is supported. So, for example, if you have a slice of structs in Go that will encode to an array of objects in JSON.

• Channels, functions and complex number types cannot be encoded. If you try to do so, you’ll get a json.UnsupportedTypeError error at runtime.

• Any pointer values will encode as the value pointed to.

Using json.Encoder

At the start of this chapter I mentioned that it’s also possible to use Go’s json.Encoder type to perform the encoding. This allows you to encode an object to JSON and write that JSON to an output stream in a single step.

For example, you could use it in a handler like this:

func (app *application) exampleHandler(w http.ResponseWriter, r *http.Request) {
    data := map[string]string{
        "hello": "world",
    }

    // Set the "Content-Type: application/json" header on the response.
    w.Header().Set("Content-Type", "application/json")

    // Use the json.NewEncoder() function to initialize a json.Encoder instance that
    // writes to the http.ResponseWriter. Then we call its Encode() method, passing in 
    // the data that we want to encode to JSON (which in this case is the map above). If
    // the data can be successfully encoded to JSON, it will then be written to our 
    // http.ResponseWriter.
    err := json.NewEncoder(w).Encode(data)
    if err != nil {
        app.logger.Error(err.Error())
        http.Error(w, "The server encountered a problem and could not process your request", http.StatusInternalServerError)
    }
}

This pattern works, and it’s very neat and elegant, but if you consider it carefully you might notice a slight problem…

When we call json.NewEncoder(w).Encode(data) the JSON is created and written to the http.ResponseWriter in a single step, which means there’s no opportunity to set HTTP response headers conditionally based on whether the Encode() method returns an error or not.

Imagine, for example, that you want to set a Cache-Control header on a successful response, but not set a Cache-Control header if the JSON encoding fails and you have to return an error response.

Implementing that cleanly while using the json.Encoder pattern is quite difficult.

You could set the Cache-Control header and then delete it from the header map again in the event of an error — but that’s pretty hacky.

Another option is to write the JSON to an interim bytes.Buffer instead of directly to the http.ResponseWriter. You can then check for any errors, before setting the Cache-Control header and copying the JSON from the bytes.Buffer to http.ResponseWriter. But once you start doing that, it’s simpler and cleaner (as well as slightly faster) to use the alternative json.Marshal() approach instead.

Performance of json.Encoder and json.Marshal

Talking of speed, you might be wondering if there’s any performance difference between using json.Encoder and json.Marshal(). The short answer to that is yes… but the difference is small and in most cases you should not worry about it.

The following benchmarks demonstrate the performance of the two approaches using the code in this gist (note that each benchmark test is repeated three times):

$ go test -run=^$ -bench=. -benchmem -count=3 -benchtime=5s
goos: linux
goarch: amd64
BenchmarkEncoder-8     3477318     1692 ns/op     1046 B/op    15 allocs/op
BenchmarkEncoder-8     3435145     1704 ns/op     1048 B/op    15 allocs/op
BenchmarkEncoder-8     3631305     1595 ns/op     1039 B/op    15 allocs/op
BenchmarkMarshal-8     3624570     1616 ns/op     1119 B/op    16 allocs/op
BenchmarkMarshal-8     3549090     1626 ns/op     1123 B/op    16 allocs/op
BenchmarkMarshal-8     3548070     1638 ns/op     1123 B/op    16 allocs/op

In these results we can see that json.Marshal() requires ever so slightly more memory (B/op) than json.Encoder, and also makes one extra heap memory allocation (allocs/op).

There’s no obvious observable difference in the average runtime (ns/op) between the two approaches. Perhaps with a larger benchmark sample or a larger data set a difference might become clear, but it’s likely to be in the order of microseconds, rather than anything larger.

Additional JSON encoding nuances

Encoding things to JSON in Go is mostly quite intuitive. But there are a handful of behaviors which might either catch you out or surprise you when you first encounter them.

We’ve already mentioned a couple of these in this chapter (in particular — map entries being sorted alphabetically and byte slices being base-64 encoded), but I’ve included a full list in this appendix.

Using Iterators

If you like, you could update the writeJSON() function to use the iterator types and helper functions introduced in Go 1.23.

Specifically, instead of using a regular range statement to loop over the headers map and copy the data to w.Header(), you could leverage the generic maps.All() and maps.Insert() functions instead. So instead of writing this:

func (app *application) writeJSON(w http.ResponseWriter, status int, data envelope, headers http.Header) error {
    ...

    for key, value := range headers {
        w.Header()[key] = value
    }

    ...
}

You could use the iterator functionality instead and write this:

func (app *application) writeJSON(w http.ResponseWriter, status int, data envelope, headers http.Header) error {
    ...
    
    maps.Insert(w.Header(), maps.All(headers))

    ...
}

Both the iterator version and the regular range version will probably compile down to the same assembly code, so in terms of performance there’s unlikely to be a difference between the two.

Both versions also feel equally readable, despite being quite different. For both, it’s fairly easy to understand (or at least accurately guess) what the code is doing at a glance.

Where they start to differ is complexity behind-the-scenes. If you look at the function signatures for maps.All() and maps.Insert(), I think it’s fair to say that they are quite complicated — even as an experienced Go developer it takes some time to parse them and understand what these functions actually do and how to use them.

Because of that I think that the iterator version is overall less clear and potentially harder to maintain than using the basic range, especially for anyone who isn’t already familiar and comfortable with Go generics and iterators.

While Go iterators are a nice addition to the language, in this specific instance using them doesn’t seem a worthwhile trade off for saving a couple of lines of code. So for that reason, we’ll stick with using a regular range statement in writeJSON() throughout the rest of this book.