Go JSON (Un)Marshalling, Missing Fields and Omitempty
When working with JSON in Go recently, I wondered: “What the heck does json:omitempty do?”. I was looking for ways to handle missing fields while unmarshalling some JSON into a struct, and got confused for a while.
Here are a few examples, as a reminder how most of it works.
Those are not fancy edgecases. Those are self-contained, completely functional code snippets which should be easy to follow. You’re welcome future-me.
Just parse some JSON
You provide field names as annotations. If the JSON payload matches the struct 1:1, it doesn’t get more complicated than this:
package main
import (
"encoding/json"
"fmt"
"os"
)
type Dummy struct {
Name string `json:"name"`
Number int64 `json:"number"`
Pointer *string `json:"pointer"`
}
func main() {
data := []byte(`
{
"name": "Mr Dummy",
"number": 4,
"pointer": "yes"
}
`)
var dummy Dummy
err := json.Unmarshal(data, &dummy)
if err != nil {
fmt.Println("An error occured: %v", err)
os.Exit(1)
}
// we want to print the field names as well
fmt.Printf("%+v\n", dummy)
}
The output is a bit ugly, but it’s good enough to see what’s going on:
{Name:Mr Dummy Number:4 Pointer:0xc00000e280}
All fields got used. The pointer is non-nil, it points to a string value (you can’t see here, but the pointed-to string contains “yes”).
So far so good.
Omit fields when parsing JSON
You got way too much stuff in the JSON data. You don’t need all of it.
Nothing fancy needed here. Just don’t specify those extra fields in your struct. Go won’t mind.
In fact, you could use the same JSON data with multiple small structs to get out parts of the data one after the other.
Cool trick: you can parse the same JSON using multiple structs, to adjust to different contents dynamically this way.
package main
import (
"encoding/json"
"fmt"
"os"
)
// let's change the struct:
type Dummy struct {
// we only care about the name now
Name string `json:"name"`
// all other fields are gone
}
func main() {
data := []byte(`
{
"name": "Mr Dummy",
"number": 4,
"pointer": "yes"
}
`)
var dummy Dummy
err := json.Unmarshal(data, &dummy)
if err != nil {
fmt.Println("An error occured: %v", err)
os.Exit(1)
}
// we want to print the field names as well
fmt.Printf("%+v\n", dummy)
}
Output:
{Name:Mr Dummy}
Just one field. Nothing else got used. Neat!
Ignore JSON fields when parsing it, even though your struct has them
Here’s where the json:"-" annotation comes in handy. You tell Go
to leave this particular struct field alone, when pumping values from
JSON into it.
package main
import (
"encoding/json"
"fmt"
"os"
)
type Dummy struct {
Name string `json:"name"`
Number int64 `json:"number"`
Pointer *string `json:"-"` // we want to ignore JSON for this one
}
func main() {
data := []byte(`
{
"name": "Mr Dummy",
"number": 4,
"pointer": "yes"
}
`)
var dummy Dummy
err := json.Unmarshal(data, &dummy)
if err != nil {
fmt.Println("An error occured: %v", err)
os.Exit(1)
}
// we want to print the field names as well
fmt.Printf("%+v\n", dummy)
}
Output:
{Name:Mr Dummy Number:4 Pointer:<nil>}
Even though the JSON specifies something for the pointer field,
our structs ignores the value and leaves its Pointer unchanged (at nil).
Don’t output fields when writing out JSON
json:"-" once again! It works both ways.
Even though the struct has the field, we won’t see it in the JSON output.
package main
import (
"encoding/json"
"fmt"
"os"
)
type Dummy struct {
Name string `json:"name"`
Number int64 `json:"number"`
Pointer *string `json:"-"`
}
func main() {
pointer := "yes"
dummy := Dummy{
Name: "Mr Dummy",
Number: 4,
Pointer: &pointer,
}
data, err := json.Marshal(dummy)
if err != nil {
fmt.Println("An error occured: %v", err)
os.Exit(1)
}
fmt.Println(string(data))
}
Output:
{"name":"Mr Dummy","number":4}
See ma? No pointer field.
Leave out fields if they are empty when writing out JSON
json:omitempty to the rescue. We only want to
ignore the field of the struct if it’s empty.
Now there’s a gotcha here: you gotta be pretty sure what Go takes as empty.
package main
import (
"encoding/json"
"fmt"
"os"
)
type Dummy struct {
Name string `json:"name,omitempty"`
Number int64 `json:"number,omitempty"`
Pointer *string `json:"pointer,omitempty"`
}
func main() {
pointer := "yes"
dummyComplete := Dummy{
Name: "Mr Dummy",
Number: 4,
Pointer: &pointer,
}
data, err := json.Marshal(dummyComplete)
if err != nil {
fmt.Println("An error occured: %v", err)
os.Exit(1)
}
fmt.Println(string(data))
// ALL of those are considered empty by Go
dummyNilPointer := Dummy{
Name: "",
Number: 0,
Pointer: nil,
}
dataNil, err := json.Marshal(dummyNilPointer)
if err != nil {
fmt.Println("An error occured: %v", err)
os.Exit(1)
}
fmt.Println(string(dataNil))
}
Output
{"name":"Mr Dummy","number":4,"pointer":"yes"}
{}
The second struct is completely empty. All fields are left out
due to their values thanks to the omitempty annotation.
Zero-values and nil pointers are considered
empty, and this can feel counterintuitive sometimes. If you want to output
a 0, you better work with something like *int64, if you really want to use
omitempty.
But there’s no need to make it trickier than it needs to be. Everything else is pretty much straightforward.
Additional Resources
If you want to read up more on this topic, and get details for more tricky edgecases, you can check out those links:
Happy (Un)Marshalling!