Type Safe Validation in Go with govy

Objects API

One of the key features of the Nobl9 platform is that it can be configured declaratively.

Users can work with a familiar YAML schema, derived from Kubernetes Objects’ API.

Each distinct configuration entity is called an object, the type of the object is marked by the kind field’s value.

Here’s an example of our Service object:

Service can represent an HTTP endpoint for an internal or external API, a database, or anything else you care about setting SLOs for. It is the “service” in Service Level Objectives ;)

For simplicity's sake we’ll focus on this particular object kind, we’re also going to disregard the metadata.labels field entirely. That being said, it’s good to keep in mind that objects such as SLO are vastly more complex and were the main reason we sought a different solution for validation in the first place.

If you’re interested in inspecting these more complex objects, feel free to visit our Go SDK and poke around!

Validation

One of the most important parts of defining, maintaining and developing such a vast and structured schema is validation.

Ideally the mechanism used to perform validation should:

• Return meaningful errors to the user (arguably the most important point).
• Return predictable and reproducible results.
• Be easy to reason about, provide a way to inspect the rules for a specific object’s property.
• Be easy to develop and maintain code wise (designed with developer experience in mind).
• Adhere to the language’s standards. Go is a statically-typed language and so our solution should be type-safe.

The ancient days

Declarative objects’ definitions were part of Nobl9 from the get go, so was the validation mechanism written in Go using the popular go-playground/validator library, which to this day is ruling the Go’s validation landscape.

Back then Go 1.14 had just been released and generics were in their early, experimental stage, yet to see their official release in 2 years.

Let us first define Service in Go code:

Validator library performs validation based on struct tags.

We define a struct tag key “validation” and assign it a comma separated list of values, where each value is a validation rule corresponding to either predefined or user-defined function. For instance, given the following struct tag: `validation:”required,eq=1”`, we can discern two rules, required which ensures the field has a non-zero value and eq which further specifies the value must be equal to 1.

The validation functions have the following shape:

• Accepts a validator.FieldLevel interface, it contains all the information and helper functions necessary to validate a struct field.
• Returns a boolean value. The validation rule defined by the function either passes or fails.

We can immediately see a problem here, validator functions do not return errors! This means there’s no easy way to dynamically construct error messages.

With that in mind let’s take a look at how validation rules for Service object used to look back in the validator days:

The output of the printed error is:

Key: 'Service.apiVersion' Error:Field validation for 'apiVersion' failed on the 'eq' tag

Key: 'Service.kind' Error:Field validation for 'kind' failed on the 'eq' tag

Key: 'Service.metadata.name' Error:Field validation for 'name' failed on the 'objectName' tag 

Key: 'Service.metadata.project' Error:Field validation for 'project' failed on the 'objectName' tag

The Good

Struct tag-based validation allows you to move fast with simple scenarios when there’s no need to create custom rules. Making sure apiVersion for Service object is equal to n9/v1alpha is as easy as adding eq=n9/v1alpha to the tag’s value string.

Since validator operates on struct tags and traverses the whole struct using reflection, it can automatically assign names to fields based on JSON tags.

The Bad

Validation functions operate on reflection, each requires explicit, runtime type casting which can cause code to panic if not handled with care. It strips your code of type-safety guard rails which are imposed by the compiler making it easier to make mistakes and harder to safely reuse validation functions. The developer needs to take extra care, making sure a given function can handle the validated value’s type.

Figuring out what set of validation rules applies to a given property can be non-trivial as there are many ways to include validation logic and there’s no easy way to inspect all the registered rules.

The Ugly

Default error messages returned by the validator convey very limited information, it’s not something you’d want to return to your API’s users. For instance, objectName function name in of itself does not tell the end user what’s the valid object name. Sure, we could try naming these functions more verbosely, but with more complex logic it becomes impractical and downright ugly. While validator comes with a mechanism for overriding default error messages in form of translation handlers, it is cumbersome to say the least, not to mention the fact that each predefined rule which we’re using must be explicitly translated as well.

The rough error messages were in fact the tipping point which pushed us towards finding a better validation solution.

Enter govy

With the shortcomings and pain points of the existing mechanism, we eventually started exploring our options. At the time, generics were already part of the language for some time and we’ve seen a clear potential there for a type-safe validation API. However, the open source libraries’ landscape looked barren, with few exceptions which under closer inspection were underwhelming.

There seemed to be only one path onwards, we needed to home cook.

Initial PoC for govy first saw daylight in October 2023 with this commit. We were preparing for the public release of our Go SDK, heavy refactoring works were undertaken and it was a perfect moment for object’s schema validation rewrite to be snuck under the watchful, Product Management’s eye ;)

Full year has passed, our custom validation mechanism has been battle tested and matured into a fully-fledged, feature rich library. Here’s how Service validation looks like when defined with govy:

The output of the printed error is:

Validation for Service has failed for the following properties:

  - 'apiVersion' with value 'n9/v2alpha':

    - should be equal to 'n9/v1alpha'

  - 'kind' with value 'Project':

    - should be equal to 'Service'

  - 'metadata.name' with value 'slo-status api':

    - string must match regular expression: '^[a-z0-9]([-a-z0-9]*[a-z0-9])?$' (e.g. 'my-name', '123-abc'); an RFC-1123 compliant label name must consist of lower case alphanumeric characters or

 '-', and must start and end with an alphanumeric character

  - 'metadata.project' with value 'default project':

    - string must match regular expression: '^[a-z0-9]([-a-z0-9]*[a-z0-9])?$' (e.g. 'my-name', '123-abc'); an RFC-1123 compliant label name must consist of lower case alphanumeric characters or

 '-', and must start and end with an alphanumeric character

The error message, out of the box looks much better than what we’ve seen coming from the validator. It was designed with end users in mind to be verbose, information rich and human readable.

The returned errors are structured (as in, each is a struct) and allow customization and transformation according to the use case.

Since our objects’ API uses the SDK under the hood, improving errors helped us greatly improve UX and ease the burden on customer support for explaining obscure errors’ causes to our users.

API

When designing the API for govy, we took heavy influence from C# Fluent Validation library. While Go does not have lambda functions and the generics implementation is rather simple compared to C#, we were able to create a similar, functional interface which is 100% type safe.

Thanks to the functional approach, the API is lazy loaded, it benefits greatly from being defined once, only invoking the Validate function on specific instances of validated structs. Furthermore, it’s immutable, which increases modularity. Modifying rules, properties or validators and creating their variants is safe and easy.

The validation rules are aggregated into hierarchical containers, each moving one abstraction level up.

1. We start with govy.Rule, it defines a single validation rule in the form of a function that accepts a value and returns an error. The type parameter of govy.Rule constraints the type of the validated value. Multiple rules can be aggregated into a single rule using govy.RuleSet.
2. Next, we define an object’s property we wish to validate using govy.PropertyRules. In Go, the “object” will most often refer to a struct and property will be a struct field.
   It operates with two type constraints, one is the type of its parent’s value, the second is the type of the property’s value itself. The property rules are instantiated with a getter function, which defines runtime logic for accessing the property given its parent’s context.
   Slices and maps have separate, designated containers: govy.PropertyRulesForSlice and govy.PropertyRulesForMap, each extending the standard govy.PropertyRules.
3. Finally, we can define a govy.Validator which can represent a single struct and all its fields.
   It can be embedded into govy.PropertyRules which allows defining validation flow for nested structs (see Include function in the code snippet above).

Plan

Validation rules can be inspected and not just through debugger!

With govy.Plan called on govy.Validator users can produce a validation plan which lists all properties and their validation rules. This plan can be easily encoded into JSON, this way it can be ingested by other programs in order to create documentation in any format desired, facilitating automatic documentation of users’ validation rules.

Property names

One shortcoming of govy when compared with validator was that it couldn’t initially infer struct fields’ names. Getter function returns a value with no context of where it came from.

Fortunately, we’ve come up with a solution for that which does not compromise the API’s type-safety, it’s currently in the experimental phase and not turned on by default, you can read more about it here.

Found this interesting?

Govy comes with a rich documentation, packed with testable examples, If you want to learn more about it, be sure to check out its repository, the README.md is a great place to start your journey with govy 🙂