Input Validation
Overview
This page contains recommendations for the implementation of input validation.
General
Perform input validation for data from all potentially untrusted sources, including suppliers, partners, vendors, regulators, internal components or applications.
Perform input validation for all user-controlled data, see Data source for validation.
Perform input validation as early as possible in a data flow, preferably as soon as data is received from an external party before it is processed by an application.
Implement input validation on the server side. Do not rely solely on validation on the client side.
Implement centralized input validation functionality.
Implement the following validation scheme:
Normalize processed data, see Normalization.
Perform a syntactic validation of the processed data, see Syntactic validation.
Perform a semantic validation of the processed data, see Semantic validation.
Implement protection against mass parameter assignment attacks.
Log errors in the input validation, see the Logging and Monitoring page.
Comply with requirements from the Error and Exception Handling page.
Data sources for validation
Validate all user-controlled data that are passed in:
Query parameters or URL/GET parameters.
GET /items?sort_by=created_at HTTP/1.1 Host: website.local Cookie: session_id=cf565bc0aaf6560a56c9e6d8632baa58HTTP headers.
GET /v2/user HTTP/1.1 Host: api.website.local PRIVATE-TOKEN: cf565bc0aaf6560a56c9e6d8632baa58GET /v2/user HTTP/1.1 Host: api.website.local Authorization: Bearer cf565bc0aaf6560a56c9e6d8632baa58Cookies.
GET / HTTP/1.1 Host: website.local Cookie: session_id=cf565bc0aaf6560a56c9e6d8632baa58HTTP body.
POST /v2/user HTTP/1.1 Host: api.website.local Authorization: Bearer cf565bc0aaf6560a56c9e6d8632baa58 Content-Type: application/json { "email": "username@domain.local" }POST /v2/user HTTP/1.1 Host: api.website.local Authorization: Bearer cf565bc0aaf6560a56c9e6d8632baa58 Content-Type: application/x-www-form-urlencoded email=username@domain.localUploaded files, see the File Upload page.
POST /v2/user HTTP/1.1 Host: api.website.local Authorization: Bearer cf565bc0aaf6560a56c9e6d8632baa58 Content-Type: multipart/form-data;boundary="c93793a9523238b854bbf0f336" --c93793a9523238b854bbf0f336 Content-Disposition: form-data; name="logo"; filename="image.png" ‰PNG... --c93793a9523238b854bbf0f336--
Normalization
Ensure all the processed data is encoded in an expected encoding (for instance,
UTF-8) and no invalid characters are present.Use
NFKCcanonical encoding form to treat canonically equivalent symbols.Define a list of allowed Unicode characters for data input and reject input with characters outside the allowed character list. For example, avoid
Cf(Format) Unicode characters, commonly used to bypass validation or sanitization.
Syntactic validation
Use data type validators built into the used web framework.
Validate against JSON Schema and XML Schema (XSD) for input in these formats.
Validate input against expected data type, such as integer, string, date, etc.
Validate input against expected value range for numerical parameters and dates. If the business logic does not define a value range, consider value range imposed by language or database.
Validate input against minimum and/or maximum length for strings.
Semantic validation
Define an allow list and validate all data against this list. Avoid using block list validation.
Define an array of allowed values as a small set of string parameters (e.g. days of a week).
Define a list of allowed characters such as
decimal digitsorletters.You can use regular expressions to define allowed values, see the Regular Expressions page.
Implement file validation according to the File Upload page.
Implement email validation according to the Authentication: Email Address Confirmation page.
Server-side validation implementation
You can use the go-playground/validator package to implement input validation.
package main
import (
"fmt"
"github.com/go-playground/validator/v10"
)
// User contains user information
type User struct {
FirstName string `validate:"required"`
LastName string `validate:"required"`
Age uint8 `validate:"gte=0,lte=130"`
Email string `validate:"required,email"`
FavouriteColor string `validate:"iscolor"` // alias for 'hexcolor|rgb|rgba|hsl|hsla'
Addresses []*Address `validate:"required,dive,required"` // a person can have a home and cottage...
}
// Address houses a users address information
type Address struct {
Street string `validate:"required"`
City string `validate:"required"`
Planet string `validate:"required"`
Phone string `validate:"required"`
}
func validateStruct(validate *validator.Validate) {
address := &Address{
Street: "Eavesdown Docks",
Planet: "Persphone",
Phone: "none",
}
user := &User{
FirstName: "Badger",
LastName: "Smith",
Age: 135,
Email: "Badger.Smith@gmail.com",
FavouriteColor: "#000-",
Addresses: []*Address{address},
}
// returns nil or ValidationErrors ( []FieldError )
err := validate.Struct(user)
if err != nil {
// this check is only needed when your code could produce
// an invalid value for validation such as interface with nil value.
if _, ok := err.(*validator.InvalidValidationError); ok {
fmt.Println(err)
return
}
for _, err := range err.(validator.ValidationErrors) {
fmt.Println(err.Namespace())
fmt.Println(err.Field())
fmt.Println(err.StructNamespace())
fmt.Println(err.StructField())
fmt.Println(err.Tag())
fmt.Println(err.ActualTag())
fmt.Println(err.Kind())
fmt.Println(err.Type())
fmt.Println(err.Value())
fmt.Println(err.Param())
fmt.Println()
}
// from here you can create your own error messages in whatever language you wish
return
}
// handle user
}
func validateVariable(validate *validator.Validate) {
myEmail := "joeybloggs.gmail.com"
errs := validate.Var(myEmail, "required,email")
if errs != nil {
// output: Key: "" Error:Field validation for "" failed on the "email" tag
fmt.Println(errs)
return
}
// email ok, move on
}
func main() {
validate = validator.New()
validateStruct(validate)
validateVariable(validate)
}Normalization implementation
You can use the golang.org/x/text/unicode/norm package to implement normalization.
package normalizaton
import (
"fmt"
"strings"
"unicode"
"unicode/utf8"
"golang.org/x/text/unicode/norm"
)
// allowed_unicode_chars contains a list of allowed Unicode character categories
var allowed_unicode_chars = []*unicode.RangeTable{
unicode.Letter, // L category (Letters)
unicode.Digit, // N category (Digits)
unicode.Punct, // P category (Punctuations)
unicode.Symbol, // S category (Symbols)
unicode.Space, // Z category (Spaces)
}
// hasOnlyAllowedCharacters verifies that the string s has only allowed unicode characters
func hasOnlyAllowedCharacters(s string) bool {
for len(s) > 0 {
r, size := utf8.DecodeLastRuneInString(s)
if !unicode.IsOneOf(allowed_unicode_chars, r) {
return false
}
s = s[:len(s)-size]
}
return true
}
func normalize(s string) (string, error) {
// Verify the string s has valid UTF-8 encoding
if !utf8.ValidString(s) {
return nil, fmt.Errorf("Invalid UTF-8 characters found")
}
// Normalize string to NFKC form
normS := norm.NFKC.String(s)
// Verifies the string s has only allowed characters
if !hasOnlyAllowedCharacters(normS) {
return nil, fmt.Errorf("It contains characters from outside allowed categories list")
}
return normS, nil
}References
Last updated