# Input Validation
## Overview
This page contains recommendations for the implementation of input validation.
{% hint style="info" %}
Do **not** use input validation as the primary method of preventing Cross-Site Scripting, SQL injection and other attacks which are covered by the [Vulnerability Mitigation](https://0xn3va.gitbook.io/application-security-handbook/web-application/vulnerability-mitigation) section. Nevertheless, input validation can make a significant contribution to reducing the impact of these attacks if implemented properly.
{% endhint %}
## General
* Perform input validation for data from all potentially untrusted sources, including suppliers, partners, vendors, regulators, internal components or applications.
* Perform input validation for all user-controlled data, see [Data source for validation](#data-sources-for-validation).
* Perform input validation as early as possible in a data flow, preferably as soon as data is received from an external party before it is processed by an application.
* Implement input validation on the server side. Do not rely solely on validation on the client side.
* Implement centralized input validation functionality.
Clarification
It means that input validation must be implemented as a separate component, package or some kind of middleware. This is necessary to achieve ease of implementation of input validation mechanisms, their improvement and reuse.
* Implement the following validation scheme:
* Normalize processed data, see [Normalization](#normalization).
* Perform a syntactic validation of the processed data, see [Syntactic validation](#syntactic-validation).
* Perform a semantic validation of the processed data, see [Semantic validation](#semantic-validation).
* Implement protection against mass parameter assignment attacks.
* Log errors in the input validation, see the [Logging and Monitoring](https://0xn3va.gitbook.io/application-security-handbook/web-application/logging-and-monitoring) page.
* Comply with requirements from the [Error and Exception Handling](https://0xn3va.gitbook.io/application-security-handbook/web-application/error-and-exception-handling) page.
## Data sources for validation
Clarification
Unicode classifies characters within different categories. Unicode characters have multiple uses, and their category is determined based on the primary characteristic of the character. There are printable characters such as:
* `Lu` uppercase letters.
* `Mn` nonspacing marks, for example, accents and other letter decorations.
* `Nd` decimal numbers.
* `Po` other punctuation, for example, the Full Stop dot `.`.
* `Zs` space separator.
* `Sm` math symbol, for example, `<` or `=`.
and not printable characters such as:
* `CC` control.
* `Cf` format.
The last two categories (not printable characters) are the most used for attacks trying to bypass input validation, and therefore they should be avoided if not needed. For more information on categories, please see .
There are three approaches to handle not-allowed characters:
* **Recommended (rigorous and safe)**: reject the input completely if it has any character outside of allowed character lists.
* **Not recommended (lenient but safe)**: replace not-allowed characters with the Replacement Character `U+FFFD`.
* **Not recommended (unsafe)**: remove not-allowed characters from input. This approach could have unexpected behaviors and be used by attackers to bypass subsequent validation. Therefore, ***this approach should never be used***.
Example of an allowed character list:
* `L` (Letter, contains `Lu` | `Ll` | `Lt` | `Lm` | `Lo` categories)
* `N` (Number, contains `Nd` | `Nl` | `No` categories)
* `P` (Punctuation, contains `Pc` | `Pd` | `Ps` | `Pe` | `Pi` | `Pf` | `Po` categories)
* `S` (Symbol, contains `Sm` | `Sc` | `Sk` | `So` categories)
* `Z` (Separator, contains `Zs` | `Zl` | `Zp` categories)
## Syntactic validation
Clarification
There are two main validation approaches:
* Allow list validation verifies that data complies with a known list of allowed values, anything else is considered as invalid data.
* Block list validation verifies that data does not match any known blocked values. If so, the data is considered invalid, anything else is considered valid data. **Note** that if a block list validation is used, input data **must be normalized** before any comparison, validation or processing. If normalization is not done properly, block list validation can be easily bypassed.
Unfortunately, block list validation may miss unknown bad values that an attacker could leverage to bypass the validation. For example, if an application handles IP addresses, an attacker can bypass a block list that contains `127.0.0.1` using rare IP formats:
```
127.1
0x7f.0x0.0x0.0x1
0x7f001
...
```
* Define an array of allowed values as a small set of string parameters (e.g. days of a week).
* Define a list of allowed characters such as `decimal digits` or `letters`.
* You can use regular expressions to define allowed values, see the [Regular Expressions](https://0xn3va.gitbook.io/application-security-handbook/web-application/regular-expressions) page.
* Implement file validation according to the [File Upload](https://0xn3va.gitbook.io/application-security-handbook/web-application/file-upload) page.
* Implement email validation according to the [Authentication: Email Address Confirmation](https://0xn3va.gitbook.io/application-security-handbook/web-application/authentication/email-address-confirmation) page.
## Server-side validation implementation
{% tabs %}
{% tab title="Go" %}
You can use the [go-playground/validator](https://github.com/go-playground/validator) package to implement input validation.
```go
package main
import (
"fmt"
"github.com/go-playground/validator/v10"
)
// User contains user information
type User struct {
FirstName string `validate:"required"`
LastName string `validate:"required"`
Age uint8 `validate:"gte=0,lte=130"`
Email string `validate:"required,email"`
FavouriteColor string `validate:"iscolor"` // alias for 'hexcolor|rgb|rgba|hsl|hsla'
Addresses []*Address `validate:"required,dive,required"` // a person can have a home and cottage...
}
// Address houses a users address information
type Address struct {
Street string `validate:"required"`
City string `validate:"required"`
Planet string `validate:"required"`
Phone string `validate:"required"`
}
func validateStruct(validate *validator.Validate) {
address := &Address{
Street: "Eavesdown Docks",
Planet: "Persphone",
Phone: "none",
}
user := &User{
FirstName: "Badger",
LastName: "Smith",
Age: 135,
Email: "Badger.Smith@gmail.com",
FavouriteColor: "#000-",
Addresses: []*Address{address},
}
// returns nil or ValidationErrors ( []FieldError )
err := validate.Struct(user)
if err != nil {
// this check is only needed when your code could produce
// an invalid value for validation such as interface with nil value.
if _, ok := err.(*validator.InvalidValidationError); ok {
fmt.Println(err)
return
}
for _, err := range err.(validator.ValidationErrors) {
fmt.Println(err.Namespace())
fmt.Println(err.Field())
fmt.Println(err.StructNamespace())
fmt.Println(err.StructField())
fmt.Println(err.Tag())
fmt.Println(err.ActualTag())
fmt.Println(err.Kind())
fmt.Println(err.Type())
fmt.Println(err.Value())
fmt.Println(err.Param())
fmt.Println()
}
// from here you can create your own error messages in whatever language you wish
return
}
// handle user
}
func validateVariable(validate *validator.Validate) {
myEmail := "joeybloggs.gmail.com"
errs := validate.Var(myEmail, "required,email")
if errs != nil {
// output: Key: "" Error:Field validation for "" failed on the "email" tag
fmt.Println(errs)
return
}
// email ok, move on
}
func main() {
validate = validator.New()
validateStruct(validate)
validateVariable(validate)
}
```
{% endtab %}
{% tab title="Java" %}
Use [Jakarta Bean Validation](https://beanvalidation.org/) to implement input validation.
{% endtab %}
{% tab title="Python" %}
In Django, use [Validators](https://docs.djangoproject.com/en/4.2/ref/validators/) to implement input validation.
{% endtab %}
{% endtabs %}
## Normalization implementation
{% tabs %}
{% tab title="Go" %}
You can use the [golang.org/x/text/unicode/norm](https://pkg.go.dev/golang.org/x/text/unicode/norm) package to implement normalization.
```go
package normalizaton
import (
"fmt"
"strings"
"unicode"
"unicode/utf8"
"golang.org/x/text/unicode/norm"
)
// allowed_unicode_chars contains a list of allowed Unicode character categories
var allowed_unicode_chars = []*unicode.RangeTable{
unicode.Letter, // L category (Letters)
unicode.Digit, // N category (Digits)
unicode.Punct, // P category (Punctuations)
unicode.Symbol, // S category (Symbols)
unicode.Space, // Z category (Spaces)
}
// hasOnlyAllowedCharacters verifies that the string s has only allowed unicode characters
func hasOnlyAllowedCharacters(s string) bool {
for len(s) > 0 {
r, size := utf8.DecodeLastRuneInString(s)
if !unicode.IsOneOf(allowed_unicode_chars, r) {
return false
}
s = s[:len(s)-size]
}
return true
}
func normalize(s string) (string, error) {
// Verify the string s has valid UTF-8 encoding
if !utf8.ValidString(s) {
return nil, fmt.Errorf("Invalid UTF-8 characters found")
}
// Normalize string to NFKC form
normS := norm.NFKC.String(s)
// Verifies the string s has only allowed characters
if !hasOnlyAllowedCharacters(normS) {
return nil, fmt.Errorf("It contains characters from outside allowed categories list")
}
return normS, nil
}
```
{% endtab %}
{% tab title="Java" %}
You can use the [Normalizer](https://docs.oracle.com/javase/8/docs/api/java/text/Normalizer.html) class to implement normalization.
```java
import java.text.Normalizer;
import java.text.Normalizer.Form;
import java.io.UnsupportedEncodingException;
public static String normalize(String str) {
if (!validUTF8(str)) {
throw new NormalizationException("Invalid UTF-8 characters found");
}
String normalizedStr = Normalizer.normalize(str, Normalizer.Form.NFKC);
if (!hasOnlyAllowedCharacters(normalizedStr)) {
throw new NormalizationException("It contains characters from outside allowed categories list");
}
return normalizedStr;
}
private static boolean validUTF8(String str) {
try {
str.getBytes("UTF-8");
return true;
} catch (UnsupportedEncodingException e) {
return false;
}
}
```
{% endtab %}
{% tab title="Python" %}
You can use the [unicodedata](https://docs.python.org/3/library/unicodedata.html) package to implement normalization.
```python
import unicodedata
def normalize(s: str) -> str:
if not valid_utf8(s):
raise NormalizationError('Invalid UTF-8 characters found')
normalized_s: str = unicodedata.normalize('NFKC', s)
if not has_only_allowed_characters(normalized_s):
raise NormalizationError('It contains characters from outside allowed categories list')
return normalized_s
def valid_utf8(s: str) -> bool:
try:
s.decode('utf-8')
return True
except UnicodeError:
return False
```
{% endtab %}
{% endtabs %}
## References
* [OWASP Cheat Sheet Series: Input Validation Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Input_Validation_Cheat_Sheet.html)