Tips from a Pentester on Data Validation

Input validation must always take place on the server. Everything done on the client side is essentially bypassable. For example, if JavaScript is used for input validation, a simple script blocker is sufficient.

Attackers can also intercept and modify generated HTTP requests by using a web interception proxy. This can also be used to bypass client-side validation.

In principle, there is nothing to be said against additionally performing data validation on the client side. This prevents the server from rejecting many incorrectly formatted requests and the user from having to re-enter all the data. However, this is a usability feature rather than a security feature.

So, in overview:
Client side = Usability
Server side = Security

Keep this principle in your mind at all times. Even if the data appears to be trustworthy, do not expect it to be properly formatted.

All input data, whether from an interface, configuration files, or the database, must be validated.

Even if the data is generated by a self-written program, it is still recommended that it be validated.

If one of the validation steps fails, the entire content should be rejected. We do not recommend attempting to convert false input to real input because the code that does so can be abused by attackers.

A common mistake is to use blacklists for data validation. A blacklist or block list contains all characters (and strings) that are not allowed.

The problem with this is that a blacklist is almost never complete. Attackers often find ways to circumvent a blacklist, for example by encoding forbidden characters.

A whitelist, or allow list, contains all values that are allowed. All other values are rejected by default. Such a system can hardly be circumvented.

For structured data, there is the possibility to create a whitelist regular expression that specifies which characters may be used and what length the input must have.

Validation of data can be accomplished in two ways.

The purpose of syntactic data validation is to ensure that data has the correct syntax, or structure. A social security number, for example, is always a ten-digit number. Date fields and e-mail addresses always have a consistent structure that can be verified. Regular expressions, for example, are useful for this.

Semantic data validation involves assessing data values in their context. For example, if an input field is a date of birth, it cannot be in the future.

To ensure the highest level of security, the syntax, and semantics must be validated.

The output encoding is an important player in data validation. Essentially, not every “bad” character can be strictly forbidden as input.

If you want to avoid XSS but still allow symbols like, output encoding is required.

The context always dictates how a character should be encoded. Angle brackets, for example, can be encoded in an HTML context using HTML entities (& lt ; gt ;). Characters that are not permitted in URLs must be encoded in the HTTP context. Using URL encoding, the & symbol, for example, can be represented as%26.

Other common encodings include ASCII and UNICODE for text and Base64 for binary data.