Modsecurity Core Rule Sets and Custom Rules

ModSecurity is an open-source, cross-platform Apache, IIS, and Nginx Web Application Firewall (WAF) engine developed by Trustwave's SpiderLabs. It features a robust, event-based programming language that provides protection against a range of web application attacks and enables HTTP traffic monitoring, logging, and real-time analysis.

ModSecurity is just the engine itself, it needs rules to be useful. These rules are instructions on what to look for in requests and what to do when a request matches a rule.

Because rules must work with other rules to provide broad security coverage, rules are grouped into groups called "rulesets." In addition, ModSecurity version 3 supports rules that affect the processing of future rules, so rules don't always have to act alone and have to be designed to work together.

Table of Contents

Core rules set

Core Rule Set (CRS) is a set of generic attack detection rules for use with ModSecurity or compatible web application firewalls. The CRS aims to protect web applications from a wide range of attacks, including the OWASP Top Ten, with a minimum of false positives.

Custom Rules

In general, it is recommended to restrict changes to the core ruleset files as much as possible. The more you change the individual pre-existing rules yourself, the less likely it is that you'll want to upgrade to newer versions because you'd have to re-create your customizations. We recommend that you try to limit your changes to the custom rules file(s) that are specific to your site. Here you should add new signatures and also create rules to exclude false positives from the normal Core Rules files.  

Rule IDs

For custom rules, you must create your own rule IDs, which must be unique. The "id:" fields contain the IDs of the rules. For custom rules, you should use the local (internal) usage scope (see below for the reserved ID ranges). Do not use assigned areas.

 

ID'sUsage
1-99.999Reserved for local (internal) use. Use this space at your own discretion, but not for rules distributed to others.
100.000-199.999Oracle Rules
200.000-299.999Rules of Comodo
300.000-399.999Rules from gotroot.com
400.000-419.999Unused (available for reservation)
420.000-429.999Rules by ScallyWhack
430.000-439.999Flameeyes rules
440.000-599.999Unused (available for reservation)
600.000-699.999Akamai Rules
700.000-799.999Rules by Ivan Ristic
900.000-999.999Rules of the OWASP ModSecurity Core Rule Set project
1.000.000-1.009.999Rules of the Redhat Security Team
1.010.000-1.999.999Unused (available for reservation)
2.000.000-2.999.999Rules of Trustwave's SpiderLabs Research Team
3.000.000-3.999.999Akamai Rules
4.000.000-4.099.999Rules by AviNetworks
 4.100.000-4.199.999 Rules by Fastly
4.200.000 and moreUnused (available for reservation)

rule syntax

A ModSecurity rule is also sometimes referred to as a SecRule because each rule definition begins with the word "SecRule" that appears at the beginning of the rule definition.

Following the word "SecRule" are the four usable parts of the rule:

Variables to define which parts of the query should be examined.
Operators determine when a rule match should be triggered.
Transformations determine how to normalize the variable's data.
Actions define what to do when a rule applies.

These are combined into one rule as follows:

				
					SecRule VARIABLEN "OPERATOREN" “TRANSFORMATIONEN,AKTIONEN“

				
			

variables

VariableUsage
ARGSIs a collection, i.e. all arguments including the POST payload.
ARGS_GETContains only query string parameters.
ARGS_POSTContains arguments from the POST body.
FILESContains a collection of original file names. Only available for inspected multipart/form-data requests.
FULL_REQUESTContains the full request: request line, request headers, and request body.
QUERY_STRINGContains the query string portion of a request URI. The value in QUERY_STRING is always provided in the raw state without any URL decoding taking place.
REQUEST_BODYContains the raw body of the request. This variable is only available if the URLENCODED request body processor was used, which is the case by default when the content type application/x-www-form-urlencoded is detected, or if the URLENCODED request body parser was forced to be used.
REQUEST_HEADERSThis variable can be used either as a collection of all headers in a request or to validate selected headers.
REQUEST_METHODThis variable contains the request method used in the transaction.
REQUEST_URIThis variable contains the full request URL including the query string data (e.g. /index.php?p=X).

operators

Here a "regular expression" (regex), a pattern or a keyword is specified that is to be checked in the variable(s). The operators begin with the @ character. The full list of operators can be found here.

Transformations

Transformation functions are used to manipulate input data before it is used for matching (ie operator execution). The input data is never changed when you request the use of a transformation function. ModSecurity makes a copy of the data, transforms it, and then runs the operator on the result.
More information here.

Discounts

SALEUsage
disruptiveUsed to allow ModSecurity to perform an action, eg allow or block
Non disruptiveDo something, but this something has no impact on the flow of rule processing. Setting a variable or changing its value is an example of a non-interrupting action. Non-breaking actions can appear in any rule, including any rule belonging to a chain.
FlowThese actions affect the rule flow (eg skip or skipAfter).
Meta dataMetadata actions are used to provide more information about rules. Examples are id, rev, severity and msg.
DataThese aren't really actions, just containers that hold data used by other actions. For example, the status action contains the status used for the lock (if it takes place).

writing rules

Here's an example of a simple rule that blocks a request if the request path (after normalization to lowercase) is equal to /index.php.

				
					SecRule REQUEST_URI "@streq /index.php” “id:1,phase:1,t:lowercase,deny"
				
			
SALEUsage
SecRuleEach rule definition begins with the word "SecRule" as the beginning of the rule definition.
REQUEST_URIThis variable contains the full request URL including the query string data (e.g. /index.php?p=X). However, it never contains a domain name, even if one was provided on the request line.
"@streq /index.php"Performs a string comparison and returns true if the parameter string is identical to the input string. In our case /index.php.
"id:1,phase:1,t:lowercase,deny"Converts all characters to lowercase, stops rule processing, and intercepts the transaction.

In practice, most regulations are not so simple. Below is an example of a current rule from the CRS.

				
					SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "@rx (?i)union.*?select.*?from" \
    "id:942270,\
    phase:2,\
    block,\
    capture,\
    t:none,t:urlDecodeUni,\
    msg:'Looking for basic sql injection. Common attack string for mysql, oracle and others',\
    logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',\
    tag:'application-multi',\
    tag:'language-multi',\
    tag:'platform-multi',\
    tag:'attack-sqli',\
    tag:'paranoia-level/1',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/152/248/66',\
    tag:'PCI/6.5.2',\
    ver:'OWASP_CRS/3.3.0',\
    severity:'CRITICAL',\
    setvar:'tx.sql_injection_score=+%{tx.critical_anomaly_score}',\
    setvar:’tx.anomaly_score_pl1=+%{tx.critical_anomaly_score}'"
				
			

You might have noticed the "operator" part in the rule above:

				
					@rx (?i)union.*?select.*?from

				
			

This strange-looking string of characters is what is known as a regular expression, also known as a regex. Regex is a powerful pattern-matching language used in the computer and software industries. This particular regex checks if a string contains the word "union" anywhere in it, followed by any other character, then followed by the word "select", followed in turn by any other character

 

characters followed by the word "union". These are SQL keywords that, when appearing in this order, could mean an attacker exploited a SQL injection vulnerability in an application.

The ModSecurity reference manual should be consulted in all cases where questions about the syntax of the commands arise: GitHub

You want to see the consequences of a successful hacker attack
Spare your IT system?
Test your IT now with a professional penetration test!
For the penetration test

rule installation

Create your own rules directory:

				
					mkdir /etc/httpd/modsecurity.custom.d

				
			

Create a configuration file for your own rules in the /etc/httpd/conf.d directory.

For example:

Modsecurity

Navigate to /etc/httpd/conf.d and create the 01_modsecurity.conf file and add this line to the 01_modsecurity.conf file:

				
					Include modsecurity.custom.d/99_PSN_custom.conf
				
			
Modsecurity

Install your own rules in the /etc/httpd/modsecurity.custom.d directory. For example:

Modsecurity

Test your Apache configuration.

				
					service httpd configtest

				
			

If your test was successful, restart apache.

				
					service httpd restart
				
			
Modsecurity

Test your rule. For example:

				
					curl http://localhost/index.php
				
			
Modsecurity
				
					cat logs/access_log
				
			
Modsecurity

Dealing with false positives

It is inevitable that you will encounter some false positives when using web application firewalls. This is not unique to ModSecurity. All web application firewalls generate false positives from time to time. The information below will guide you through the process of identifying, remediating, implementing and testing new custom rules to resolve false positives.

Any ruleset can cause false positives in new environments


False positives in ModSecurity + the Core Rules mainly occur as a by-product, due to the fact that the rules are "generic" in nature. There is no way to know exactly what web application is running behind it. Because of this, the ground rules are geared toward blocking the known bad stuff and enforcing some HTTP compliance. This intercepts the vast majority of attacks.

Use the DetectionOnly mode


On a clean install, the Log only Rule Set version should be used first, or if no such version is available, set ModSecurity to Detection only using the SecRuleEngine DetectionOnly command. After running ModSecurity in detection-only mode for a while, review the events generated and decide if changes should be made to the ruleset before switching to protection mode.

Don't just delete a rule


Just because a particular rule produces a false positive on your site doesn't mean you should remove the rule entirely. Remember, these rules were created for a reason. They are intended to block a known attack. If you remove this rule entirely, you may be exposing your website to the very attack that the rule was created to deal with. This would be the dreaded false negative.

ModSecurity rules are open-source


Luckily, since ModSecurity's rules are open source, you can see exactly what the rule applies to, and you can also create your own rules. With closed-source rules, you can't verify what the rule is looking for, so you really have no choice but to remove the offending rule.

The logs are your friend


To verify that it is indeed a false alarm, you need to check your logs. This means that you must first look in the audit_log file to see what the ModSecurity message says. It provides information about which rule was triggered. The same information can also be found in the error_log file. The last place to look and the best source of information is the modsec_debug.log file. In this file you can read everything that ModSecurity does, especially if you increase the SecDebugLogLevel to 9. Note, however, that increasing the verbosity of the debug log will impact performance. Increasing the verbosity for all traffic is usually not possible. However, you can create a new rule that uses the ctl action to selectively increase the debug log level. If you e.g. For example, to detect a false positive error only for a specific user, you can insert a rule like this:

				
					SecRule REMOTE_ADDR "^192\.168\.10\.100$" phase:1,log,pass,ctl:debugLogLevel=9

				
			

This sets the debugLogLevel to 9 only for requests coming from that specific source IP address. Maybe that still generates a bit too much traffic. You could narrow this down a bit to increase logging only for the specific file or argument causing the false positive:

				
					SecRule REQUEST_URI "^/path/to/script.pl$" phase:1,log,pass,ctl:debugLogLevel=9

				
			

or

				
					SecRule ARGS:variablename "something" phase:1,pass,ctl:debugLogLevel=9
				
			

Now that you have detailed information in the debug log file, you can review it to make sure you understand what part of the request was being examined when the specific rule was triggered. You can review these to ensure you understand which part of the request was inspected when the specific rule was triggered. Likewise, you can also view the payload after all transformation functions have been applied.

Fixed the false positive

OK, so now you have identified the specific core rule that is causing the false positive. positively caused. Let's assume that the rule that causes a false positive is the following in the REQUEST-901-INITIALIZATION.conf (/etc/httpd/modsecurity.d/activated_rules/) file.

				
					# Default HTTP policy: allowed_request_content_type (rule 900220)
SecRule &TX:allowed_request_content_type "@eq 0" \
	"id:901162,\
	phase:1,\
	pass,\
	nolog,\
	ver:'OWASP_CRS/3.3.0',\
	setvar:'tx.allowed_request_content_type=|application/x-www-form-urlencoded| |multipart/form-data| |multipart/related| |text/xml| |application/xml| |application/soap+xml| |application/x-amf| |application/json| |application/cloudevents+json| |application/cloudevents-batch+json| |application/octet-stream| |application/csp-report| |application/xss-auditor-report| |text/plain|'"
				
			

The next step is to copy and paste the rule into the new 901_customrules.conf file. Let's assume that this rule gets a false positive when checking a specific MIME type of a file or, in our example, a digitally encrypted message.

You now need to make some changes to the rule to update it and remove the false match. The section of code colored blue is the corresponding update.

More information about the MIME types of files here.

				
					# Default HTTP policy: allowed_request_content_type (rule 900220)
SecRule &TX:allowed_request_content_type "@eq 0" \
	"id:901162,\
	phase:1,\
	pass,\
	nolog,\
	ver:'OWASP_CRS/3.3.0',\
	setvar:'tx.allowed_request_content_type=|application/x-www-form-urlencoded| |multipart/form-data| |multipart/related| |text/xml| |application/xml| |application/soap+xml| |application/x-amf| |application/json| |application/cloudevents+json| |application/cloudevents-batch+json| |application/octet-stream| |application/csp-report| 	|application/xss-auditor-report| |text/plain| |application/x-pkcs7-mime|'"
				
			
The last thing you need to do is disable the core rule that was causing the problem with SecRuleRemoveById.
				
					SecRuleRemoveById	 901162
				
			
For this we need to edit the /etc/httpd/conf.d/default-site.conf file.
Modsecurity

Testing the new rules


The final step is to test your new configurations and verify that the old rule is not running and the new rule is not throwing a false positive. The simplest method is to resend the previously offending request to the web server and then monitor the audit_log file to see if the request is blocked or the ModSecurity message is generated.

Implementation of new core rules


This kind of methodology allows you to create custom exclusions and fix false positives, and it also allows for easy updating of the Core Rules themselves. What we don't want is that current mod users have changed the Core Rules files that much for their environment , that they don't want to update when new Core Rule versions are available, for fear of having to re-implement all their custom configurations. With this scenario, you can download new Core Rules versions as they are released, then simply copy over your new custom ModSecurity rules files and you're good to go!

DIVIDE
Newsletter Form

Become a Cyber ​​Security Insider

Get early access and exclusive content!


By signing up, you agree to receive occasional marketing emails from us.
Please accept the cookies at the bottom of this page to be able to submit the form!
OTHER CONTRIBUTIONS

Table of Contents

PSN_KU_Cover
NewsLetter Form Pop Up New

Become a Cyber ​​Security Insider

Subscribe to our knowledge base and get:

Early access to new blog posts
Exclusive content
Regular updates on industry trends and best practices


By signing up, you agree to receive occasional marketing emails from us.
Please accept the cookies at the bottom of this page to be able to submit the form!