Metadata-Version: 2.1
Name: secrules-parsing
Version: 0.2.4
Summary: ModSecurity DSL Parser package using textX
Home-page: https://github.com/coreruleset/secrules_parsing
License: Apache-2.0
Keywords: secrule,modsecurity,parser,textX
Author: Felipe Zipitria
Author-email: felipe.zipitria@owasp.org
Requires-Python: >=3.7,<4.0
Classifier: License :: OSI Approved :: Apache Software License
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Requires-Dist: importlib-metadata (>=4.11.3,<5.0.0)
Requires-Dist: textX (>=2.3.0,<3.0.0)
Project-URL: Bug Tracker, https://github.com/coreruleset/secrules_parsing/issues
Project-URL: Documentation, https://github.com/coreruleset/secrules_parsing
Project-URL: Repository, https://github.com/coreruleset/secrules_parsing
Description-Content-Type: text/markdown

# OWASP CRS Rules parser

Incomplete parser model and sample application for parsing [Core Rule Set](https://github.com/coreruleset/coreruleset/) written in the ModSecurity DSL SecRule language. It uses the python library [textX](http://www.igordejanovic.net/textX/) for parsing.

## How to use it (CLI):

1. Install dependencies
    Dependencies can be installed system-wide, or just for your user (using `--user`).

    System-wide:
    ```shell
    sudo pip install secrules-parsing
    ```
    User:
    ```shell
    pip install --user secrules-parsing
    ```
2. Execute `secrules-parser` specifying the location of the files you want to scan using the -f/--files argument. This takes wildcards or individual files.
   `$ secrules-parser -c -f /owasp-crs/rules/*.conf`

3. Add flags to accomplish needed tasks:

 - -h, --help:
    * *Description:* show the help message and exit
    * *Example:* `$ secrules-parser -h`

 - -r, --regex:
    * *Description:* Extract regular expressions from rules file
    * *Example:*
    ```shell
    $ secrules-parser --regex /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
    {"/owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf": [{"920100": ["^(?i:(?:[a-z]{3,10}\\s+(?:\\w{3,7}?://[\\w\\-\\./]*(?::\\d+)?)?/[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?|connect (?:\\d{1,3}\\.){3}\\d{1,3}\\.?(?::\\d+)?|options \\*)\\s+[\\w\\./]+|get /[^?#]*(?:\\?[^#\\s]*)?(?:#[\\S]*)?)$"]}, {"920120": ["(?<!&(?:[aAoOuUyY]uml)|&(?:[aAeEiIoOuU]circ)|&(?:[eEiIoOuUyY]acute)|&(?:[aAeEiIoOuU]grave)|&(?:[cC]cedil)|&(?:[aAnNoO]tilde)|&(?:amp)|&(?:apos));|['\\\"=]"]}, {"920160": ["^\\d+$"]}, {"920170": ["^(?:GET|HEAD)$"]}, {"920171": ["^(?:GET|HEAD)$"]}, {"920180": ["^POST$"]}, {"920190": ["(\\d+)\\-(\\d+)\\,"]}, {"920210": ["\\b(?:keep-alive|close),\\s?(?:keep-alive|close)\\b"]}, {"920220": ["\\%(?:(?!$|\\W)|[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})"]}, {"920240": ["^(?:application\\/x-www-form-urlencoded|text\\/xml)(?:;(?:\\s?charset\\s?=\\s?[\\w\\d\\-]{1,18})?)??$"]}, {"920260": ["\\%u[fF]{2}[0-9a-fA-F]{2}"]}, {"920290": ["^$"]}, {"920310": ["^$"]}, {"920311": ["^$"]}, {"920330": ["^$"]}, {"920340": ["^0$"]}, {"920350": ["^[\\d.:]+$"]}, {"920420": ["^(?:GET|HEAD|PROPFIND|OPTIONS)$"]}, {"920440": ["\\.(.*)$"]}, {"920450": ["^.*$"]}, {"920200": ["^bytes=(?:(?:\\d+)?\\-(?:\\d+)?\\s*,?\\s*){6}"]}, {"920230": ["\\%((?!$|\\W)|[0-9a-fA-F]{2}|u[0-9a-fA-F]{4})"]}, {"920121": ["['\\\";=]"]}, {"920460": ["(?<!\\Q\\\\\\E)\\Q\\\\\\E[cdeghijklmpqwxyz123456789]"]}]}
    ```

 * -c, --correctness:
    * *Description:* Check the validity of the syntax
    * *Example:*
    ```
    $ secrules-parser -c -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
    Syntax OK: ../../../rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
    ```

 * -v, --verbose
    * *Description:* Print verbose messages
    * *Example:*
    ```
    $ secrules-parser -c -v -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
    ...
    ```

 * -o FILE, --output FILE
    * *Description:* Output results to file
    * *Example:*
    ```
    $ secrules-parser -c -o out.json -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf    
    ```

 * --output-type github | plain
    * *Description:* Desired output format. Useful if running from Github Actions and you want annotated output
    * *Example:*
    ```
    $ secrules-parser -c --output-type github -f /owasp-crs/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf
    ```

## How to use it (API):

### process_rules(list files)
Takes a list of file path's and returns models
```python
import glob
import os
from secrules_parsing import parser

# Extract all of our pathing
files = glob.glob("../../rules/*.conf")
# Pass absolute paths because of module location
files = [os.path.abspath(path) for path in files]
models = parser.process_rules(files)
```

### get_correctness(list files, list models)
```python
import glob
import os
from secrules_parsing import parser

# Extract all of our pathing
files = glob.glob("../../rules/*.conf")
# Pass absolute paths because of module location
files = [os.path.abspath(path) for path in files]
models = parser.process_rules(files)
parser.get_correctness(files, models)
```

## Misc

To visualize the syntax tree, use:

```
textx visualize secrules.tx
dot -Tpng -O secrules.tx.dot
```

Then review the generated PNG modsec.tx.dot.png!

Please file an [issue](https://github.com/coreruleset/secrules_parsing/issues) if you find a bug or you want some feature added.

