Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

request help: How to import rule set files in Coraza waf #2044

Closed
DokiDoki1103 opened this issue Nov 15, 2023 · 29 comments
Closed

request help: How to import rule set files in Coraza waf #2044

DokiDoki1103 opened this issue Nov 15, 2023 · 29 comments
Assignees
@kayx23

Comments

@DokiDoki1103
Copy link

DokiDoki1103 commented Nov 15, 2023 •
edited
Loading

Issue description

see: #2043

I know the configuration method for a ruleset, but if multiple rulesets are referenced? Or is it possible to import files to make the firewall plugin effective? Configuration by configuration is too cumbersome.

This is the ruleset https://github.com/coreruleset/coreruleset/tree/v4.0/dev/rules
@kayx23
Copy link
Member

kayx23 commented Nov 15, 2023

Try Include @owasp_crs/*.conf

@DokiDoki1103
Copy link
Author

Try Include @owasp_crs/*.conf

Hello sister, can you provide a complete demo? Also, which folder should these confs be placed in?

@kayx23
Copy link
Member

kayx23 commented Nov 16, 2023

APISIX uses coraza-proxy-wasm, in which I found:

image

I have not tested this myself but I imagine updating configs here would serve the purpose:

image

Let me know if this works.

@kayx23
Copy link
Member

kayx23 commented Nov 16, 2023

cc: @Sn0rt @Huanli-Meng

@DokiDoki1103
Copy link
Author

DokiDoki1103 commented Nov 16, 2023 •
edited
Loading

Elder sister means that I need to import the firewall rules into a folder in apixsix, and then configure them directly like this?

curl -i http://127.0.0.1:9180/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -d '{
  "uri": "/anything",
  "plugins": {
    "coraza-filter": {
      "conf": {
        "directives_map": {
          "default": [
            "SecDebugLogLevel 9",
            "SecRuleEngine On",
            "SecRule Include @owasp_crs/xxxx/xx.conf"
          ]
        },
        "default_directives": "default"
      }
    }
  },
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "httpbin.org:80": 1
    }
  }
}'

/xxxx/xx.conf is my path

@kayx23
Copy link
Member

kayx23 commented Nov 16, 2023 •
edited
Loading

The CRS should come embedded with the wasm module and you should not need to worry about the path.

The path is internal. owasp_crs gets mapped to crs so owasp_crs/*.conf correspond to these files that end with .conf: https://github.com/corazawaf/coraza-proxy-wasm/tree/main/wasmplugin/rules/crs

So I'd suggest you try this:

  "plugins": {
    "coraza-filter": {
      "conf": {
        "directives_map": {
          "default": [
            "SecDebugLogLevel 9",
            "SecRuleEngine On",
            "Include @crs-setup-demo-conf",
            "Include @owasp_crs/*.conf"
          ]
        },
        "default_directives": "default"
      }
    }
  }

@DokiDoki1103
Copy link
Author

Dear sister, I still need your help

I configured yaml to enable firewall plugin support for a certain route, but it reported an error

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
  name: httpbin-route
spec:
  http:
  - name: route-ip
    plugins:
    - name: coraza-filter
      enable: true
      config:
        default_directives: "default"
        directives_map:
          defalut: 
          - "SecDebugLogLevel 9"
          - "SecRuleEngine On"
          - "Include @crs-setup-demo-conf"
          - "Include @owasp_crs/*.conf"
    match:
      hosts:
      - test.lyck6.cn
      paths:
      - "/ip"
    upstreams:
    - name: httpbin-upstream

The error is because the plugin was not found

Events:
  Type     Reason               Age                   From           Message
  ----     ------               ----                  ----           -------
  Warning  ResourceSyncAborted  25s (x51 over 5m30s)  ApisixIngress  ApisixIngress synced failed, with error: 1 error occurred:
           * unknown plugin [coraza-filter]

Then I will configure configMap to support plugins, which is the document I referenced

https://apisix.apache.org/blog/2023/09/08/APISIX-integrates-with-Coraza/

图片

There is a path that I don't know what I should fill in

/Home/ubuntu/coraza proxy wasm/build/main.wasm

The path I write by default does not work, do I still have any configuration errors?

@DokiDoki1103
Copy link
Author

I have read this document, but I still don't quite understand. What should I write in this path, or where should I find the file?

https://apisix.apache.org/zh/docs/apisix/wasm/

@DokiDoki1103
Copy link
Author

But I don't have this file /Home/ubuntu/coraza proxy wasm/build/main.wasm

@kayx23
Copy link
Member

kayx23 commented Nov 16, 2023 •
edited
Loading

Assuming the file path is correct. Have you reload APISIX after you update the config.yaml? Asking because the error message says unknown plugin [coraza-filter]

@DokiDoki1103
Copy link
Author

Should this be an external file? I don't have this document

@kayx23
Copy link
Member

kayx23 commented Nov 16, 2023

Yes the wasm module needs to be downloaded first. Here's a doc based off the blog but offers more clarity in steps. It is based off an APISIX instance in Docker so just modify accordingly for your need.

@DokiDoki1103
Copy link
Author

Yes the wasm module needs to be downloaded first. Here's a doc based off the blog but offers more clarity in steps. It is based off an APISIX instance in Docker so just modify accordingly for your need.

I'll take a look at the documents first. Sister, please rest early. Thank you, sister

@DokiDoki1103
Copy link
Author

Sister, I'm sorry to bother you at night。

图片

I followed the tutorial and moved the wasm file to my container. I also modified the configmap configuration file and restarted Apisix. However, when I configured it, I still prompted that the plugin was not found

图片

The following is the version I am using

apache/apisix-ingress-controller:1.7.1
apache/apisix:3.6.0-centos

I came across this document. https://apisix.apache.org/zh/docs/apisix/wasm/

He said I need to build an apisix base environment, and I looked at this document again https://github.com/api7/apisix-build-tools

But when I build it myself, it will fail, so I used the Github action to build the already built image。

I replaced the image apache/apisix:3.6.0-centos -> api7/apisix-base:1.21.4.2.2

Then the container cannot start, I don't know if I want to directly replace the image .

@DokiDoki1103
Copy link
Author

我通过官网的最简单示例，成功配置单条规则，并且生效，但是我配置 Inlcude 规则，就会报错 500

图片

我并没有将 这些 conf 拷贝到容器里面，因为我不知道拷贝到哪个容器里面

这是日志
图片

@kayx23
Copy link
Member

kayx23 commented Nov 16, 2023 •
edited
Loading

If you used the image we provide (e.g. apache/apisix:3.6.0-centos), you do not need to build APISIX Base as the APISIX provided is already on APISIX Base. You may safely ignore that note.

As for the issue with importing core rule sets, I'll take a look and get back to you.

@kayx23
Copy link
Member

kayx23 commented Nov 17, 2023

The CRS should come embedded with the wasm module and you should not need to worry about the path.

The path is internal. owasp_crs gets mapped to crs so owasp_crs/*.conf correspond to these files that end with .conf: https://github.com/corazawaf/coraza-proxy-wasm/tree/main/wasmplugin/rules/crs

So I'd suggest you try this:

  "plugins": {
    "coraza-filter": {
      "conf": {
        "directives_map": {
          "default": [
            "SecDebugLogLevel 9",
            "SecRuleEngine On",
            "Include @crs-setup-demo-conf",
            "Include @owasp_crs/*.conf"
          ]
        },
        "default_directives": "default"
      }
    }
  }

@DokiDoki1103 I can reproduce the error you see. You need to add "Include @crs-setup-demo-conf", as well, exactly as shown in my earlier comment. After that everything should work (I tested).

@kayx23 kayx23 self-assigned this Nov 17, 2023
@kayx23
Copy link
Member

kayx23 commented Nov 17, 2023

Just a caution note: once this PR (corazawaf/coraza-proxy-wasm#243) is merged and if you use the new release in the future, you would update "Include @crs-setup-demo-conf" with Include @crs-setup-conf.

@DokiDoki1103
Copy link
Author

DokiDoki1103 commented Nov 17, 2023 •
edited
Loading

I tested in apisix quickstart and successfully configured it

{
    "id": "getting-started-waf",
    "uri": "/anything/*",
    "plugins": {
        "coraza-filter": {
            "conf": {
                "directives_map": {
                    "default": [
                        "SecDebugLogLevel 9",
                        "SecRuleEngine On",
                        "Include @crs-setup-demo-conf",
                        "Include @owasp_crs/*.conf"
                    ]
                },
                "default_directives": "default"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}

My configuration has successfully worked

@DokiDoki1103
Copy link
Author

DokiDoki1103 commented Nov 17, 2023 •
edited
Loading

Then I tested a single rule and it didn't work

https://github.com/corazawaf/coraza-proxy-wasm/blob/main/wasmplugin/rules/crs/REQUEST-913-SCANNER-DETECTION.conf

{
   "id": "getting-started-waf",
   "uri": "/anything/*",
   "plugins": {
       "coraza-filter": {
           "conf": {
               "directives_map": {
                   "default": [
                       "SecDebugLogLevel 9",
                       "SecRuleEngine On",
                       "Include @crs-setup-demo-conf",
                       "Include @owasp_crs/REQUEST-913-SCANNER-DETECTION.conf"
                   ]
               },
               "default_directives": "default"
           }
       }
   },
   "upstream": {
       "type": "roundrobin",
       "nodes": {
           "httpbin.org:80": 1
       }
   }
}

curl -H "User-Agent: urlgrabber/1.0" http://120.26.1.38:9080/anything/1

It did not intercept malicious requests and return 200

this is log

124.42.51.91 - - [17/Nov/2023:01:42:39 +0000] lyck6.cn "GET /anything/1 HTTP/1.1" 200 435 0.423 "-" "urlgrabber/1.0" 35.153.249.234:80 200 0.415 "http://lyck6.cn"
2023/11/17 01:42:46 [error] 239#239: *3592185 Invalid value tx_id="ScUeUyFLLIeDmStPISf" var_value="+tx.critical_anomaly_score" rule_id=0 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "GET /anything/1 HTTP/1.1", host: "lyck6.cn"
2023/11/17 01:42:46 [emerg] 239#239: *3592185 [client ""] Coraza: Warning. Found User-Agent associated with security scanner [file "@owasp_crs/REQUEST-913-SCANNER-DETECTION.conf"] [line "786"] [id "913100"] [rev ""] [msg "Found User-Agent associated with security scanner"] [data "Matched Data: grabber found within MATCHED_VARS:REQUEST_HEADERS:user-agent: urlgrabber/1.0"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-reputation-scanner"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/118/224/541/310"] [tag "PCI/6.5.10"] [hostname ""] [uri "/anything/1"] [unique_id "ScUeUyFLLIeDmStPISf"], client: 124.42.51.91, server: _, request: "GET /anything/1 HTTP/1.1", host: "lyck6.cn"
124.42.51.91 - - [17/Nov/2023:01:42:46 +0000] lyck6.cn "GET /anything/1 HTTP/1.1" 200 435 0.740 "-" "urlgrabber/1.0" 54.161.176.214:80 200 0.741 "http://lyck6.cn"
图片

@kayx23
Copy link
Member

kayx23 commented Nov 17, 2023

See this https://groups.google.com/a/owasp.org/g/modsecurity-core-rule-set-project/c/rFeNiHGAFdg

@DokiDoki1103
Copy link
Author

DokiDoki1103 commented Nov 17, 2023 •
edited
Loading

I set it but it doesn't take effect ： "SecDefaultAction phase:1,log,auditlog,deny,status:403",

{
    "id": "getting-started-waf",
    "uri": "/anything/*",
    "plugins": {
        "coraza-filter": {
            "conf": {
                "directives_map": {
                    "default": [
                        "SecDebugLogLevel 9",
                        "SecRuleEngine On",
                        "SecDefaultAction phase:1,log,auditlog,deny,status:403",
                        "SecDefaultAction phase:2,log,auditlog,deny,status:403",
                        "Include @crs-setup-demo-conf",
                        "Include @owasp_crs/REQUEST-913-SCANNER-DETECTION.conf",
                        "Include @owasp_crs/REQUEST-921-PROTOCOL-ATTACK.conf"
                    ]
                },
                "default_directives": "default"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}

phase: phase:1,log,auditlog,pass
2023/11/17 03:31:22 [error] 237#237: failed to configure plugin context 14, rc: 0
2023/11/17 03:31:22 [error] 237#237: *4015032 [lua] wasm.lua:115: phase_func(): coraza-filter: failed to fetch wasm plugin ctx: failed to run proxy_on_configure, client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
124.42.51.91 - - [17/Nov/2023:03:31:21 +0000] 120.26.1.38:9180 "PUT /apisix/admin/routes/ HTTP/1.1" 200 696 0.004 "-" "PostmanRuntime/7.34.0" - - - "http://120.26.1.38:9180"
124.42.51.91 - - [17/Nov/2023:03:31:22 +0000] lyck6.cn "POST /anything/unix HTTP/1.1" 503 269 0.000 "-" "urlgrabber/1.0" - - - "http://lyck6.cn"
2023/11/17 03:31:29 [emerg] 237#237: Failed to parse directives: invalid WAF config from string: failed to parse string: failed to compile the directive "secaction": SecDefaultAction already defined for this phase: phase:1,log,auditlog,pass
2023/11/17 03:31:29 [error] 237#237: failed to configure plugin context 14, rc: 0
2023/11/17 03:31:29 [error] 237#237: *4015032 [lua] wasm.lua:79: phase_func(): coraza-filter: failed to fetch wasm plugin ctx: failed to run proxy_on_configure, client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:31:29 [warn] 237#237: *4015032 [lua] plugin.lua:1107: run_plugin(): coraza-filter exits with http status code 503, client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:31:29 [emerg] 237#237: Failed to parse directives: invalid WAF config from string: failed to parse string: failed to compile the directive "secaction": SecDefaultAction already defined for this phase: phase:1,log,auditlog,pass
2023/11/17 03:31:29 [error] 237#237: failed to configure plugin context 14, rc: 0
2023/11/17 03:31:29 [error] 237#237: *4015032 [lua] wasm.lua:115: phase_func(): coraza-filter: failed to fetch wasm plugin ctx: failed to run proxy_on_configure, client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
124.42.51.91 - - [17/Nov/2023:03:31:29 +0000] lyck6.cn "POST /anything/unix HTTP/1.1" 503 269 0.000 "-" "PostmanRuntime/7.34.0" - - - "http://lyck6.cn"

@kayx23
Copy link
Member

kayx23 commented Nov 17, 2023

The specific issue isn't with APISIX anymore as the integration has now worked as intended.

For questions about specific rules, please redirect them to coraza-proxy-wasm or the relevant coraza communication channels. Thanks.

@DokiDoki1103
Copy link
Author

I tested that not only one plugin is not working, it seems that all plugins are not working, such as

https://github.com/coreruleset/coreruleset/blob/v3.2/dev/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf

{
   "id": "getting-started-waf",
   "uri": "/anything/*",
   "plugins": {
       "coraza-filter": {
           "conf": {
               "directives_map": {
                   "default": [
                       "SecDebugLogLevel 9",
                       "SecRuleEngine On",
                   
                       "Include @crs-setup-demo-conf",
                       "Include @owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"
                   ]
               },
               "default_directives": "default"
           }
       }
   },
   "upstream": {
       "type": "roundrobin",
       "nodes": {
           "httpbin.org:80": 1
       }
   }
}
图片 
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941160 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941160 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [emerg] 240#240: *4059042 [client ""] Coraza: Warning. NoScript XSS InjectionChecker: HTML Injection [file "@owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "943"] [id "941160"] [rev ""] [msg "NoScript XSS InjectionChecker: HTML Injection"] [data "Matched Data: <body  found within REQUEST_COOKIES_NAMES:<body onload: <body onload"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname ""] [uri "/anything/unix"] [unique_id "hlvdigxudQknOIWCbie"], client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941390 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941390 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [emerg] 240#240: *4059042 [client ""] Coraza: Warning. Javascript method detected [file "@owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "1437"] [id "941390"] [rev ""] [msg "Javascript method detected"] [data "Matched Data: alert( found within REQUEST_COOKIES:<body onload: \"alert(1)\">"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname ""] [uri "/anything/unix"] [unique_id "hlvdigxudQknOIWCbie"], client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941320 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [error] 240#240: *4059042 Invalid value tx_id="hlvdigxudQknOIWCbie" var_value="+tx.critical_anomaly_score" rule_id=941320 error="strconv.Atoi: parsing "tx.critical_anomaly_score": invalid syntax", client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"
2023/11/17 03:41:34 [emerg] 240#240: *4059042 [client ""] Coraza: Warning. Possible XSS Attack Detected - HTML Tag Handler [file "@owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "1668"] [id "941320"] [rev ""] [msg "Possible XSS Attack Detected - HTML Tag Handler"] [data "Matched Data: <body  found within REQUEST_COOKIES_NAMES:<body onload: <body onload"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc1"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "OWASP_CRS"] [tag "capec/1000/152/242/63"] [tag "PCI/6.5.1"] [tag "paranoia-level/2"] [hostname ""] [uri "/anything/unix"] [unique_id "hlvdigxudQknOIWCbie"], client: 124.42.51.91, server: _, request: "POST /anything/unix HTTP/1.1", host: "lyck6.cn"

@DokiDoki1103
Copy link
Author

The problem is that I can refer to * to make all rule sets effective, but referring to a single rule set does not

@kayx23
Copy link
Member

kayx23 commented Nov 17, 2023

The log you provided showed the single rule was effective and Coraza was detecting. It just wasn't blocking. This isn't an APISIX issue per se. Please do more research on your end.

@DokiDoki1103
Copy link
Author

Dear sister, after continuous attempts, I have also successfully configured a single rule。

{
    "id": "getting-started-waf",
    "uri": "/anything/*",
    "plugins": {
        "coraza-filter": {
            "conf": {
                "directives_map": {
                    "default": [
                        "SecDebugLogLevel 9",
                        "SecRuleEngine On",
                        "Include @crs-setup-demo-conf",
                        "Include @owasp_crs/REQUEST-901-INITIALIZATION.conf",
                        "Include @owasp_crs/REQUEST-913-SCANNER-DETECTION.conf",
                        "Include @owasp_crs/REQUEST-949-BLOCKING-EVALUATION.conf"
                    ]
                },
                "default_directives": "default"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "httpbin.org:80": 1
        }
    }
}

Configuring rule 913 in this way will succeed

  • 901 must be added before this

  • 949 must be added after this

That is to say, if one's own rules need to be placed in the middle, they must ensure consistency in order. I tested that if the order is incorrect, it will not take effect

@kayx23
Copy link
Member

kayx23 commented Nov 17, 2023

Ok great, thanks for sharing that info. I've tracked this and at some point we'll have a doc update to provide more examples, especially for including the CRS.

If everything works out well now, we can close the issue?

@DokiDoki1103
Copy link
Author

Yes, this issue can be resolved, but I encountered another issue when using apisix ingress controller. He will prompt me that I haven't found the plugin

#2049

@kayx23 kayx23 closed this as completed Nov 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kayx23 kayx23

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kayx23 @DokiDoki1103