Firewall events
Overview
Cloudflare is a global network designed to make everything you connect to the Internet secure, private, fast, and reliable.
In this documentation, you will learn how to collect and send Cloudflare Firewall events to SEKOIA.IO.
Event Categories
The following table lists the data source offered by this integration.
Data Source | Description |
---|---|
Web application firewall logs |
Cloudflare protect web application with its web application firewall |
In details, the following table denotes the type of events produced by this integration.
Name | Values |
---|---|
Kind | event |
Category | network |
Type | `` |
Event Samples
Find below few samples of events and how they are normalized by SEKOIA.IO.
{
"@timestamp": "2022-07-12T12:57:06.219000Z",
"event": {
"kind": "event",
"category": [
"network"
],
"type": [
"denied"
],
"action": "block",
"dataset": "firewall-events"
},
"source": {
"ip": "113.206.179.28",
"address": "113.206.179.28"
},
"destination": {
"address": "foo-bar-baz.xyz",
"domain": "foo-bar-baz.xyz",
"registered_domain": "foo-bar-baz.xyz",
"top_level_domain": "xyz"
},
"http": {
"request": {
"method": "GET"
},
"response": {
"status_code": 403
}
},
"url": {
"domain": "foo-bar-baz.xyz",
"path": "/static/favicon.ico",
"registered_domain": "foo-bar-baz.xyz",
"top_level_domain": "xyz"
},
"related": {
"hosts": [
"foo-bar-baz.xyz"
],
"ip": [
"113.206.179.28"
]
},
"cloudflare": {
"RayID": "7299f155dda47d6b",
"EdgeResponseStatus": 403
}
}
Extracted Fields
The following table lists the fields that are extracted, normalized under the ECS format, analyzed and indexed by the parser. It should be noted that infered fields are not listed.
Name | Type | Description |
---|---|---|
@timestamp |
date |
Date/time when the event originated. |
cloudflare.ClientIPClass |
string |
None |
cloudflare.ClientRefererHost |
string |
None |
cloudflare.ClientRefererPath |
string |
None |
cloudflare.ClientRefererQuery |
string |
None |
cloudflare.ClientRefererScheme |
string |
None |
cloudflare.EdgeColoCode |
string |
None |
cloudflare.EdgeResponseStatus |
number |
None |
cloudflare.Kind |
string |
None |
cloudflare.OriginResponseStatus |
number |
None |
cloudflare.OriginatorRayID |
string |
None |
cloudflare.RayID |
string |
None |
cloudflare.clientASNDescription |
keyword |
None |
cloudflare.matchIndex |
number |
None |
cloudflare.metadata |
object |
None |
cloudflare.source |
keyword |
None |
destination.domain |
keyword |
The domain name of the destination. |
event.action |
keyword |
The action captured by the event. |
event.category |
keyword |
Event category. The second categorization field in the hierarchy. |
event.dataset |
keyword |
Name of the dataset. |
event.kind |
keyword |
The kind of the event. The highest categorization field in the hierarchy. |
http.request.bytes |
long |
Total size in bytes of the request (body and headers). |
http.request.method |
keyword |
HTTP request method. |
http.request.referrer |
keyword |
Referrer for this HTTP request. |
http.response.status_code |
long |
HTTP response status code. |
http.version |
keyword |
HTTP version. |
network.protocol |
keyword |
Application protocol name. |
rule.id |
keyword |
Rule ID |
rule.ruleset |
keyword |
Rule ruleset |
source.as.number |
long |
Unique number allocated to the autonomous system. |
source.geo.country_name |
keyword |
Country name. |
source.ip |
ip |
IP address of the source. |
source.port |
long |
Port of the source. |
tls.cipher |
keyword |
String indicating the cipher used during the current connection. |
tls.version_protocol |
keyword |
Normalized lowercase protocol name parsed from original string. |
url.domain |
keyword |
Domain of the url. |
url.path |
wildcard |
Path of the request, such as "/search". |
url.query |
keyword |
Query string of the request. |
url.scheme |
keyword |
Scheme of the url. |
user_agent.original |
keyword |
Unparsed user_agent string. |
Configuration
Create the intake on SEKOIA.IO
Go to the intake page and create a new intake from the format Cloudflare.
Configure events forwarding on Cloudflare
Retrieve necessary information
First, you will have to retrieve configuration information. Connect to Cloudflare Console to collect the following :
- Cloudflare Email Address
- This is your
Email Address
that you can find inMy Profile
.
- This is your
- Cloudflare API Token
- Go to
My Profile
, then on the left panel, click onAPI Tokens
. - Click on the
Create Token
button and select theRead analytics and logs
template.
- Go to
- Cloudflare Zone ID :
- This information is specific to a Website.
- On the left panel, click on
Websites
and select the Website you want. - On the right panel, there is an
API
section where you can retrieve theZone ID
.
Configure Logpush
Retrieve all available fields with your subscription
Depending on your subscription, you first need to get all the fields that you can forward to SEKOIA.IO by using the Cloudflare API Retrieve all available fields for a dataset.
With cURL you can use this command:
curl -s -H "X-Auth-Email: <Cloudflare Email Address>" \
-H "X-Auth-Key: <Cloudflare API Token>" \
"https://api.cloudflare.com/client/v4/zones/<Cloudflare Zone ID>/logpush/datasets/firewall_events/fields" \
| jq '.result' # (1)
- will return
{ "Action": "string; the code of the first-class action the Cloudflare Firewall took on this request. Possible actions are unknown | allow | block | challenge | jschallenge | log | connectionclose | challengesolved | challengefailed | challengebypassed | jschallengesolved | jschallengefailed | jschallengebypassed | bypass | managedchallenge | managedchallengeskipped | managedchallengenoninteractivesolved | managedchallengeinteractivesolved | managedchallengebypassed", "ClientASN": "int; the ASN number of the visitor", "ClientASNDescription": "string; the ASN of the visitor as string", "ClientCountry": "string; country from which request originated", "ClientIP": "string; the visitor's IP address (IPv4 or IPv6)", "ClientIPClass": "string; the classification of the visitor's IP address, possible values are: unknown | badHost | searchEngine | allowlist | monitoringService | noRecord | scan | tor", "ClientRefererHost": "string; the referer host", "ClientRefererPath": "string; the referer path requested by visitor", "ClientRefererQuery": "string; the referer query-string was requested by the visitor", "ClientRefererScheme": "string; the referer URL scheme requested by the visitor", "ClientRequestHost": "string; the HTTP hostname requested by the visitor", "ClientRequestMethod": "string; the HTTP method used by the visitor", "ClientRequestPath": "string; the path requested by visitor", "ClientRequestProtocol": "string; the version of HTTP protocol requested by the visitor", "ClientRequestQuery": "string; the query-string was requested by the visitor", "ClientRequestScheme": "string; the URL scheme requested by the visitor", "ClientRequestUserAgent": "string; visitor's user-agent string", "Datetime": "int or string; the date and time the event occurred at the edge", "EdgeColoCode": "string; the airport code of the Cloudflare datacenter that served this request", "EdgeResponseStatus": "int; HTTP response status code returned to browser", "Kind": "string; the kind of event, currently only possible values are: firewall", "MatchIndex": "int; rules match index in the chain", "Metadata": "object; additional product-specific information. Metadata is organized in key:value pairs. Key and Value formats can vary by Cloudflare security product and can change over time", "OriginResponseStatus": "int; HTTP origin response status code returned to browser", "OriginatorRayID": "string; the RayID of the request that issued the challenge/jschallenge", "RayID": "string; the RayID of the request", "RuleID": "string; the Cloudflare security product-specific RuleID triggered by this request", "Source": "string; the Cloudflare security product triggered by this request. Possible sources are unknown | asn | country | ip | iprange | securitylevel | zonelockdown | waf | firewallrules | uablock | ratelimit | bic | hot | l7ddos | validation | botfight | apishield | botmanagement | dlp | firewallmanaged | firewallcustom" }
Save the fields you want to send to SEKOIA.IO. It will be used to configure the Logpush job. It this example :
Action,ClientASN,ClientASNDescription,ClientCountry,ClientIP,ClientIPClass,ClientRefererHost,ClientRefererPath,ClientRefererQuery,ClientRefererScheme,ClientRequestHost,ClientRequestMethod,ClientRequestPath,ClientRequestProtocol,ClientRequestQuery,ClientRequestScheme,ClientRequestUserAgent,Datetime,EdgeColoCode,EdgeResponseStatus,Kind,MatchIndex,Metadata,OriginResponseStatus,OriginatorRayID,RayID,RuleID,Source
Create a Logpush job
Configure a Logpush job with the following destination:
https://intake.sekoia.io/plain/batch?header_X-SEKOIAIO-INTAKE-KEY=<YOUR_INTAKE_KEY>
To do so, you can manage Logpush with cURL:
$ curl -X POST https://api.cloudflare.com/client/v4/zones/<Cloudflare Zone ID>/logpush/jobs \
-h "x-auth-email: <Cloudflare Email Address>" \
-H "X-Auth-Key: <Cloudflare API Token>" \
-H "Content-Type: application/json" \
--data '{
"dataset": "firewall_events",
"enabled": true,
"max_upload_bytes": 5000000,
"max_upload_records": 1000,
"logpull_options":"fields=<LIST_OF_FIELDS>×tamps=rfc3339",
"destination_conf": "https://intake.sekoia.io/plain/batch?header_X-SEKOIAIO-INTAKE-KEY=<YOUR_INTAKE_KEY>"
}' # (1)
- will return
{ "errors": [], "messages": [], "result": { "id": 147, "dataset": "firewall_events", "enabled": false, "name": "<DOMAIN_NAME>", "logpull_options": "fields=<LIST_OF_FIELDS>×tamps=rfc3339", "destination_conf": "https://intake.sekoia.io/plain/batch?header_X-SEKOIAIO-INTAKE-KEY=<YOUR_INTAKE_KEY>", "last_complete": null, "last_error": null, "error_message": null }, "success": true }
Important
Replace :
<YOUR_INTAKE_KEY>
with the Intake key you generated in the Create the intake on SEKOIA.IO step.<LIST_OF_FIELDS>
with the fields you identified in the Retrieve all available fields with your subscription section (for instanceAction,ClientASN,ClientASNDescription,ClientCountry,ClientIP,ClientIPClass,ClientRefererHost,ClientRefererPath,ClientRefererQuery,ClientRefererScheme,ClientRequestHost,ClientRequestMethod,ClientRequestPath,ClientRequestProtocol,ClientRequestQuery,ClientRequestScheme,ClientRequestUserAgent,Datetime,EdgeColoCode,EdgeResponseStatus,Kind,MatchIndex,Metadata,OriginResponseStatus,OriginatorRayID,RayID,RuleID,Source
).
Useful Cloudflare API endpoints
On their documentation, Cloudflare provides a list API endpoints you can use. Here is some useful endpoints :
https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs/<JOB_ID>
to verify the job you previously created is correct (you need to specify theJOB_ID
)https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/datasets/<DATASET>/jobs
to get all the jobs for a specific dataset (dns_log
,firewalls_events
orhttp_requests
in our case)https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs/<JOB_ID>
to update a job if you noticed a mistake after the creation of the job (wrong fields, wrong SEKOIA API Key...)