Skip to content

DNS logs

Overview

Cloudflare is a global network designed to make everything you connect to the Internet secure, private, fast, and reliable.

In this documentation, you will learn how to collect and send Cloudflare DNS logs to SEKOIA.IO.

Event Categories

The following table lists the data source offered by this integration.

Data Source Description
DNS records Cloudflare DNS logs provides detailed logs on handled DNS queries

In details, the following table denotes the type of events produced by this integration.

Name Values
Kind event
Category network
Type info

Event Samples

Find below few samples of events and how they are normalized by SEKOIA.IO.

{
    "@timestamp": "2022-07-07T22:42:06.004000Z",
    "dns": {
        "question": {
            "name": "foO-BAR-baz.xYz",
            "type": "A"
        },
        "response_code": "NoError"
    },
    "event": {
        "kind": "event",
        "category": [
            "network"
        ],
        "type": [
            "info"
        ],
        "dataset": "dns_logs"
    }
}
{
    "@timestamp": "2022-07-07T22:42:06.004000Z",
    "dns": {
        "question": {
            "name": "foO-BAR-baz.xYz",
            "type": "NS"
        },
        "response_code": "NoError"
    },
    "event": {
        "kind": "event",
        "category": [
            "network"
        ],
        "type": [
            "info"
        ],
        "dataset": "dns_logs"
    }
}
{
    "@timestamp": "2022-07-07T22:42:06.004000Z",
    "dns": {
        "question": {
            "name": "foO-BAR-baz.xYz",
            "type": "AAAA"
        },
        "response_code": "NoError"
    },
    "event": {
        "kind": "event",
        "category": [
            "network"
        ],
        "type": [
            "info"
        ],
        "dataset": "dns_logs"
    }
}
{
    "@timestamp": "2022-07-07T22:42:06.004000Z",
    "dns": {
        "question": {
            "name": "foO-BAR-baz.xYz"
        },
        "response_code": "NoError"
    },
    "event": {
        "kind": "event",
        "category": [
            "network"
        ],
        "type": [
            "info"
        ],
        "dataset": "dns_logs"
    }
}

Extracted Fields

The following table lists the fields that are extracted, normalized under the ECS format, analyzed and indexed by the parser. It should be noted that infered fields are not listed.

Name Type Description
@timestamp date Date/time when the event originated.
cloudflare.ColoCode keyword None
cloudflare.EDNSSubnet keyword None
cloudflare.EDNSSubnetLength number None
cloudflare.ResponseCached bool None
dns.question.name keyword The name being queried.
event.category keyword Event category. The second categorization field in the hierarchy.
event.dataset keyword Name of the dataset.
event.kind keyword The kind of the event. The highest categorization field in the hierarchy.
event.type keyword Event type. The third categorization field in the hierarchy.
source.ip ip IP address of the source.

Configuration

Create the intake on SEKOIA.IO

Go to the intake page and create a new intake from the format Cloudflare.

Configure events forwarding on Cloudflare

Retrieve necessary information

First, you will have to retrieve configuration information. Connect to Cloudflare Console to collect the following :

  1. Cloudflare Email Address
    • This is your Email Address that you can find in My Profile.
  2. Cloudflare API Token
    • Go to My Profile, then on the left panel, click on API Tokens.
    • Click on the Create Token button and select the Read analytics and logs template.
  3. Cloudflare Zone ID :
    • This information is specific to a Website.
    • On the left panel, click on Websites and select the Website you want.
    • On the right panel, there is an API section where you can retrieve the Zone ID.

Configure Logpush

Retrieve all available fields with your subscription

Depending on your subscription, you first need to get all the fields that you can forward to SEKOIA.IO by using the Cloudflare API Retrieve all available fields for a dataset.

With cURL you can use this command:

curl -s -H "X-Auth-Email: <Cloudflare Email Address>" \
-H "X-Auth-Key: <Cloudflare API Token>" \
"https://api.cloudflare.com/client/v4/zones/<Cloudflare Zone ID>/logpush/datasets/dns_logs/fields" \
| jq '.result' # (1)

  1. will return
    {
      "ColoCode": "string; IATA airport code of data center that received the request",
      "EDNSSubnet": "string; EDNS Client Subnet (IPv4 or IPv6). See here: [EDNS Client Subnet](/logs/reference/glossary/#edns-client-subnet-ecs)",
      "EDNSSubnetLength": "int; EDNS Client Subnet length. See here: [EDNS Client Subnet](/logs/reference/glossary/#edns-client-subnet-ecs)",
      "QueryName": "string; name of the query that was sent",
      "QueryType": "int; integer value of query type. See here: [Query type](https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-parameters-4)",
      "ResponseCached": "bool; whether the response was cached or not",
      "ResponseCode": "int; integer value of response code. See here: [Response code](https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#dns-parameters-6)",
      "SourceIP": "string; IP address of the client (IPv4 or IPv6)",
      "Timestamp": "int or string; timestamp at which the query occurred"
    }
    

Save the fields you want to send to SEKOIA.IO. It will be used to configure the Logpush job. It this example :

ColoCode,EDNSSubnet,EDNSSubnetLength,QueryName,QueryType,ResponseCached,ResponseCode,SourceIP,Timestamp

Create a Logpush job

Configure a Logpush job with the following destination:

https://intake.sekoia.io/plain/batch?header_X-SEKOIAIO-INTAKE-KEY=<YOUR_INTAKE_KEY>

To do so, you can manage Logpush with cURL:

$ curl -X POST https://api.cloudflare.com/client/v4/zones/<Cloudflare Zone ID>/logpush/jobs \
-h "X-Auth-Email: <Cloudflare Email Address>" \
-H "X-Auth-Key: <Cloudflare API Token>" \
-H "Content-Type: application/json" \
--data '{
    "dataset": "dns_logs",
    "enabled": true,
    "max_upload_bytes": 5000000,
    "max_upload_records": 1000,
    "logpull_options":"fields=<LIST_OF_FIELDS>&timestamps=rfc3339",
    "destination_conf": "https://intake.sekoia.io/plain/batch?header_X-SEKOIAIO-INTAKE-KEY=<YOUR_INTAKE_KEY>"
}' # (1)
  1. will return
    {
      "errors": [],
      "messages": [],
      "result": {
        "id": 148,
        "dataset": "dns_log",
        "enabled": false,
        "name": "<DOMAIN_NAME>",
        "logpull_options": "fields=<LIST_OF_FIELDS>=rfc3339",
        "destination_conf": "https://intake.sekoia.io/plain/batch?header_X-SEKOIAIO-INTAKE-KEY=<YOUR_INTAKE_KEY>",
        "last_complete": null,
        "last_error": null,
        "error_message": null
      },
      "success": true
    }
    

Important

Replace :

Useful Cloudflare API endpoints

On their documentation, Cloudflare provides a list API endpoints you can use. Here is some useful endpoints :

  • https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs/<JOB_ID> to verify the job you previously created is correct (you need to specify the JOB_ID)
  • https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/datasets/<DATASET>/jobs to get all the jobs for a specific dataset (dns_log, firewalls_events or http_requests in our case)
  • https://api.cloudflare.com/client/v4/zones/<ZONE_ID>/logpush/jobs/<JOB_ID> to update a job if you noticed a mistake after the creation of the job (wrong fields, wrong SEKOIA API Key...)