Skip to content

Netskope Log Streaming (Transaction Events)

Overview

Netskope Log Streaming allows you to access all Netskope-generated logs directly within your preferred cloud storage and further SIEM tools without the need for additional infrastructure like VMs, improving scalability, cost efficiency, and real-time data availability.

  • Supported environment: SaaS
  • Detection based on: Telemetry
  • Supported application or feature:
    • Transaction Events

Warning

Important note - This format is currently in beta. We highly value your feedback to improve its performance.

Configure

Deploying the Data Collection Architecture

This section will guide you through creating all the AWS resources needed to collect AWS logs. If you already have existing resources that you want to use, you may do so, but any potential issues or incompatibilities with this tutorial will be your responsibility.

Prerequisites

In order to set up the AWS architecture, you need an administrator access to the Amazon console with the permissions to create and manage S3 buckets, SQS queues, S3 notifications and users.

Ensure that the IAM user dedicated to accessing the S3 bucket and SQS Queue has the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:GetQueueUrl"
            ],
            "Principal": {
                "AWS": "arn:aws:iam::XXXXXXXXXXXX:user/USERNAME_HERE"
            },
            "Effect": "Allow",
            "Resource": "arn:aws:sqs:REGION_HERE:XXXXXXXXXXXX:NAME_HERE"
        },
        {
            "Action": [
                "s3:GetObject"
            ],
            "Principal": {
                "AWS": "arn:aws:iam::XXXXXXXXXXXX:user/USERNAME_HERE"
            },
            "Effect": "Allow",
            "Resource": "arn:aws:s3:REGION_HERE:XXXXXXXXXXXX:NAME_HERE/*"
        },
    ]
}

To get started, click on the button below and fill the form on AWS to set up the required environment for Sekoia Deploy to AWS

You need to fill 4 inputs:

  • Stack name - Name of the stack in CloudFormation (Name of the template)
  • BucketName - Name of the S3 Bucket
  • IAMUserName - Name of the dedicated user to access the S3 and SQS queue
  • SQSName - Name of the SQS queue

Read the different pages and click on Next, then click on Submit.

You can follow the creation in the Events tab (it can take few minutes).

Once finished, it should be displayed on the left CREATE_COMPLETE. Click on the Outputs tab in order to retrieve the information needed for Sekoia playbook.

image

Create a S3 Bucket

Please refer to this guide to create a S3 Bucket.

Create a SQS queue

The collect will rely on S3 Event Notifications (SQS) to get new S3 objects.

  1. Create a queue in the SQS service by following this guide
  2. In the Access Policy step, choose the advanced configuration and adapt this configuration sample with your own SQS Amazon Resource Name (ARN) (the main change is the Service directive allowing S3 bucket access): 
    {
  "Version": "2008-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "__owner_statement",
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "SQS:SendMessage",
      "Resource": "arn:aws:sqs:XXX:XXX"
    }
  ]
}

Important

Keep in mind that you have to create the SQS queue in the same region as the S3 bucket you want to watch.

** Create a S3 Event Notification **

Use the following guide to create S3 Event Notification. Once created:

  1. Select the notification for object creation in the Event type section
  2. As the destination, choose the SQS service
  3. Select the queue you created in the previous section

Configure Netskope Log stream

  1. Navigate to Settings > Tools > Log Streaming to add names and properties for the streams you want to monitor.

    step

  2. Click Create Stream. In Name, enter a human-readable name for the stream

    step

  3. Select Transaction Events dataset

  4. Choose GZIP

  5. For the Amazon S3 destination field, fill in the fields

    step

  6. Activate the stream upon saving

Note

Select Amazon S3 with SQS for optimum performance. Stream activation takes about 60 minutes from creation.

Instruction on Sekoia

Create the intake

Go to the intake page and create a new intake from the format Netskope Transaction Events with AWS S3.

Pull events

Go to the playbook page and create a new playbook with the AWS Fetch new logs on S3 trigger.

Set up the module configuration with the AWS access Key ID, the AWS secret access Key and the AWS region name. Set up the trigger configuration with the AWS SQS queue name and the intake key, from the intake previously created.

Start the playbook and enjoy your events.

Raw Events Samples

In this section, you will find examples of raw logs as generated natively by the source. These examples are provided to help integrators understand the data format before ingestion into Sekoia.io. It is crucial for setting up the correct parsing stages and ensuring that all relevant information is captured.

2025-05-26,13:41:00,64,1977,651,2628,5.6.7.8,5.6.7.8,john.doe@example.com,POST,https,-,aws-sdk-go/1.55.5 (go1.23.8; windows; amd64) amazon-ssm-agent/3.3.2471.0,application/x-amz-json-1.1,200,application/x-amz-json-1.1,example.eu-north-1.amazonaws.com,example.eu-north-1.amazonaws.com,/,443,-,2222222222222222222,Client,Amazon Systems Manager,SE,59.328700,18.071700,Stockholm,Stockholm County,100 04,SE,59.328700,18.071700,Stockholm,Stockholm County,100 04,Windows 11,Native,-,Windows Device,-,1748266860,111111111111111111111111,5.6.7.8,CloudApp,-,IT Service/Application Management,"IT Service/Application Management, Technology, All Categories",http_transaction,-,-,3333333333333333333,3333333333333333333,example.eu-north-1.amazonaws.com,-,14,"14, 564, 10001",-,-,2d908070f157946cc4ea9dca39dbe374,NotAvailable,No,-,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,No,No,NotChecked,NotChecked,NotChecked,No,Allow,Established,None,NotEstablished,5.6.7.8,5.6.7.8,example.eu-north-1.amazonaws.com,Sni,"Technology, All Categories, IT Service/Application Management",Decrypt,-,TLSv1.3,TLS_AES_256_GCM_SHA384,NotChecked,NotChecked,5.6.7.8,FR-PAR1,5.6.7.8,64362,5.6.7.8,443,-,-,5.6.7.8,790,-,-,-,-,-,https://example.eu-north-1.amazonaws.com/,/,HTTP1.1,200,IT Service/Application Management,92,excellent,"Enterprise,Unsanctioned",Amazon,202533540828,-,Untagged,View,-,-,-,-,-,-,-,-,-,-,-,-,2025-05-26 15:41:00,alert,Web Access Allow,5.6.7.8,5.6.7.8,example.eu-north-1.amazonaws.com,HttpHostHeader,-,-,-

2025-05-26,13:41:17,292,731,9567,10298,1.2.3.4,1.2.3.4,john.doe@example.com,GET,https,clientId=1234567890&agents=Edge%2CEdgeConfig%2CEdgeServices%2CEdgeFirstRun%2CEdgeFirstRunConfig&osname=win&client=edge&channel=stable&scpfre=0&osarch=x86_64&osver=10.0.22631&wu=1&devicefamily=desktop&soobedate=1742540504&uma=0&sessionid=1011&mngd=0&installdate=1742540615&edu=0&vm=1&bphint=0&fg=1&lbfgdate=1747398666&lafgdate=0,"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.36 Edg/1.2.3.4",-,200,application/json,config.edge.skype.com,config.edge.skype.com,/config/v1/Edge/136.0.3240.92?clientId=1234567890&agents=Edge%2CEdgeConfig%2CEdgeServices%2CEdgeFirstRun%2CEdgeFirstRunConfig&osname=win&client=edge&channel=stable&scpfre=0&osarch=x86_64&osver=10.0.22631&wu=1&devicefamily=desktop&soobedate=1742540504&uma=0&sessionid=1011&mngd=0&installdate=1742540615&edu=0&vm=1&bphint=0&fg=1&lbfgdate=1747398666&lafgdate=0,443,-,2071157430488732926,Client,-,US,47.682220,-122.123009,Redmond,Washington,N/A,SE,59.328700,18.071700,Stockholm,Stockholm County,100 04,Windows 11,Edge,1.2.3.4,Windows Device,skype,1748266877,0,1.2.3.4,Web,-,Technology,"Technology, All Categories",http_transaction,-,-,3333333333333333333,3333333333333333333,config.edge.skype.com,-,564,"564, 10001",-,-,2d908070f157946cc4ea9dca39dbe374,NotAvailable,No,-,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,No,No,NotChecked,NotChecked,NotChecked,No,Allow,Established,None,NotEstablished,1.2.3.4,1.2.3.4,config.edge.skype.com,Sni,"Technology, All Categories",Decrypt,-,TLSv1.3,TLS_AES_256_GCM_SHA384,NotChecked,NotChecked,1.2.3.4,SE-STO1,1.2.3.4,64362,1.2.3.4,443,-,-,1.2.3.4,795,-,-,-,-,-,https://config.edge.skype.com/config/v1/Edge/136.0.3240.92?clientId=1234567890&agents=Edge%2CEdgeConfig%2CEdgeServices%2CEdgeFirstRun%2CEdgeFirstRunConfig&osname=win&client=edge&channel=stable&scpfre=0&osarch=x86_64&osver=10.0.22631&wu=1&devicefamily=desktop&soobedate=1742540504&uma=0&sessionid=1011&mngd=0&installdate=1742540615&edu=0&vm=1&bphint=0&fg=1&lbfgdate=1747398666&lafgdate=0,/config/v1/Edge/136.0.3240.92,HTTP1.1,200,-,-,-,-,-,-,-,-,Browse,-,-,File,-,-,text/plain,Text,-,8715,-,-,-,2025-05-26 15:41:18,alert,Web Access Allow,1.2.3.4,1.2.3.4,config.edge.skype.com,HttpHostHeader,-,-,-

2025-05-26,13:41:27,7,872,1075,1947,9.10.11.12,9.10.11.12,john.doe@example.com,GET,https,-,"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/9.10.11.12 Safari/537.36 Edg/9.10.11.12",-,304,-,pypi.org,pypi.org,/,443,-,111111111111111111111111,Client,PyNLPl,US,37.764200,-122.399300,San Francisco,California,94107,SE,59.328700,18.071700,Stockholm,Stockholm County,100 04,Windows 11,Edge,9.10.11.12,Windows Device,pypi,1748266887,0,9.10.11.12,CloudApp,-,Development Tools,"Development Tools, All Categories",http_transaction,-,-,2222222222222222222,2222222222222222222,pypi.org,-,29,"29, 10001",-,-,2d908070f157946cc4ea9dca39dbe374,NotAvailable,No,-,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,No,No,NotChecked,NotChecked,NotChecked,No,Allow,Established,None,NotEstablished,9.10.11.12,9.10.11.12,pypi.org,Sni,"All Categories, Development Tools",Decrypt,-,TLSv1.3,TLS_AES_256_GCM_SHA384,NotChecked,NotChecked,9.10.11.12,FR-PAR1,9.10.11.12,64362,9.10.11.12,443,-,-,9.10.11.12,836,-,-,-,-,-,https://pypi.org/,/,HTTP1.1,304,Development Tools,47,poor,"Enterprise,Unsanctioned",-,-,-,-,Browse,-,-,-,-,-,-,-,-,-,-,-,-,2025-05-26 15:41:28,alert,Web Access Allow,9.10.11.12,9.10.11.12,pypi.org,HttpHostHeader,-,-,-

Detection section

The following section provides information for those who wish to learn more about the detection capabilities enabled by collecting this intake. It includes details about the built-in rule catalog, event categories, and ECS fields extracted from raw events. This is essential for users aiming to create custom detection rules, perform hunting activities, or pivot in the events page.

No related built-in rules was found. This message is automatically generated.

Event Categories

The following table lists the data source offered by this integration.

Data Source Description
Web logs Netskope Transaction Events provide granular information about the web sites that users have accessed.

In details, the following table denotes the type of events produced by this integration.

Name Values
Kind ``
Category network
Type info

Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the built-in detection rules and hunting activities in the events page. Understanding these transformations is essential for analysts to create effective detection mechanisms with custom detection rules and to leverage the full potential of the collected data.

{
    "message": "2025-05-26,13:41:00,64,1977,651,2628,5.6.7.8,5.6.7.8,john.doe@example.com,POST,https,-,aws-sdk-go/1.55.5 (go1.23.8; windows; amd64) amazon-ssm-agent/3.3.2471.0,application/x-amz-json-1.1,200,application/x-amz-json-1.1,example.eu-north-1.amazonaws.com,example.eu-north-1.amazonaws.com,/,443,-,2222222222222222222,Client,Amazon Systems Manager,SE,59.328700,18.071700,Stockholm,Stockholm County,100 04,SE,59.328700,18.071700,Stockholm,Stockholm County,100 04,Windows 11,Native,-,Windows Device,-,1748266860,111111111111111111111111,5.6.7.8,CloudApp,-,IT Service/Application Management,\"IT Service/Application Management, Technology, All Categories\",http_transaction,-,-,3333333333333333333,3333333333333333333,example.eu-north-1.amazonaws.com,-,14,\"14, 564, 10001\",-,-,2d908070f157946cc4ea9dca39dbe374,NotAvailable,No,-,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,No,No,NotChecked,NotChecked,NotChecked,No,Allow,Established,None,NotEstablished,5.6.7.8,5.6.7.8,example.eu-north-1.amazonaws.com,Sni,\"Technology, All Categories, IT Service/Application Management\",Decrypt,-,TLSv1.3,TLS_AES_256_GCM_SHA384,NotChecked,NotChecked,5.6.7.8,FR-PAR1,5.6.7.8,64362,5.6.7.8,443,-,-,5.6.7.8,790,-,-,-,-,-,https://example.eu-north-1.amazonaws.com/,/,HTTP1.1,200,IT Service/Application Management,92,excellent,\"Enterprise,Unsanctioned\",Amazon,202533540828,-,Untagged,View,-,-,-,-,-,-,-,-,-,-,-,-,2025-05-26 15:41:00,alert,Web Access Allow,5.6.7.8,5.6.7.8,example.eu-north-1.amazonaws.com,HttpHostHeader,-,-,-",
    "event": {
        "action": "alert",
        "category": [
            "network"
        ],
        "module": "netskope.logstream",
        "type": [
            "info"
        ]
    },
    "@timestamp": "2025-05-26T13:41:00Z",
    "destination": {
        "address": "5.6.7.8",
        "bytes": 1977,
        "geo": {
            "city_name": "Stockholm",
            "country_name": "SE",
            "location": {
                "lat": 59.3287,
                "lon": 18.0717
            },
            "postal_code": "100 04",
            "region_name": "Stockholm County"
        },
        "ip": "5.6.7.8"
    },
    "http": {
        "request": {
            "method": "POST",
            "mime_type": "application/x-amz-json-1.1"
        },
        "response": {
            "status_code": 200
        }
    },
    "netskope": {
        "events": {
            "category": "IT Service/Application Management",
            "category_id": "14"
        },
        "log_stream": {
            "cloud": {
                "app": {
                    "category": "IT Service/Application Management",
                    "name": "Amazon Systems Manager",
                    "suite_name": "Amazon",
                    "tags": "Enterprise,Unsanctioned"
                }
            }
        }
    },
    "network": {
        "application": "Amazon Systems Manager",
        "bytes": 2628
    },
    "observer": {
        "product": "Netskope log stream",
        "vendor": "Netskope"
    },
    "related": {
        "hosts": [
            "example.eu-north-1.amazonaws.com"
        ],
        "ip": [
            "5.6.7.8"
        ],
        "user": [
            "john.doe@example.com"
        ]
    },
    "rule": {
        "ruleset": "Web Access Allow"
    },
    "source": {
        "address": "5.6.7.8",
        "bytes": 651,
        "geo": {
            "city_name": "Stockholm",
            "country_name": "SE",
            "location": {
                "lat": 59.3287,
                "lon": 18.0717
            },
            "postal_code": "100 04",
            "region_name": "Stockholm County"
        },
        "ip": "5.6.7.8",
        "nat": {
            "ip": "5.6.7.8"
        }
    },
    "tls": {
        "cipher": "TLS_AES_256_GCM_SHA384",
        "client": {
            "ja3": "2d908070f157946cc4ea9dca39dbe374",
            "server_name": "example.eu-north-1.amazonaws.com"
        }
    },
    "url": {
        "domain": "example.eu-north-1.amazonaws.com",
        "original": "https://example.eu-north-1.amazonaws.com/",
        "path": "/",
        "port": 443,
        "registered_domain": "amazonaws.com",
        "scheme": "https",
        "subdomain": "example.eu-north-1",
        "top_level_domain": "com"
    },
    "user": {
        "email": "john.doe@example.com",
        "name": "john.doe@example.com"
    },
    "user_agent": {
        "device": {
            "name": "Other"
        },
        "name": "aws-sdk-go",
        "original": "aws-sdk-go/1.55.5 (go1.23.8; windows; amd64) amazon-ssm-agent/3.3.2471.0",
        "os": {
            "name": "Other",
            "type": "Windows Device"
        },
        "version": "1.55.5"
    }
}

{
    "message": "2025-05-26,13:41:17,292,731,9567,10298,1.2.3.4,1.2.3.4,john.doe@example.com,GET,https,clientId=1234567890&agents=Edge%2CEdgeConfig%2CEdgeServices%2CEdgeFirstRun%2CEdgeFirstRunConfig&osname=win&client=edge&channel=stable&scpfre=0&osarch=x86_64&osver=10.0.22631&wu=1&devicefamily=desktop&soobedate=1742540504&uma=0&sessionid=1011&mngd=0&installdate=1742540615&edu=0&vm=1&bphint=0&fg=1&lbfgdate=1747398666&lafgdate=0,\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.36 Edg/1.2.3.4\",-,200,application/json,config.edge.skype.com,config.edge.skype.com,/config/v1/Edge/136.0.3240.92?clientId=1234567890&agents=Edge%2CEdgeConfig%2CEdgeServices%2CEdgeFirstRun%2CEdgeFirstRunConfig&osname=win&client=edge&channel=stable&scpfre=0&osarch=x86_64&osver=10.0.22631&wu=1&devicefamily=desktop&soobedate=1742540504&uma=0&sessionid=1011&mngd=0&installdate=1742540615&edu=0&vm=1&bphint=0&fg=1&lbfgdate=1747398666&lafgdate=0,443,-,2071157430488732926,Client,-,US,47.682220,-122.123009,Redmond,Washington,N/A,SE,59.328700,18.071700,Stockholm,Stockholm County,100 04,Windows 11,Edge,1.2.3.4,Windows Device,skype,1748266877,0,1.2.3.4,Web,-,Technology,\"Technology, All Categories\",http_transaction,-,-,3333333333333333333,3333333333333333333,config.edge.skype.com,-,564,\"564, 10001\",-,-,2d908070f157946cc4ea9dca39dbe374,NotAvailable,No,-,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,No,No,NotChecked,NotChecked,NotChecked,No,Allow,Established,None,NotEstablished,1.2.3.4,1.2.3.4,config.edge.skype.com,Sni,\"Technology, All Categories\",Decrypt,-,TLSv1.3,TLS_AES_256_GCM_SHA384,NotChecked,NotChecked,1.2.3.4,SE-STO1,1.2.3.4,64362,1.2.3.4,443,-,-,1.2.3.4,795,-,-,-,-,-,https://config.edge.skype.com/config/v1/Edge/136.0.3240.92?clientId=1234567890&agents=Edge%2CEdgeConfig%2CEdgeServices%2CEdgeFirstRun%2CEdgeFirstRunConfig&osname=win&client=edge&channel=stable&scpfre=0&osarch=x86_64&osver=10.0.22631&wu=1&devicefamily=desktop&soobedate=1742540504&uma=0&sessionid=1011&mngd=0&installdate=1742540615&edu=0&vm=1&bphint=0&fg=1&lbfgdate=1747398666&lafgdate=0,/config/v1/Edge/136.0.3240.92,HTTP1.1,200,-,-,-,-,-,-,-,-,Browse,-,-,File,-,-,text/plain,Text,-,8715,-,-,-,2025-05-26 15:41:18,alert,Web Access Allow,1.2.3.4,1.2.3.4,config.edge.skype.com,HttpHostHeader,-,-,-",
    "event": {
        "action": "alert",
        "category": [
            "network"
        ],
        "module": "netskope.logstream",
        "type": [
            "info"
        ]
    },
    "@timestamp": "2025-05-26T13:41:17Z",
    "destination": {
        "address": "1.2.3.4",
        "bytes": 731,
        "geo": {
            "city_name": "Redmond",
            "country_name": "US",
            "location": {
                "lat": 47.68222,
                "lon": -122.123009
            },
            "region_name": "Washington"
        },
        "ip": "1.2.3.4"
    },
    "file": {
        "mime_type": "text/plain"
    },
    "http": {
        "request": {
            "method": "GET"
        },
        "response": {
            "status_code": 200
        }
    },
    "netskope": {
        "events": {
            "category": "Technology",
            "category_id": "564"
        }
    },
    "network": {
        "bytes": 10298
    },
    "observer": {
        "product": "Netskope log stream",
        "vendor": "Netskope"
    },
    "related": {
        "hosts": [
            "config.edge.skype.com"
        ],
        "ip": [
            "1.2.3.4"
        ],
        "user": [
            "john.doe@example.com"
        ]
    },
    "rule": {
        "ruleset": "Web Access Allow"
    },
    "source": {
        "address": "1.2.3.4",
        "bytes": 9567,
        "geo": {
            "city_name": "Stockholm",
            "country_name": "SE",
            "location": {
                "lat": 59.3287,
                "lon": 18.0717
            },
            "postal_code": "100 04",
            "region_name": "Stockholm County"
        },
        "ip": "1.2.3.4",
        "nat": {
            "ip": "1.2.3.4"
        }
    },
    "tls": {
        "cipher": "TLS_AES_256_GCM_SHA384",
        "client": {
            "ja3": "2d908070f157946cc4ea9dca39dbe374",
            "server_name": "config.edge.skype.com"
        }
    },
    "url": {
        "domain": "config.edge.skype.com",
        "original": "https://config.edge.skype.com/config/v1/Edge/136.0.3240.92?clientId=1234567890&agents=Edge%2CEdgeConfig%2CEdgeServices%2CEdgeFirstRun%2CEdgeFirstRunConfig&osname=win&client=edge&channel=stable&scpfre=0&osarch=x86_64&osver=10.0.22631&wu=1&devicefamily=desktop&soobedate=1742540504&uma=0&sessionid=1011&mngd=0&installdate=1742540615&edu=0&vm=1&bphint=0&fg=1&lbfgdate=1747398666&lafgdate=0",
        "path": "/config/v1/Edge/136.0.3240.92",
        "port": 443,
        "query": "clientId=1234567890&agents=Edge%2CEdgeConfig%2CEdgeServices%2CEdgeFirstRun%2CEdgeFirstRunConfig&osname=win&client=edge&channel=stable&scpfre=0&osarch=x86_64&osver=10.0.22631&wu=1&devicefamily=desktop&soobedate=1742540504&uma=0&sessionid=1011&mngd=0&installdate=1742540615&edu=0&vm=1&bphint=0&fg=1&lbfgdate=1747398666&lafgdate=0",
        "registered_domain": "skype.com",
        "scheme": "https",
        "subdomain": "config.edge",
        "top_level_domain": "com"
    },
    "user": {
        "email": "john.doe@example.com",
        "name": "john.doe@example.com"
    },
    "user_agent": {
        "device": {
            "name": "Other"
        },
        "name": "Edge",
        "original": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/1.2.3.4 Safari/537.36 Edg/1.2.3.4",
        "os": {
            "name": "Windows",
            "type": "Windows Device",
            "version": "10"
        },
        "version": "1.2.3"
    }
}

{
    "message": "2025-05-26,13:41:27,7,872,1075,1947,9.10.11.12,9.10.11.12,john.doe@example.com,GET,https,-,\"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/9.10.11.12 Safari/537.36 Edg/9.10.11.12\",-,304,-,pypi.org,pypi.org,/,443,-,111111111111111111111111,Client,PyNLPl,US,37.764200,-122.399300,San Francisco,California,94107,SE,59.328700,18.071700,Stockholm,Stockholm County,100 04,Windows 11,Edge,9.10.11.12,Windows Device,pypi,1748266887,0,9.10.11.12,CloudApp,-,Development Tools,\"Development Tools, All Categories\",http_transaction,-,-,2222222222222222222,2222222222222222222,pypi.org,-,29,\"29, 10001\",-,-,2d908070f157946cc4ea9dca39dbe374,NotAvailable,No,-,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,NotChecked,No,No,NotChecked,NotChecked,NotChecked,No,Allow,Established,None,NotEstablished,9.10.11.12,9.10.11.12,pypi.org,Sni,\"All Categories, Development Tools\",Decrypt,-,TLSv1.3,TLS_AES_256_GCM_SHA384,NotChecked,NotChecked,9.10.11.12,FR-PAR1,9.10.11.12,64362,9.10.11.12,443,-,-,9.10.11.12,836,-,-,-,-,-,https://pypi.org/,/,HTTP1.1,304,Development Tools,47,poor,\"Enterprise,Unsanctioned\",-,-,-,-,Browse,-,-,-,-,-,-,-,-,-,-,-,-,2025-05-26 15:41:28,alert,Web Access Allow,9.10.11.12,9.10.11.12,pypi.org,HttpHostHeader,-,-,-",
    "event": {
        "action": "alert",
        "category": [
            "network"
        ],
        "module": "netskope.logstream",
        "type": [
            "info"
        ]
    },
    "@timestamp": "2025-05-26T13:41:27Z",
    "destination": {
        "address": "9.10.11.12",
        "bytes": 872,
        "geo": {
            "city_name": "San Francisco",
            "country_name": "US",
            "location": {
                "lat": 37.7642,
                "lon": -122.3993
            },
            "postal_code": "94107",
            "region_name": "California"
        },
        "ip": "9.10.11.12"
    },
    "http": {
        "request": {
            "method": "GET"
        },
        "response": {
            "status_code": 304
        }
    },
    "netskope": {
        "events": {
            "category": "Development Tools",
            "category_id": "29"
        },
        "log_stream": {
            "cloud": {
                "app": {
                    "category": "Development Tools",
                    "name": "PyNLPl",
                    "tags": "Enterprise,Unsanctioned"
                }
            }
        }
    },
    "network": {
        "application": "PyNLPl",
        "bytes": 1947
    },
    "observer": {
        "product": "Netskope log stream",
        "vendor": "Netskope"
    },
    "related": {
        "hosts": [
            "pypi.org"
        ],
        "ip": [
            "9.10.11.12"
        ],
        "user": [
            "john.doe@example.com"
        ]
    },
    "rule": {
        "ruleset": "Web Access Allow"
    },
    "source": {
        "address": "9.10.11.12",
        "bytes": 1075,
        "geo": {
            "city_name": "Stockholm",
            "country_name": "SE",
            "location": {
                "lat": 59.3287,
                "lon": 18.0717
            },
            "postal_code": "100 04",
            "region_name": "Stockholm County"
        },
        "ip": "9.10.11.12",
        "nat": {
            "ip": "9.10.11.12"
        }
    },
    "tls": {
        "cipher": "TLS_AES_256_GCM_SHA384",
        "client": {
            "ja3": "2d908070f157946cc4ea9dca39dbe374",
            "server_name": "pypi.org"
        }
    },
    "url": {
        "domain": "pypi.org",
        "original": "https://pypi.org/",
        "path": "/",
        "port": 443,
        "registered_domain": "pypi.org",
        "scheme": "https",
        "top_level_domain": "org"
    },
    "user": {
        "email": "john.doe@example.com",
        "name": "john.doe@example.com"
    },
    "user_agent": {
        "device": {
            "name": "Other"
        },
        "name": "Edge",
        "original": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/9.10.11.12 Safari/537.36 Edg/9.10.11.12",
        "os": {
            "name": "Windows",
            "type": "Windows Device",
            "version": "10"
        },
        "version": "9.10.11"
    }
}

Extracted Fields

The following table lists the fields that are extracted, normalized under the ECS format, analyzed and indexed by the parser. It should be noted that infered fields are not listed.

Name Type Description
@timestamp date Date/time when the event originated.
destination.bytes long Bytes sent from the destination to the source.
destination.geo.city_name keyword City name.
destination.geo.country_name keyword Country name.
destination.geo.postal_code keyword Postal code.
destination.geo.region_name keyword Region name.
destination.ip ip IP address of the destination.
event.action keyword The action captured by the event.
event.category keyword Event category. The second categorization field in the hierarchy.
event.module keyword Name of the module this data is coming from.
event.reason keyword Reason why this event happened, according to the source
event.type keyword Event type. The third categorization field in the hierarchy.
file.hash.md5 keyword MD5 hash.
file.mime_type keyword Media type of file, document, or arrangement of bytes.
http.request.method keyword HTTP request method.
http.request.mime_type keyword Mime type of the body of the request.
http.request.referrer keyword Referrer for this HTTP request.
http.response.status_code long HTTP response status code.
netskope.events.category keyword Primary category name applicable for the url in this transaction
netskope.events.category_id keyword Primary category ID applicable for the url in this transaction
netskope.log_stream.cloud.app.category keyword Cloud application category from the CCI database
netskope.log_stream.cloud.app.name keyword Cloud application name
netskope.log_stream.cloud.app.suite_name keyword The cloud application suite name
netskope.log_stream.cloud.app.tags keyword Cloud application tags from the CCI database
network.application keyword Application level protocol name.
network.bytes long Total bytes transferred in both directions.
observer.product keyword The product name of the observer.
observer.vendor keyword Vendor name of the observer.
rule.ruleset keyword Rule ruleset
source.bytes long Bytes sent from the source to the destination.
source.geo.city_name keyword City name.
source.geo.country_name keyword Country name.
source.geo.postal_code keyword Postal code.
source.geo.region_name keyword Region name.
source.ip ip IP address of the source.
source.nat.ip ip Source NAT ip
tls.cipher keyword String indicating the cipher used during the current connection.
tls.client.ja3 keyword A hash that identifies clients based on how they perform an SSL/TLS handshake.
tls.client.server_name keyword Hostname the client is trying to connect to. Also called the SNI.
tls.server.ja3s keyword A hash that identifies servers based on how they perform an SSL/TLS handshake.
tls.server.not_after date Timestamp indicating when server certificate is no longer considered valid.
tls.server.not_before date Timestamp indicating when server certificate is first considered valid.
url.domain keyword Domain of the url.
url.original wildcard Unmodified original url as seen in the event source.
url.port long Port of the request, such as 443.
url.query keyword Query string of the request.
url.scheme keyword Scheme of the url.
user.email keyword User email address.
user.name keyword Short name or login of the user.
user_agent.name keyword Name of the user agent.
user_agent.original keyword Unparsed user_agent string.
user_agent.os.name keyword Operating system name, without the version.
user_agent.os.type keyword Which commercial OS family (one of: linux, macos, unix or windows).
user_agent.version keyword Version of the user agent.

For more information on the Intake Format, please find the code of the Parser, Smart Descriptions, and Supported Events here.