Azure Network Watcher

Overview

Azure Network Watcher provides tools to monitor, diagnose, view metrics, and enable or disable logs for resources in an Azure virtual network. It also allows to log information about IP traffic flowing through a network security group: NSG flow logs.

Event Categories

The following table lists the data source offered by this integration.

Data Source	Description
`Host network interface`	every packets passing through the Network Security Group are logged
`Netflow/Enclave netflow`	Azure Network Watcher NSG Flow Logs are Netflow-like
`Network device logs`	packets logged by NSG Flow Logs
`Network protocol analysis`	traffic analysis at levels 2/3/4

In details, the following table denotes the type of events produced by this integration.

Name	Values
Kind	`event`
Category	`network`
Type	``

Event Samples

Find below few samples of events and how they are normalized by Sekoia.io.

test_begin.jsontest_end.jsontest_short.json

{
    "message": "{\"flow_state\": \"begin\",\"resourceId\":\"/SUBSCRIPTIONS/13C8046C-DB72-4C35-9D71-60667ED9E869/RESOURCEGROUPS/INTEGRATION/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/TEST-NSG\",\"macAddress\":\"DB831EFEC376\",\"flow.0\":\"1493763938,1.2.3.4,5.6.7.8,35370,23,T,I,A,B,,,,\",\"rule\":\"DefaultRule_AllowVnetOutBound\",\"operationName\":\"NetworkSecurityGroupFlowEvents\",\"time\":\"2020-12-14T22:16:46.3528160Z\",\"version\":\"2\"}",
    "event": {
        "kind": "event",
        "category": [
            "network"
        ],
        "code": "NetworkSecurityGroupFlowEvents",
        "action": "accept",
        "type": [
            "allowed"
        ]
    },
    "rule": {
        "name": "DefaultRule_AllowVnetOutBound"
    },
    "action": {
        "type": "DefaultRule_AllowVnetOutBound",
        "target": "network-traffic",
        "properties": [
            {
                "OperationName": "NetworkSecurityGroupFlowEvents",
                "FlowState": "begin",
                "Version": "2"
            }
        ],
        "name": "accept"
    },
    "host": {
        "name": "/SUBSCRIPTIONS/13C8046C-DB72-4C35-9D71-60667ED9E869/RESOURCEGROUPS/INTEGRATION/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/TEST-NSG"
    },
    "network": {
        "transport": "tcp",
        "direction": "inbound"
    },
    "source": {
        "ip": "1.2.3.4",
        "port": 35370,
        "mac": "DB831EFEC376",
        "address": "1.2.3.4"
    },
    "destination": {
        "ip": "5.6.7.8",
        "port": 23,
        "address": "5.6.7.8"
    },
    "related": {
        "ip": [
            "1.2.3.4",
            "5.6.7.8"
        ]
    }
}

{
    "message": "{\"flow_state\": \"end\", \"resourceId\":\"/SUBSCRIPTIONS/13C8046C-DB72-4C35-9D71-60667ED9E869/RESOURCEGROUPS/INTEGRATION/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/TEST-NSG\",\"macAddress\":\"DB831EFEC376\",\"flow.0\":\"1607984156,1.2.3.4,5.6.7.8,36422,8086,T,O,A,E,1,74,1,74\",\"rule\":\"DefaultRule_AllowVnetOutBound\",\"operationName\":\"NetworkSecurityGroupFlowEvents\",\"time\":\"2020-12-14T22:16:46.3528160Z\",\"version\":\"2\"}",
    "event": {
        "kind": "event",
        "category": [
            "network"
        ],
        "code": "NetworkSecurityGroupFlowEvents",
        "action": "accept",
        "type": [
            "allowed"
        ]
    },
    "rule": {
        "name": "DefaultRule_AllowVnetOutBound"
    },
    "action": {
        "type": "DefaultRule_AllowVnetOutBound",
        "target": "network-traffic",
        "properties": [
            {
                "OperationName": "NetworkSecurityGroupFlowEvents",
                "FlowState": "end",
                "Version": "2"
            }
        ],
        "name": "accept"
    },
    "host": {
        "name": "/SUBSCRIPTIONS/13C8046C-DB72-4C35-9D71-60667ED9E869/RESOURCEGROUPS/INTEGRATION/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/TEST-NSG"
    },
    "network": {
        "transport": "tcp",
        "direction": "outbound"
    },
    "source": {
        "ip": "1.2.3.4",
        "port": 36422,
        "packets": 1,
        "bytes": 74,
        "mac": "DB831EFEC376",
        "address": "1.2.3.4"
    },
    "destination": {
        "ip": "5.6.7.8",
        "port": 8086,
        "packets": 1,
        "bytes": 74,
        "address": "5.6.7.8"
    },
    "related": {
        "ip": [
            "1.2.3.4",
            "5.6.7.8"
        ]
    }
}

{
    "message": "{\"flow_state\": \"begin\", \"source_addr\": \"1.3.4.2\", \"macAddress\": \"DB831EFEC376\", \"operationName\": \"NetworkSecurityGroupFlowEvents\", \"resourceId\": \"/SUBSCRIPTIONS/13C8046C-DB72-4C35-9D71-60667ED9E869/RESOURCEGROUPS/INTEGRATION/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/TEST-NSG\", \"time\": \"2021-03-24T10:55:03.0680749Z\", \"rule\": \"DefaultRule_AllowInternetOutBound\", \"flow.0\": \"1616583277,1.2.3.4,5.6.7.8,55486,443,T,O,A\"}",
    "event": {
        "kind": "event",
        "category": [
            "network"
        ],
        "code": "NetworkSecurityGroupFlowEvents",
        "action": "accept",
        "type": [
            "allowed"
        ]
    },
    "rule": {
        "name": "DefaultRule_AllowInternetOutBound"
    },
    "action": {
        "type": "DefaultRule_AllowInternetOutBound",
        "target": "network-traffic",
        "properties": [
            {
                "OperationName": "NetworkSecurityGroupFlowEvents",
                "FlowState": "begin"
            }
        ],
        "name": "accept"
    },
    "host": {
        "name": "/SUBSCRIPTIONS/13C8046C-DB72-4C35-9D71-60667ED9E869/RESOURCEGROUPS/INTEGRATION/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/TEST-NSG"
    },
    "network": {
        "transport": "tcp",
        "direction": "inbound"
    },
    "source": {
        "ip": "1.3.4.2",
        "port": 55486,
        "mac": "DB831EFEC376",
        "address": "1.3.4.2"
    },
    "destination": {
        "ip": "5.6.7.8",
        "port": 443,
        "address": "5.6.7.8"
    },
    "related": {
        "ip": [
            "1.3.4.2",
            "5.6.7.8"
        ]
    }
}

Extracted Fields

The following table lists the fields that are extracted, normalized under the ECS format, analyzed and indexed by the parser. It should be noted that infered fields are not listed.

Name	Type	Description
`@timestamp`	`date`	Date/time when the event originated.
`action.properties`	`array`	action.properties
`action.target`	`keyword`	The target of the action
`destination.bytes`	`long`	Bytes sent from the destination to the source.
`destination.ip`	`ip`	IP address of the destination.
`destination.packets`	`long`	Packets sent from the destination to the source.
`destination.port`	`long`	Port of the destination.
`event.category`	`keyword`	Event category. The second categorization field in the hierarchy.
`event.code`	`keyword`	Identification code for this event.
`event.kind`	`keyword`	The kind of the event. The highest categorization field in the hierarchy.
`host.name`	`keyword`	Name of the host.
`rule.name`	`keyword`	Rule name
`source.bytes`	`long`	Bytes sent from the source to the destination.
`source.ip`	`ip`	IP address of the source.
`source.mac`	`keyword`	MAC address of the source.
`source.packets`	`long`	Packets sent from the source to the destination.
`source.port`	`long`	Port of the source.

Configure

Please contact your support to discuss about the network security group to monitor in your Azure infrastructure in order to find the appropriate solution to forward your logs to Sekoia.io.

This setup guide will show you a method to enable and give us access to NSG flow logs produced by Azure Network Watcher service to Sekoia.io.

Enable NSG flow logs

The following instructions are provided for the Azure web portal (https://portal.azure.com).

As a prerequisite you need at least one virtual machine with a network security group, to enable Network Watcher and to register the Microsoft.Insights provider.

Navigate to the Network Watcher service, and select NSG flow logs under LOGS. From the list of NSGs, select your VM(s), and under Flow logs settings, select On to enable the NSG flow logs. Please, select the Version 2 NSG flow log format sample which is integrated to the Operations Center.

These instructions are illustrated and more detailled here.

This part should be discussed with Sekoia.io people to find an appropriate solution to forward your flow logs to Sekoia.io.

A possible solution consists to share us: - An access key for the Azure Blob Storage - A storage token associated with the resources to share - The name of the container where the NSG flow logs are stored

From this information, we will be able to retrieve each PT1h.json blob which contains the flow logs.