Office 365 Message Trace

Overview

Vendor: Microsoft
Supported environment: Cloud
Version compatibility:
Detection based on: Telemetry
Supported application or feature:

Microsoft 365 Message trace follows email messages as they travel through your Exchange Online organization. You can determine if a message was received, rejected, deferred, or delivered by the service. It also shows what actions were taken on the message before it reached its final status. You can use the information from message trace to efficiently answer user questions about what happened to messages, troubleshoot mail flow issues, and validate policy changes (More information on microsoft.com).

Prerequisite

According to docs.microsoft.com, Message Trace is available to the following plans :

Exchange Online Protection
Microsoft Defender for Office 365 plan 1 and plan 2
Microsoft 365 Defender

In Sekoia.io XDR, create a new intake key using the "Message Trace" format.

Configure OAuth

Collect your Tenant ID from your Azure Portal (for more information read (How to find your Microsoft Entra ID (Azure AD) tenant ID).

Add application:

Azure Portal, navigate to App registrations
Register an application
Name and create your application
Collect the Application (client) ID (client_id) from the Overview page

Create client's secret:

From your newly created client page, navigate to Certificates & secrets
In the Client secrets view
Add a client secret by choosing + New client secret
Once create, copy the secret value (client_secret)

Add required permission:

From your newly created client page, navigate to API permissions
Add a permissions
APIs my organization uses
Office 365 Exchange Online
Application permissions
ReportingWebService.Read.All
Add permissions
To finish, use the Grant admin consent for TENANT_NAME button

Add required role:

From the Microsoft Entra ID (Azure AD) page
Open Roles and administrators
Search and open Global Reader
Use the + Add assignments to add this role to your application

For a detailed explanation, please read the related documentation Azure portal - Assign a role.

You can now create the playbook "Create a new playbook > Create a playbook from scratch" and add the "Office 365 Message Trace OAuth" trigger.

Create a trigger configuration and input the following information:

client_id
client_secret
intake_key
tenant_id

Save your configuration and start the trigger.

Configure Basic Auth (Deprecated)

In Microsoft Azure, create a service account with the Reports reader and Security reader rights and a strong password (for more information: MessageTrace report required permissions and About admin roles in the Microsoft 365 admin center).

Once a dedicated service account as been created, you can validate user's rights by running the following command:

curl --user USERNAME:PASSWORD 'https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$format=json'

This command will contact Microsoft Reporting Web Service API with the provided credentials.

The response should look like this one (you can use |

jq

to format the response): data-linenos=" 1 ">{ "d": { "results": [ { "__metadata": { "id": "https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(0)", "uri": "https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(0)", "type": "TenantReporting.MessageTrace" }, "Organization": "corp.onmicrosoft.com", "MessageId": "<123@sender.foo>", "Received": "/Date(1659020996164)/", "SenderAddress": "sender@mail.sender.foo", "RecipientAddress": "user@corp.net", "Subject": "Subject", "Status": "Delivered", "ToIP": null, "FromIP": "1.1.1.1", "Size": 33435, "MessageTraceId": "579cd25c-9b30-4ce4-d5eb-08da70ab3b38", "StartDate": "/Date(1658851300604)/", "EndDate": "/Date(1659024100604)/", "Index": 0 } ] } }

You can now create the playbook "Create a new playbook > Create a playbook from scratch" and add the "Message Trace" trigger.

Create a trigger configuration and input the following information:

account_name
account_password
intake_key

Save your configuration and start the trigger.

Configure time range

Configure the trigger timedelta parameter to pull only events from 24 hours ago (1440 minutes) to be compliant with Microsoft documentation:

Events may be delayed by up to 24 hours before they appear in a report.

Configure the trigger start_time parameter to pull events from X hours ago if you want to import events that happened in the past. You can, for example, pull events from 30 days ago to now as explain in Microsoft documentation:

The information for this report is available for a period of 30 days, or until the subscription is canceled.

Debug

If your user cannot access the MessageTrace API, please visit the Azure Sign-in Logs dashboard and use the Add filters button to filter on the username. You can then choose an authentication event to learn about the issue and use the Launch the Sign-in Diagnostic. button to go further.

Raw Events Samples

In this section, you will find examples of raw logs as generated natively by the source. These examples are provided to help integrators understand the data format before ingestion into Sekoia.io. It is crucial for setting up the correct parsing stages and ensuring that all relevant information is captured.

messagetracemessagetrace2

{
    "__metadata": {
        "id": "https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(0)",
        "uri": "https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(0)",
        "type": "TenantReporting.MessageTrace"
    },
    "Organization": "examplecorp.onmicrosoft.com",
    "MessageId": "<3a273efc-cd65-4335-96ec-5f6934f0fb10@az.uksouth.production.microsoft.com>",
    "Received": "/Date(1658751973240)/",
    "SenderAddress": "azure-noreply@microsoft.com",
    "RecipientAddress": "foo.bar@example.corp",
    "Subject": "PIM: MessageTrace API service account has the Privileged Role Administrator role",
    "Status": "GettingStatus",
    "ToIP": null,
    "FromIP": "1.1.1.1",
    "Size": 87680,
    "MessageTraceId": "3b4fc661-180d-4c2f-60c9-08da6e38dd10",
    "StartDate": "/Date(1658579297628)/",
    "EndDate": "/Date(1658752097628)/",
    "Index": 0
}

{
    "__metadata": {
        "id": "https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(5)",
        "uri": "https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(5)",
        "type": "TenantReporting.MessageTrace"
    },
    "Organization": "abc.onmicrosoft.com",
    "MessageId": "<123456@abc-prod2.mcc-soft.com>",
    "Received": "/Date(1661344992170)/",
    "SenderAddress": "support@abc.com",
    "RecipientAddress": "user@abc.fr",
    "Subject": null,
    "Status": "Delivered",
    "ToIP": null,
    "FromIP": null,
    "Size": 0,
    "MessageTraceId": "1203cd7a-18d5-4a92-4343-08da85ce34c9",
    "StartDate": "/Date(1661344937835)/",
    "EndDate": "/Date(1661344997835)/",
    "Index": 5
}

Detection section

The following section provides information for those who wish to learn more about the detection capabilities enabled by collecting this intake. It includes details about the built-in rule catalog, event categories, and ECS fields extracted from raw events. This is essential for users aiming to create custom detection rules, perform hunting activities, or pivot in the events page.

The following Sekoia.io built-in rules match the intake Microsoft 365 Message Trace. This documentation is updated automatically and is based solely on the fields used by the intake which are checked against our rules. This means that some rules will be listed but might not be relevant with the intake.

SEKOIA.IO x Microsoft 365 Message Trace on ATT&CK Navigator

Cryptomining

Detection of domain names potentially related to cryptomining activities.

Effort: master

Dynamic DNS Contacted

Detect communication with dynamic dns domain. This kind of domain is often used by attackers. This rule can trigger false positive in non-controlled environment because dynamic dns is not always malicious.

Effort: master

Exfiltration Domain

Detects traffic toward a domain flagged as a possible exfiltration vector.

Effort: master

Remote Access Tool Domain

Detects traffic toward a domain flagged as a Remote Administration Tool (RAT).

Effort: master

SEKOIA.IO Intelligence Feed

Detect threats based on indicators of compromise (IOCs) collected by SEKOIA's Threat and Detection Research team.

Effort: elementary

Sekoia.io EICAR Detection

Detects observables in Sekoia.io CTI tagged as EICAR, which are fake samples meant to test detection.

Effort: master

TOR Usage Generic Rule

Detects TOR usage globally, whether the IP is a destination or source. TOR is short for The Onion Router, and it gets its name from how it works. TOR intercepts the network traffic from one or more apps on user’s computer, usually the user web browser, and shuffles it through a number of randomly-chosen computers before passing it on to its destination. This disguises user location, and makes it harder for servers to pick him/her out on repeat visits, or to tie together separate visits to different sites, this making tracking and surveillance more difficult. Before a network packet starts its journey, user’s computer chooses a random list of relays and repeatedly encrypts the data in multiple layers, like an onion. Each relay knows only enough to strip off the outermost layer of encryption, before passing what’s left on to the next relay in the list.

Effort: master

Event Categories

The following table lists the data source offered by this integration.

Data Source	Description
`Mail server`	Message trace follows email messages as they travel through your Exchange Online organization.
`Email gateway`	Message trace follows email messages as they travel through your Exchange Online organization.

In details, the following table denotes the type of events produced by this integration.

Name	Values
Kind	``
Category	`email`
Type	`info`

Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the built-in detection rules and hunting activities in the events page. Understanding these transformations is essential for analysts to create effective detection mechanisms with custom detection rules and to leverage the full potential of the collected data.

messagetrace.jsonmessagetrace2.json

{
    "message": "{\"__metadata\": {\"id\": \"https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(0)\", \"uri\": \"https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(0)\", \"type\": \"TenantReporting.MessageTrace\"}, \"Organization\": \"examplecorp.onmicrosoft.com\", \"MessageId\": \"<3a273efc-cd65-4335-96ec-5f6934f0fb10@az.uksouth.production.microsoft.com>\", \"Received\":\"/Date(1658751973240)/\", \"SenderAddress\": \"azure-noreply@microsoft.com\", \"RecipientAddress\": \"foo.bar@example.corp\", \"Subject\": \"PIM: MessageTrace API service account has the Privileged Role Administrator role\", \"Status\": \"GettingStatus\", \"ToIP\": null, \"FromIP\": \"1.1.1.1\", \"Size\": 87680, \"MessageTraceId\": \"3b4fc661-180d-4c2f-60c9-08da6e38dd10\", \"StartDate\":\"/Date(1658579297628)/\", \"EndDate\":\"/Date(1658752097628)/\", \"Index\": 0}",
    "event": {
        "action": "GettingStatus",
        "category": [
            "email"
        ],
        "type": [
            "info"
        ]
    },
    "@timestamp": "2022-07-25T12:26:13.240000Z",
    "email": {
        "from": {
            "address": [
                "azure-noreply@microsoft.com"
            ]
        },
        "message_id": "<3a273efc-cd65-4335-96ec-5f6934f0fb10@az.uksouth.production.microsoft.com>",
        "subject": "PIM: MessageTrace API service account has the Privileged Role Administrator role",
        "to": {
            "address": [
                "foo.bar@example.corp"
            ]
        }
    },
    "office365": {
        "message_trace": {
            "MessageTraceId": "3b4fc661-180d-4c2f-60c9-08da6e38dd10",
            "Size": 87680
        }
    },
    "organization": {
        "name": "examplecorp.onmicrosoft.com"
    },
    "related": {
        "ip": [
            "1.1.1.1"
        ]
    },
    "source": {
        "address": "1.1.1.1",
        "ip": "1.1.1.1"
    }
}

{
    "message": "{\"__metadata\":{\"id\":\"https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(5)\",\"uri\":\"https://reports.office365.com/ecp/ReportingWebService/Reporting.svc/MessageTrace(5)\",\"type\":\"TenantReporting.MessageTrace\"},\"Organization\":\"abc.onmicrosoft.com\",\"MessageId\":\"<123456@abc-prod2.mcc-soft.com>\",\"Received\":\"/Date(1661344992170)/\",\"SenderAddress\":\"support@abc.com\",\"RecipientAddress\":\"user@abc.fr\",\"Subject\":null,\"Status\":\"Delivered\",\"ToIP\":null,\"FromIP\":null,\"Size\":0,\"MessageTraceId\":\"1203cd7a-18d5-4a92-4343-08da85ce34c9\",\"StartDate\":\"/Date(1661344937835)/\",\"EndDate\":\"/Date(1661344997835)/\",\"Index\":5}\n",
    "event": {
        "action": "Delivered",
        "category": [
            "email"
        ],
        "type": [
            "info"
        ]
    },
    "@timestamp": "2022-08-24T12:43:12.170000Z",
    "email": {
        "from": {
            "address": [
                "support@abc.com"
            ]
        },
        "message_id": "<123456@abc-prod2.mcc-soft.com>",
        "to": {
            "address": [
                "user@abc.fr"
            ]
        }
    },
    "office365": {
        "message_trace": {
            "MessageTraceId": "1203cd7a-18d5-4a92-4343-08da85ce34c9",
            "Size": 0
        }
    },
    "organization": {
        "name": "abc.onmicrosoft.com"
    }
}

Extracted Fields

The following table lists the fields that are extracted, normalized under the ECS format, analyzed and indexed by the parser. It should be noted that infered fields are not listed.

Name	Type	Description
`@timestamp`	`date`	Date/time when the event originated.
`email.from.address`	`keyword`	The sender's email address.
`email.message_id`	`keyword`	Value from the Message-ID header.
`email.subject`	`keyword`	The subject of the email message.
`email.to.address`	`keyword`	Email address of recipient.
`event.action`	`keyword`	The action captured by the event.
`event.category`	`keyword`	Event category. The second categorization field in the hierarchy.
`event.type`	`keyword`	Event type. The third categorization field in the hierarchy.
`office365.message_trace.MessageTraceId`	`keyword`	An identifier used to get the detailed message transfer trace information.
`office365.message_trace.Size`	`number`	The size of the message, in bytes.
`organization.name`	`keyword`	Organization name.
`source.ip`	`ip`	IP address of the source.

For more information on the Intake Format, please find the code of the Parser, Smart Descriptions, and Supported Events here.