Incident Response in a Microsoft cloud environment

Microsoft Detection and Response (DART) team recently shared a PowerShell module, that they are using in their IR engagements, so I thought it would be great to blog about it. I’ve previously blogged about IR in Azure AD, but today I want to extend it further. This blog post includes more details comparing to my previous blog posts, regarding to IR.

We are going to cover different stuff, which includes the remediation part. What are the steps that we have to follow when there is an incident? Well, that’s been said. We will also hunt for some techniques that have been used in the ”wild”

In this blog post, we are going through a few examples of the PowerShell module and see what data it collects. After we have finished our data collection, which includes collecting Unified Audit Logs. We will export all the data in Azure Data Explorer and use different techniques to parse out the unstructured data, that might be stored in a JSON format.

PowerShell commands

The PowerShell module is using Graph API under the hood to access data, so it starts with obtaining an API access token. Once the token has been obtained, we can call a resource and it will return the data back.

cmdletDescription
Get-AzureADIRAuditActivityGets Audit log details for target Users, Service Principals, Event Categories or services logging the events.
Get-AzureADIRConditionalAccessPolicyGets Azure Active Directory Conditional Access policies.
Get-AzureADIRDismissedUserRiskGets all Identity Protection User Risk dismissals.
Get-AzureADIRDisplayNameToObjectIdLooks up a display name and shows its object ID.
Get-AzureADIRDomainRegistrationDetailGenerates a list of domains from Azure AD and then checks whois information to display Name Servers, Admin and Registrant.
Get-AzureADIRMfaAuthMethodAnalysisAnalyses Azure AD users to make recommendations on how to improve their MFA stance.
Get-AzureADIRMfaPhoneToLocationCheckAnalyses Azure AD users to compare usage location to MFA / alternative phone number location
Get-AzureADIRObjectIdToDisplayNameLooks up an ObjectId and displays its human-friendly properties
Get-AzureADIRPimPrivilegedRoleAssignmentGets PIM privileged roles assignments.
Get-AzureADIRPimPrivilegedRoleAssignmentRequestGets PIM assignment related activity
Get-AzureADIRPrivilegedRoleAssignmentGets a list of directory roles and members.
Get-AzureADIRPrivilegedUserOnPremCorrelationGets a list of directory roles, members and any associated on-premises privileged groups.
Get-AzureADIRSignInDetailGets Sign-In log details for target users, clients or resources
Get-AzureADIRSsprUsageHistoryGets SSPR usage history.
Get-AzureADIRTenantIdRetrieves the tenant ID for a supplied domain name.
Get-AzureADIRUserLastSignInActivityGets Azure Active Directory user last interactive sign-in activity details.
Get-AzureADIRPermissionProduces CSV reports of app permissions. Can also list all permissions.

List members in Directory roles

The first thing we are going to do is, exporting all the members in each admin role, because users in such roles are considered ‘privileged’

In order to do this, we have to use the Get-AzureADIRPrivilegedRoleAssignment cmdlet.

Results:

List PIM assignments

At this section, we are going to do something similar as the previous one. However, now we are going to export all the Privileged Identity Management (PIM) assignments of users. When assigning a role via PIM, it is possible to assign the role permanent or make it ‘eligible’. This is important to be aware of, because it will show you if an organization is using PIM on a correct way. (e.g. Too many permanent Global Admins)

In order to get all PIM assignments, we have to use the Get-AzureADIRPimPrivilegedRoleAssignment cmdlet.

Results:

List MFA status of accounts

At this section, we are going to export all the MFA status for every account in a tenant. This will count every MFA authentication method, an account has registered for, and give a recommendation around best practices for MFA settings.

In order to export all the MFA status of accounts, we have to use the Get-AzureADIRMfaAuthMethodAnalysis cmdlet.

Results:

List last logon activity

At this section, we are going to export the last logon activity of all the accounts in a tenant. This gives us a overview of accounts that may have high-privileges, but never logged in for example. It also includes guest accounts in the exported data.

In order to get the last logon activity of the accounts, we have to use the Get-AzureADIRUserLastSignInActivity cmdlet.

Results:

List application permissions

At this section, we are going to export all the Azure AD applications in a tenant. This includes the associated permissions that have been delegated to a principal.

In order to do this, we have to use the Get-AzureADIRPermission cmdlet.

Results:

Exporting Unified Audit Logs

Unified audit logs contains user, group, application, domain, and directory activities performed in the Microsoft 365 admin center or in the Azure management portal. Here we can find activities that might be ‘suspicious’, but it’s not easy as it sounds. The first thing to be aware of is that, when an organization has an E3 license. The data retention is 90 days, while organizations with an E5 license. They have a data retention of 365 days.

Another important thing to note is, that once you search for data beyond 90 days. It will only show the users with an E5 license in the results. Other users that don’t have an E5 license will be left out.

Instead of using the PowerShell module, we are going to export the data from the portal.

Instead of looking data for all the users. I’ll recommend only to select users that are part of at least one ‘admin’ role in Azure AD. We’ll get later more about this.

Use-case

We are going to pretend that we have do an incident response for an organization. The first thing we did was asking for permissions in their tenant.

Ok, so we now have ‘Global Reader’ rights, which allows us to view every setting in their tenant. The second thing we did was running the PowerShell module to export the relevant data. The last thing we had to do was exporting the unified audit logs. As a best practice, we decided to only select accounts that have an admin role (e.g. Global Admin, Helpdesk Admin, Application Admin, etc) at the ‘users’ search bar at the unified audit logs.

As an example, we have only selected accounts that are part of at least one admin role. All of the accounts do need to have the appropriate license, so you can search beyond 90 days of data.

If you did everything correctly, you should now have a couple of CSV files. Save these files, because we are going to upload them in Azure Data Explorer to do further analysis & remediation.

Uploading data in Azure Data Explorer

Azure Data Explorer is an analytic engine that is similar to Log Analytics. However, when it comes down to analyzing large amount of data. It is preferred to use Azure Data Explorer, because it has more capabilities.

The first thing we have to do is create an Azure Data Explorer cluster. Once we have done that, we can start creating a database.

In this example, we have created a database that’s called ‘ir’

In that database, we can create a couple of tables and ingest data to query it.

Ingest all the CSV files that we have exported and we are good to go!

After we have ingested all our data. It should have a few tables in the ‘ir’ database that we can query, which looks similar to this.

Analyzing & remediation

We have ingested all the data, so now we can start querying it. In this section, we will analyze the exported CSV files and do some remediation. The goal is to get ‘trust’ back at an Azure tenant.

Here are a few important steps that should be done during an IR engagement.

Members in admin roles

FireEye has published a great hardening remediation plan for Microsoft 365. In their guidance, they shared a table of admin roles that attackers often target.

Every admin role can have a user, group or a service principal assigned to it. This list contains a few examples, but it’s not everything.

Source: https://www.fireeye.com/content/dam/collateral/en/wp-m-unc2452.pdf

Step 1: Counting all the amount of members in the specified admin roles

The first step will be just counting all the members in every directory role and visualize the results, which can later be presented to the management team. Like for example, highlighting that there were too many members in a certain role for example.

Query:

let directory_roles = dynamic(["Global Administrator","Application Administrator","Cloud Application Administrator","Exchange Administrator","Privileged Role Administrator","User Administrator","SharePoint Administrator","Hybrid Identity Administrator"]);
['roles2']
| where DirectoryRole in~ (directory_roles)
| summarize Count = count() by DirectoryRole
| render barchart

Results:

At the sample results, we have visualized the data. Here it will display the amount of members in every listed admin role.

Step 2: Identifying all the members in every specified admin role

The second step is to identify all the members in every specified admin role.

Query:

let directory_roles = dynamic(["Global Administrator","Application Administrator","Cloud Application Administrator","Exchange Administrator","Privileged Role Administrator","User Administrator","SharePoint Administrator","Hybrid Identity Administrator"]);
['roles2']
| where DirectoryRole in~ (directory_roles)
| summarize by DirectoryRole, RoleMemberName, RoleMemberOnPremDn, RoleMemberEnabled
| sort by DirectoryRole

Results:

At the sample results, we have now identified every member in the specified admin roles.

An interesting thing is to look at the ‘RoleMemberOnPremDn’ column, because once you discover a row in this column that is not empty. It means that a principal from On-Premises AD has been synced to Azure AD.

Accounts from On-Prem AD that have high-privileges in the Cloud are valuable targets for attackers. This means that once an attacker has compromised a synced account. It could potentially move laterally from On-Prem to Cloud.

Step 3: Remediation

  • Remediation:
    • Replace all the existing accounts in the specified roles with new accounts
    • Reset the password for every account in the specified admin roles
    • Revoke user session for every account in the specified admin roles
    • Block sign-in from the previous accounts in the specified admin roles, which may have been compromised
    • Global password reset for every account in the tenant
    • Optional: Replacing every account in all the admin roles with new accounts

Step 4: Recommendations

The first step is to see if an organization was using Privileged Identity Management (PIM). As we still remember, we have exported all the PIM assignments with the PowerShell module. The data is ingested in Azure Data Explorer, which we can query.

Query:

let directory_roles = dynamic(["Global Administrator","Application Administrator","Cloud Application Administrator","Exchange Administrator","Privileged Role Administrator","User Administrator","SharePoint Administrator","Hybrid Identity Administrator"]);
['roles']
| where RoleName in~ (directory_roles)
| summarize by RoleName, UserUpn, AssignmentState
| sort by UserUpn

Results:

At the sample results, we can see all the members in each role, but now we will also see if the role has been assigned permanent or eligible.

Let’s now take a look at the Global Administrator role in Azure AD.

Query:

['roles']
| where RoleName =~ 'Global Administrator'
| summarize by RoleName, UserUpn, AssignmentState
| sort by UserUpn

Results:

At the sample results, we can see that there are a couple of members in Global Admin and one role has been assigned as eligible.

The second step is to review and see if there are any Conditional Access policies are in place to restrict access for admin roles from untrusted locations or non-compliant devices.

We can get all the Conditional Access policies by using the PowerShell module of DART and use the Get-AzureADIRConditionalAccessPolicy cmdlet.

  • Recommendations:
    • Use Privileged Identity Management (PIM) to assign temporary high-privileges.
    • Use Conditional Access to restrict access for admin roles when they sign-in from untrusted locations or perhaps devices that are not compliant.

Accounts with no MFA enabled

We have previously exported the MFA status of all the accounts in a tenant, so the second thing is now to discover accounts without MFA.

Query:

mfa
| where MfaAuthMethodCount == 0
| summarize Count = count() by MfaAuthMethodCount

Results:

At the sample results, we can see that 50 accounts don’t have registered for any MFA authentication method.

Step 1: Review all the accounts without MFA

The first step is to review and see which accounts don’t have MFA enabled yet.

Query:

mfa
| where MfaAuthMethodCount == 0
| where UserPrincipalName !startswith 'Sync_'
| project UserPrincipalName, ObjectId, Recommendations

Results:

At the sample results, we can see a couple accounts.

Step 2: Remediation

  • Remediation:
    • Review all the accounts that can sign-in and make sure MFA is enabled
    • An account needs to sign-in once to register MFA, so if an account has never signed-in before. Consider removing it from the tenant.

Removing inactive accounts

An organization might have tons of inactive accounts or even accounts that have never signed-in before. Does this mean that we have to remove all of them immediately? No, but the focus should be on accounts that are part of an admin role or guest accounts.

Step 1: Discovering inactive guest accounts

The first step is to cleanup all the inactive guest accounts in our tenant. At this example, we are going to run a simple query to find all the guest accounts that have never signed-in.

Query:

lastlogin
| where isempty(lastSignInDateTime)
| where userPrincipalName has 'EXT'

Results:

At the sample results, we can see all the guest accounts that have never signed-in before.

Step 2: Discovering inactive accounts that are part of admin roles

Reducing an attacker’s surface starts with revoking high-privileges from accounts that don’t need it anymore. In this example, we are going to discover inactive admin accounts that part of at least one admin role.

Query:

lastlogin
| where isempty(lastSignInDateTime)
| where userPrincipalName startswith 'adm'
| project-rename RoleMemberObjectId = objectId
| join (
    ['roles2']
    ) on RoleMemberObjectId
| project displayName, DirectoryRole, lastSignInDateTime
| sort by displayName

Results:

At the sample results, we can see all the inactive accounts with at least one admin role. All of these accounts have never signed-in before, which explains why the ‘lastSignInDateTime’ column is empty.

Step 3: Remediation

  • Remediation:
    • Revoke access for all the inactive Guest accounts in a tenant
    • Revoke all the admin roles from accounts that are inactive

Revoke client secrets / certificate from Azure AD Applications

A client secret is a secret known only to your application to prove it’s identity when requesting a token. It protects your resource by only granting an access token to the authorized persons that have access to the secret.

It looks similar to the following:

Step 1: Discover Azure AD apps with sensitive permissions

The first thing we are going to do is look for Azure AD applications with sensitive permissions.

Azure AD Application permissions can be assigned to delegated permissions or application permissions. The only difference is that, when delegated permissions are set. The application needs to have access as the signed-in user, while application permissions will run the app in the background, as a service without a signed-in user.

A few examples of sensitive permissions:

PermissionDescription
Directory.AccessAsUser.AllAllows the app to have the same access to information in the directory as the signed-in user.
Directory.ReadWrite.AllAllows the app to read and write data in your organization’s directory, such as users, and groups, without a signed-in user. Does not allow user or group deletion.
Directory.Read.AllAllows the app to read data in your organization’s directory, such as users, groups and apps.
Group.ReadWrite.AllAllows the app to create groups and read all group properties and memberships on behalf of the signed-in user. Additionally allows group owners to manage their groups and allows group members to update group content.
Application.ReadWrite.AllAllows the app to create, read, update and delete applications and service principals on behalf of the signed-in user.
Application.ReadWrite.OwnAllows the calling app to create other applications and service principals, and fully manage those applications and service principals (read, update, update application secrets and delete), without a signed-in user.
Device.ReadWrite.AllAllows the app to read and write all device properties without a signed in user.
Domain.ReadWrite.AllAllows the app to read and write domains without a signed-in user.
Mail.ReadApp wants to read files, mail and calendar information for the signed in user
Mail.ReadWriteAllows the app to create, read, update, and delete email in user mailboxes.
Mail.SendAllows the app to send mail as users in the organization.
Source: https://docs.microsoft.com/en-us/graph/permissions-reference

Query:

let graph_permission = dynamic(["Directory.AccessAsUser.All","Directory.ReadWrite.All","Group.ReadWrite.All","Application.ReadWrite.All","Application.ReadWrite.Own","Device.ReadWrite.All","Domain.ReadWrite.All","Mail.Read","Mail.ReadWrite"]);
apps
| where Permission in~ (graph_permission)
| summarize by ClientDisplayName, Permission, ResourceDisplayName
| sort by ClientDisplayName

Results:

At the sample results, we can see a few applications that have the mentioned sensitive permissions.

We have to validate that these applications have not been abused in an attack, so check if the organization is collecting Service Principal Sign-In logs. See if there are any service principal have signed-in from a untrusted location.

Well that’s been said. In most cases, organizations don’t collect this logs, which makes it harder to validate.

Step 3: Remediation

  • Remediation
    • Delete all the existing client secrets / certificates from applications with sensitive permissions
    • Remove all the owners of the applications and add the right one’s back to it
    • Create new client secrets and add them back to the applications

Hunting in Unified Audit Logs

After we have exported the unified audit logs and ingested the data in Azure Data Explorer. We can start analyzing it, but there are challenges around this. First, we have to be aware that the data is stored in a unstructured format, which makes it harder to analyze. Second, we need to be aware of the TTPs that have been used in the ‘wild’. This gives us a focus on the techniques we want to hunt for.

Step 1: Unified Audit logs data structure

Folks that are familiar with Log Analytics will recognize that the structure of the unified audit logs is not exactly the same. This is how an exported unified audit logs look like.

Step 2: TTPs – Azure AD

SolarWinds Post-Compromise has highlighted some TTPs that involved Azure Active Directory. Microsoft has published a blog post about this, which we can use as a reference in our hunting engagement.

During this blog post, we are going to hunt for the following TTPs:

  • New permissions granted to service principals
  • Modified application and service principal credentials/authentication methods
  • Application with permission to access mailboxes
  • Directory role and group membership updates for service principals
  • Modifying ownership of application

All the hunting queries that we ran are meant as an example, so no. It cannot be use as a detection rule, and yes. It can always be improved.

Step 3: Hunting – New permissions granted to service principals

During the SolarWinds Post-Compromise, when the attackers were not able to find a service principal or an application with high privileges. They will often attempt to add the permissions to a new service principal.

  1. First we are going to run a query that looks when a new service principal was added.

Query:

unified
| where Operations =~ 'Add service principal.'

Results:

At the sample results, we can see the column ‘AuditData’ that stores different fields in a JSON format.

2. Parsing the different fields in the ‘AuditData’ column

We are now going to parse different fields by using the parse_json function of KQL and later use the bag_unpack operator to parse the different fields into columns.

Query:

unified
| where Operations =~ 'Add service principal.'
| extend ParsedFields = parse_json(AuditData)
| evaluate bag_unpack(ParsedFields)
| extend AppName = tostring(ModifiedProperties[2].NewValue)
| extend TargetType = tostring(ModifiedProperties[3].Name)
| project CreationDate, UserIds, Operations, AppName, TargetType, ActorContextId

Result:

At the sample results, we can see which user has registered an application. However, we still don’t know which API permissions have been assigned to it.

3. Consent Permissions

We are now going to use the join operator to find out which permissions have been consented to an application. An addition to our query. We are only interested applications with specific permissions.

Query:

unified
| where Operations =~ 'Add service principal.'
| extend ParsedFields = parse_json(AuditData)
| evaluate bag_unpack(ParsedFields)
| extend AppName = tostring(ModifiedProperties[2].NewValue)
| extend TargetType = tostring(ModifiedProperties[3].Name)
| project CreationDate, UserIds, Operations, AppName, TargetType, ActorContextId
| join kind=leftouter(
    unified
    | where Operations =~ 'Consent to application.'
    | extend ParsedFields = parse_json(AuditData)
    | evaluate bag_unpack(ParsedFields)
    | extend Permissions = tostring(ModifiedProperties[4].NewValue)
    | extend Permissions = tostring(split(Permissions, "Scope:")[1])
    | extend Permissions = tostring(split(Permissions, "]]")[0])
    | extend Permissions = tostring(split(Permissions, "[Id:")[0])
    | where isnotempty(Permissions)
    | where Permissions has_any ('Directory.Read.All','User.Read.All','Application.Read.All','Group.Read.All') // Replace this with the Graph API permissions you are looking for
    ) on ActorContextId
    | project CreationDate, UserIds, Operations, AppName, Permissions, TargetType, ActorContextId

Results:

At the sample results, we can see a couple of applications that have the specified permissions that we were looking for. Now we can start investigate further and see if the application is also ‘malicious’ or not.

Step 4: Hunting – Modified application and service principal credentials/authentication methods

A common way for attackers to remain persistent in an environment is by adding new credentials to existing applications or service principals. This allows an attacker to authenticate as the targeted application or service principal and access the resources based on the permissions that have been set.

Query:

unified
| where Operations has ("Certificates and secrets management")
| extend ParsedFields = parse_json(AuditData)
| evaluate bag_unpack(ParsedFields)
| extend value_set = tostring(ModifiedProperties[0].NewValue)
| extend AppName = tostring(Target[3].ID)
| extend new_keyIdentifier = tostring(split(value_set, ",")[4])
| extend old_displayName = tostring(split(value_set, ",")[3])
| extend new_keyDisplayName = tostring(split(value_set, ",")[7])
| extend new_keyDisplayName = tostring(split(new_keyDisplayName, "DisplayName=")[1])
| extend new_keyDisplayName = tostring(split(new_keyDisplayName, "]")[0])
| extend new_keyIdentifier = tostring(split(new_keyIdentifier, "KeyIdentifier=")[1])
| parse value_set with * "KeyIdentifier=" keyIdentifier:string ",KeyType=" keyType:string ",KeyUsage=" keyUsage:string ",DisplayName=" keyDisplayName:string "]" *
| project CreationDate, UserIds, Operations, AppName, keyDisplayName, new_keyDisplayName, keyIdentifier, new_keyIdentifier, keyType, keyUsage

Results:

At the sample results, we can see that someone has added a client secret to an existing application. The ‘new_keyIdentifier’ column has a row, where the results is not empty. This means that someone added credentials to an existing application.

Tip:

When existing credentials has been added to an application. Look at to which application it was added and review if the app has sensitive permissions.

Step 5: Hunting – Application with permission to access mailboxes

During the SolarWinds Post-Compromise, the attackers have been using applications to access users mailboxes within a target environment. In this example, we are going to look for applications that have been granted mailbox permissions with a follow-up of permissions being consented.

Query:

unified
| where Operations =~ 'Add delegated permission grant.'
| extend ParsedFields = parse_json(AuditData)
| evaluate bag_unpack(ParsedFields)
| extend Permissions = tostring(ModifiedProperties[0].NewValue)
| extend ResourceDisplayName = tostring(Target[3].ID)
| where Permissions has_any ("Mail.Read", "Mail.ReadWrite","Mail.ReadWrite.All")
| join kind=leftouter(
    unified
    | where Operations =~ 'Consent to application.'
    | extend ParsedFields = parse_json(AuditData)
    | evaluate bag_unpack(ParsedFields)
    | extend Permissions = tostring(ModifiedProperties[4].NewValue)
    | extend AppName = tostring(Target[3].ID)
    | where Permissions has "Mail.Read"
    or Permissions has "Mail.ReadWrite"
    or Permissions has "Mail.ReadWrite.All"
    ) on ActorContextId
    | project CreationDate, UserIds, OperationName = Operations1, Permissions, AppName, ResourceDisplayName

Results:

At the sample results, we can see which principal has created an application and consented the mailbox permissions.

Step 6: Hunting – Directory role and group membership updates for service principals

Besides of creating or adding credentials to applications. Another approach is to add a service principal to directory roles or groups.

Query:

unified
| where Operations =~ 'Add member to role.'
| extend ParsedFields = parse_json(AuditData)
| evaluate bag_unpack(ParsedFields)
| extend Role = tostring(ModifiedProperties[1].NewValue)
| extend UserType = tostring(Target[0].ID)
| extend UserType = tostring(split(UserType, "_")[0])
| extend Principal = tostring(Target[3].ID)
| where UserType == 'ServicePrincipal'
| project CreationDate, UserIds, Operations, Principal, Role, UserType

Results:

At the sample result, we can see that a service principal was added to the ‘Application Administrator’ role. Since it is not very common that a service principal is being added to a directory role. It something worth to look at.

Step 7: Hunting – Modifying ownership of application

An identity that register an application will become automatically the owner. An attacker could persist by creating a regular account in Azure AD, and assign it the ‘Owner’ role to an application with sensitive permissions. This allows an attacker to have control over the application, so it could add new credentials to persist.

Query:

unified
| where Operations =~ 'Add owner to application.'
| extend ParsedFields = parse_json(AuditData)
| evaluate bag_unpack(ParsedFields)
| extend AppId = tostring(ModifiedProperties[2].NewValue)
| extend AppName = tostring(ModifiedProperties[1].NewValue)
| project CreationDate, Operations, UserIds, ObjectId, AppName, AppId

Result:

At the sample result, we can see which user has assigned ‘Owner’ permissions to an application.

Tip:

Look at which application the ‘Owner’ role has been assigned to it, but keep an eye on applications with sensitive permissions.

Summary

We have used the PowerShell module of Microsoft DART to export the relevant logs and ingested the data in Azure Data Explorer to analyze it further. After the data was ingested in an ADX cluster. We have ran a couple of KQL queries to audit the data, so we could start the remediation process. (e.g. Resetting passwords of admin accounts, revoking client secrets, and so on)

Once we have done everything, we started to hunt in the unified audit logs to look for ”evil” 😉 Based on the TTPs of the SolarWinds Post-Compromise hack. We showed a few examples on how to hunt for those techniques.

Unified Audit logs stores everything in a unstructured format, which can be a hard to parse out all the fields into different columns. However, as we demonstrated in this blog post. It is possible though!

Reference

2 comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s