Azure AD Smart Lock-Out

Have you ever wondered how Microsoft prevents password guessing attacks or brute force attacks on Azure AD? Well, it is basically the same method as you would do on your on-premises User Directory! It is just smarter!
Azure AD Smart lockout is a feature being applied to every sign-in processed by Azure AD, regardless if the user has a managed account or a synced accounts using password hash sync or pass-through authentication.
The smart part comes from the ability to distinguish valid users from attackers. It locks out the attackers while letting your users continue to access their accounts and be productive.

Note: If you still run ADFS, there is also a Feature available named Extranet Smart Lockout but this one is not as smart as the one in Azure AD. https://support.microsoft.com/en-us/help/4096478/extranet-smart-lockout-feature-in-windows-server-2016

The default lockout setting kicks in after ten invalid login attempts for one minute. The account locks again after each subsequent failed attempt, for one minute at first and longer periods in subsequent attempts. Also, smart lockout tracks the last three bad password hashes to avoid incrementing the lockout counter for the same password, basically, if the same bad password is entered multiple times, it will not cause another lockout.

Note: The monitoring of the same sign-in attempt is only available for the Password Hash sync scenario, as the Pass-Through password validation happens against your on-premise AD domain controllers.

Smart Lockout is always turned on for all Azure AD customers. If you want to modify the default behavior of 10 invalid attempts to trigger a one-minute lockout, then you require Azure AD P1 or P2 licenses for your users. You can then modify the number of trigger attempts and the lockout duration.

User experience:
If a user account is locked, you will get the following message while the account is locked:

Azure AD Logs

Azure Active Directory provides multiple logs to track what is going on:

Activity-based logs

Sign-ins – This log provides data about user sign-in activities.

Audit logs – These logs provide system activity information about users and group management, managed applications and directory activities. Basically what was modified by whom at which point in time.

Security logs in Azure AD Identity protection

This feature requires an Azure AD Premium P2 plan. The Azure AD Identity Protection Overview page provides information on trends for risky sign-in attempts that might have been performed by someone who is not the legitimate owner of a user account and also for users flagged for risk. Where a risky user is an indicator for a user account that might have been compromised.

How to monitor Smart Lockout?

You can integrate Azure AD activity logs with Azure Monitor logs. So you can use the power of Azure Monitor logs to gain insights into your environment. You can also install the Log analytics views for Azure AD activity logs to get access to pre-built reports around audit and sign-in events in your environment.

I will explain how to create a Smart Lockout specific Alert and Monitor here.

1. In Azure Portal,
   Select Azure Active Directory > Diagnostic settings -> Add diagnostic setting. select Export Settings from the Audit Logs or Sign-ins page to get to the diagnostic settings configuration page.
2. In the Diagnostic settings menu, select the Send to Log Analytics workspace check box, and then select Configure.
3. Select the Log Analytics workspace you want to send the logs to, or create a new workspace in the provided dialog box. (If do do not have any Analytics Workspace yet, please read here.
4. Select one or both of the following:
   • To send audit logs to the Log Analytics workspace, select the AuditLogs check box
   • To send sign-in logs to the Log Analytics workspace, select the SignInLogs check box
5. Select Save to save the setting.

Now we need to wait about 5-10 minutes until the first data flows into our Log Analytics workspace. In Azure Portal, Select Azure Active Directory, and then select Logs from the Monitoring section to open your Log Analytics workspace. The workspace will open with a default query.

The logs we are looking for are in the AuditLogs and SigninLogs tables in the workspace . You can type queries in the Kusto language, like

SigninLogs
| limit 50

and click on run to get the last 50 sign-ins listed. As we are interested into the actual Smart-Lockouts (ResultType 50053), we need to add a filter criteria to our query and we probably want to limit it to the last seven days only.

SigninLogs 
| where CreatedDateTime >= ago(7d)
| where ResultType == "50053" 

And because we like to get notified, we can also set an Alert Rule on this type of query. Click on New Alert Rule, which will bring you the next blade.

Click on Done.

Select the Action Group or create one that will be alerted when the signal occurs. You can choose to notify your team via email, text message Push message to the Azure App, or you could automate the action using webhooks, Azure functions or logic apps.

Enter a Alert Rule name and description for the alert, and choose the severity level and create the rule. I personally like the Azure

Please note, the events for a Smart Lockout account need about 2-5 minutes to show up in the Log Analytics workspace. Only when they have arrived there, they can trigger a signal and finally an alert.