As more organizations move their operations online, cloud accounts have become increasingly popular targets for cybercriminals. These adversaries use phishing, password-based attacks, or other methods to gain illegitimate access. The ideal place to detect account takeover, or “ATO”, is at the login stage — the head of the attack chain. Although many detections try to get away with using simple logic on single variables to catch malicious logins, these are typically ineffective and generate many false positives (FPs). In this post, I’ll explain how using more precise features — customized to every user account — results in better detections and fewer false positives.
The problems with simple detections
Simple cloud login detections employ explicit rules or heuristics that work on single variables to spot anomalous behavior. There are two flaws here. First, legitimate logins are commonly unusual in one way or another, which is why these detections can be noisy. For example, detecting logins at unusual times may create a false positive when a user works on the weekend because of a deadline, even if all other aspects of the login event are normal for that user. This is why multivariate anomaly detection — which looks for anomalies across multiple variables — can be much more discerning than univariate anomaly detection. 
By considering additional variables, multivariate anomaly detection makes it much easier to identify outliers.
The second flaw with these detections is that their simple logic often requires manual tuning, and the correct parameter values are not always known. Furthermore, if and when the data changes (e.g., a company grows, a user moves to a new city, etc), the logic can grow stale and can cease to be effective or, worse, can generate false positives.
A better approach: multivariate detection using UEBA and aggregate features
At ActZero, we’ve created a machine learning detection that looks for malicious logins to Microsoft 365 (M365) accounts using features that are tailored to each individual user (reminder: a feature is similar to a variable and is an input to a predictive detection). Rather than relying on a single variable, the detection uses a suite of features that provide a holistic view of the login event. Broadly speaking, these features are either UEBA or aggregate features.
What are UEBA features? UEBA (shorthand for user and entity behavior analytics) is a fancy way of describing features that reflect how weird an event is given the historical behavior of a user. For example, while it’s normal for me to log in from Seattle, it might be unusual for teammates based elsewhere. We create UEBA features by continuously learning and updating our understanding of what is normal login behavior for each user. This ability to adapt is a distinct advantage over simple detections that are brittle and can fail when a user’s behavior changes.
What are aggregate features? Aggregate features consider not just individual login events, but rather events over a period of time. Hence, they can capture spikes in activity, which is useful for identifying attacks such as brute-force password cracking. For example, a successful login from a new country might look a lot more suspicious in the context of failed login attempts from many different countries than it would on its own.
A real-world example
The figure below shows login activity into a user’s M365 account that we observed before we enabled our ATO detection. The x-axis shows several weeks of time and the y-axis represents the countries from which logins were attempted. The blue and red dots show successful and failed login attempts, respectively.
What you can see is that the user typically logged in from Canada and then, over the course of a couple of weeks, there were numerous successful logins from around the world. Not what you want to see on your account! Here I’m only showing country data; however, the graphs for other variables — login time of day, login device, etc — paint a similar picture. On January 15, the account was locked down and reset by an administrator who confirmed the malicious activity, resulting in no further logins from outside of Canada.
We’ve run our detection on this data, and it catches the anomalous behavior at its onset. The detection learned that the user normally logs in from one of a few nearby cities in Canada, always uses Chrome on a device running Mac OS X, and works typical business hours. Then, when there are new logins, the algorithm calculates UEBA features by determining how anomalous they are from that learned behavior. It also identifies any spikes in activity over the prior 24 hours.
While other, simpler detections might also catch these malicious logins, as discussed above, such detections also tend to be noisy and generate excessive false positives. For example, the first malicious login in this example was from Romania. I’ve travelled to Romania on vacation — without knowledge of the future logins from around the world, how can we know whether that login reflects legitimate travel as opposed to malicious behavior? A simpler detection might trigger on the unusual location alone. However, our detection considers not only the location likelihood for the user, but also whether colleagues have logged in from there, the fact that the login came at an unusual time of day, and that the login reported an unknown browser and operating system. The result is more effective detections and fewer FPs.
To see our high-quality alerts in action, you can request a demo of our service here. Also, check out this Azure Active Directory security operations guide as a reference.
