From 86246f940261addc0e21fef510aea14eed9ea46b Mon Sep 17 00:00:00 2001 From: Ian Hellen Date: Fri, 3 Feb 2023 19:03:36 -0800 Subject: [PATCH 1/2] Initial version of account evaluation notebook --- AccountSignInEvaluation.ipynb | 2809 +++++++++++++++++++++++++++++++++ 1 file changed, 2809 insertions(+) create mode 100644 AccountSignInEvaluation.ipynb diff --git a/AccountSignInEvaluation.ipynb b/AccountSignInEvaluation.ipynb new file mode 100644 index 00000000..cd5bb661 --- /dev/null +++ b/AccountSignInEvaluation.ipynb @@ -0,0 +1,2809 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Unusual Account Activity\n", + "
\n", + "  Notebook Details...\n", + "\n", + " **Notebook Version:** 2.0
\n", + " **Python Version:** Python 3.8+
\n", + " **Required Packages**: msticpy, msticnb
\n", + "\n", + " **Data Sources Required**:\n", + " - Sentinel - SecurityAlert, SecurityEvent, HuntingBookmark, Syslog, AAD SigninLogs, AzureActivity, OfficeActivity, ThreatIndicator\n", + " - (Optional) - VirusTotal, AlienVault OTX, IBM XForce, Open Page Rank, (all require accounts and API keys)\n", + "
\n", + "\n", + "## TBD" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "toc": true + }, + "source": [ + "\n", + "\n", + " \n", + " \n", + " \n", + "

Contents

\n", + "
\n", + " \n", + "
\n", + " \n", + " \n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Hunting Hypothesis\n", + "TBD\n", + "\n", + "\n", + "Flow:\n", + "- Query - risk-flagged sign-ins\n", + "- Add supplemental queries\n", + "- Query alerts for related accounts\n", + "- TI lookup for source IP (other?)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "# Notebook initialization\n", + "This should complete without errors. If you encounter errors or warnings look at the following notebooks:\n", + "\n", + "- Getting Started Notebook\n", + "- [TroubleShootingNotebooks](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/TroubleShootingNotebooks.ipynb)\n", + "- [ConfiguringNotebookEnvironment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)\n", + "\n", + "
\n", + "  Details...\n", + "The next cell:\n", + "- Checks for the correct Python version\n", + "- Checks versions and optionally installs required packages\n", + "- Imports the required packages into the notebook\n", + "- Sets a number of configuration options.\n", + "\n", + "If you are running in the Azure Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:\n", + "- [Getting Started](./A Getting Started Guide For Azure Sentinel ML Notebooks.ipynb)\n", + "- [Run TroubleShootingNotebooks](./TroubleShootingNotebooks.ipynb)\n", + "- [Run ConfiguringNotebookEnvironment](./ConfiguringNotebookEnvironment.ipynb)\n", + "\n", + "You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup. \n", + "There are more details about this in the `ConfiguringNotebookEnvironment` notebook and in these documents:\n", + "- [msticpy configuration](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)\n", + "- [Threat intelligence provider configuration](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file)\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "This product includes GeoLite2 data created by MaxMind, available from\n", + "https://www.maxmind.com.\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "Attempting connection to Key Vault using cli credentials..." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "done
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + "This library uses services provided by ipstack.\n", + "https://ipstack.com" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from datetime import datetime, timedelta, timezone\n", + "\n", + "REQ_PYTHON_VER = \"3.8\"\n", + "REQ_MSTICPY_VER = \"2.3.0\"\n", + "\n", + "# %pip install --upgrade msticpy\n", + "\n", + "import msticpy as mp\n", + "mp.init_notebook()\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "tags": [ + "parameters" + ] + }, + "outputs": [], + "source": [ + "# papermill default parameters\n", + "ws_name = \"Default\"\n", + "end = datetime.now(timezone.utc)\n", + "start = end - timedelta(days=2)\n", + "baseline_period = 28\n", + "run_date = end" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Get Workspace and Authenticate\n", + "\n", + "
\n", + " Authentication help...\n", + " If you want to use a workspace other than one you have defined in your
\n", + "msticpyconfig.yaml create a connection string with your AAD TENANT_ID and
\n", + "your WORKSPACE_ID (these should both be quoted UUID strings).\n", + "\n", + "```python\n", + " workspace_cs = \"loganalytics://code().tenant('TENANT_ID').workspace('WORKSPACE_ID')\"\n", + "```\n", + "e.g.\n", + "```python\n", + " workspace_cs = \"loganalytics://code().tenant('c3de0f06-dcb8-40fb-9d1a-b62faea29d9d').workspace('c62d3dc5-11e6-4e29-aa67-eac88d5e6cf6')\"\n", + "```\n", + "Then in the Authentication cell replace\n", + "the call to `qry_prov.connect` with the following:\n", + "```python\n", + " qry_prov.connect(connect_str=workspace_cs)\n", + "```\n", + "The cell should now look like this:\n", + "\n", + "```python\n", + "...\n", + " # Authentication\n", + " qry_prov = QueryProvider(data_environment=\"MSSentinel\")\n", + " qry_prov.connect(connect_str=workspace_cs)\n", + "...\n", + "```\n", + "\n", + "On successful authentication you should see a ```popup schema``` button.\n", + "To find your Workspace Id go to [Log Analytics](https://ms.portal.azure.com/#blade/HubsExtension/Resources/resourceType/Microsoft.OperationalInsights%2Fworkspaces). Look at the workspace properties to find the ID.\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Configured workspaces: ASIHuntOMSWorkspaceV4, CCIS, Centrica, CyberSecuritySoc, Default, GovCyberSecuritySOC, NationalGrid, RedmondSentinelDemoEnvironment\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "0576052186894a1a8969969bd4a8eb9f", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Combobox(value='Default', description='Workspace Name', options=('ASIHuntOMSWorkspaceV4', 'CCIS', 'Centrica', …" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "print(\"Configured workspaces: \", \", \".join(msticpy.settings.get_config(\"AzureSentinel.Workspaces\").keys()))\n", + "import ipywidgets as widgets\n", + "ws_param = widgets.Combobox(\n", + " description=\"Workspace Name\",\n", + " value=ws_name,\n", + " options=list(msticpy.settings.get_config(\"AzureSentinel.Workspaces\").keys())\n", + ")\n", + "ws_param" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Connecting... connected\n" + ] + }, + { + "data": { + "text/html": [ + "

" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "

Confirm time range to search

" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "5421cbd9aa08477fa1502bb6201aa14a", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "VBox(children=(HTML(value='

Set query time boundaries

'), HBox(children=(DatePicker(value=datetime.date…" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from msticpy.common.timespan import TimeSpan\n", + "from msticpy.context.azure import MicrosoftSentinel\n", + "\n", + "# Authentication\n", + "qry_prov = mp.QueryProvider(data_environment=\"MSSentinel\")\n", + "qry_prov.connect(workspace=ws_param.value)\n", + "\n", + "sentinel = MicrosoftSentinel(workspace=ws_param.value, connect=True)\n", + "\n", + "nb_timespan = TimeSpan(start, end)\n", + "qry_prov.query_time.timespan = nb_timespan\n", + "md(\"
\")\n", + "md(\"Confirm time range to search\", \"bold\")\n", + "qry_prov.query_time" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Notebook Logic\n", + "\n", + "{period} = query time period\n", + "\n", + "{baseline} = {period}.start - 28 days ... {period}.start\n", + "\n", + "1. Find users with high risk and unmitigated signin for {period}\n", + "2. Find users with high risk signins for {baseline}\n", + "3. Divide 1 into:\n", + " a. Users with on-going high risk - for triage\n", + " b. Users with new high risk status\n", + "4. For users in 3.a, check:\n", + " - Azure activity - any activity types in {period} not in baseline {baseline}\n", + " - Azure audit - any activity types in {period} not in baseline {baseline}\n", + "\n", + "Output (dynamic summary):\n", + "- List of ongoing high risk users\n", + "- New high risk users:\n", + " - Signin types and locations\n", + " - Novel Azure Activity and Audit types" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "metadata": {}, + "outputs": [], + "source": [ + "import re\n", + "import urllib\n", + "from collections import namedtuple, defaultdict\n", + "from datetime import datetime, timedelta, timezone\n", + "from typing import Dict, NamedTuple, Optional\n", + "\n", + "import httpx\n", + "import pandas as pd\n", + "import yaml\n", + "from tqdm.auto import tqdm\n", + "\n", + "from msticpy.context.azure.sentinel_dynamic_summary import DynamicSummary, DynamicSummaryItem\n", + "\n", + "\n", + "# Summary report classes\n", + "class SummaryItem(NamedTuple):\n", + " \"\"\"Data report collection for summary.\"\"\"\n", + " key: str\n", + " data: pd.DataFrame\n", + " properties: Dict[str, Any]\n", + "\n", + "\n", + "class SummaryReport:\n", + " \"\"\"Class to hold summary reports during exec of notebook.\"\"\"\n", + " def __init__(self):\n", + " self._summary_reports: Dict[str, Dict[str, SummaryItem]] = defaultdict(dict)\n", + "\n", + " def add_summary_data(self, data: pd.DataFrame, user_column: str, section: str, **kwargs):\n", + " \"\"\"Add data for users to the summary report\"\"\"\n", + " for user, user_data in data.groupby(user_column):\n", + " summary = SummaryItem(\n", + " key=user,\n", + " data=user_data,\n", + " properties=kwargs\n", + " )\n", + " self._summary_reports[user.casefold()][section] = summary\n", + "\n", + " @property\n", + " def users(self):\n", + " return sorted(self._summary_reports)\n", + "\n", + " @property\n", + " def report_types(self):\n", + " return sorted({\n", + " report for user_reports in self._summary_reports.values()\n", + " for report in user_reports\n", + " })\n", + "\n", + "\n", + "summary_report = SummaryReport()\n", + "\n", + "\n", + "# DF display function\n", + "def df_caption(data: pd.DataFrame, caption: str):\n", + " \"\"\"Display dataframe with a caption.\"\"\"\n", + " caption_css = \"; \".join([\n", + " \"caption-side: top\",\n", + " \"text-align: left\",\n", + " \"font-size: 15pt\",\n", + " \"font-weight: bold\",\n", + " \"padding: 5pt\",\n", + " ])\n", + " display(\n", + " data.style.set_caption(f\"{caption}\").set_table_styles(\n", + " [\n", + " {\n", + " \"selector\": \"caption\",\n", + " \"props\": caption_css,\n", + " }\n", + " ]\n", + " )\n", + " )\n", + "\n", + "\n", + "def get_user_param(data: pd.DataFrame) -> str:\n", + " \"\"\"Return user names from DataFrame as comma-sep string.\"\"\"\n", + " return \",\".join([\n", + " f\"'{user}'\" for user\n", + " in data.UserPrincipalName.values\n", + " ])\n", + "\n", + "\n", + "# update any changes to start/end datetimes\n", + "start = qry_prov.query_time.start\n", + "end = qry_prov.query_time.end" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Get risk-flagged sign-ins\n", + "\n", + "This query retrieves user signins that have been flagged by Azure Identity Protection\n", + "as at risk. See [Azure Identity Protection](https://learn.microsoft.com/azure/active-directory/identity-protection/overview-identity-protection)\n", + "for more background." + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Unmitigated risk users
 UserPrincipalName
1tamuto@seccxpninja.onmicrosoft.com
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Mitigated risk users
 UserPrincipalName
0pdemo@seccxpninja.onmicrosoft.com
2dwilliams@seccxp.ninja
3adm_pwatkins@seccxpninja.onmicrosoft.com
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "signing_risk_query = \"\"\"\n", + "SigninLogs\n", + "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", + "| where RiskState != \"none\"\n", + "| project UserPrincipalName, ResultDescription, RiskState, RiskDetail, RiskEventTypes,\n", + " RiskEventTypes_V2, RiskLevelAggregated, RiskLevelDuringSignIn, IPAddress\n", + "| extend SigninRisk = case(\n", + " RiskLevelDuringSignIn == \"high\", 5,\n", + " RiskLevelDuringSignIn == \"medium\", 3,\n", + " RiskLevelDuringSignIn == \"low\", 1,\n", + " 0\n", + " ),\n", + " AggRisk = case(\n", + " RiskLevelAggregated == \"high\", 5,\n", + " RiskLevelAggregated == \"medium\", 3,\n", + " RiskLevelAggregated == \"low\", 1,\n", + " 0\n", + " )\n", + "| extend RiskEventDyn = parse_json(RiskEventTypes), RiskEventV2Dyn = parse_json(RiskEventTypes_V2)\n", + "| mv-expand RiskEventDyn, RiskEventV2Dyn\n", + "| summarize SignIns=count(AggRisk), MeanAggRisk=avg(AggRisk), MeanSigninRisk=avg(SigninRisk), \n", + " RiskStates=make_set(RiskState), RiskEvents=make_set(RiskEventDyn), RiskEventsV2=make_set(RiskEventV2Dyn),\n", + " SourceIPs=make_set(IPAddress)\n", + " by UserPrincipalName\n", + "| order by MeanAggRisk, MeanSigninRisk asc nulls last\n", + "\"\"\"\n", + "\n", + "# run the query\n", + "signin_risk_users_df = qry_prov.exec_query(\n", + " signing_risk_query.format(start=start, end=end)\n", + ")\n", + "# expand RiskStates (list)\n", + "risk_states_df = signin_risk_users_df.explode(\"RiskStates\")\n", + "# Extract list of users where risk was mitigated \n", + "safe_users_df = risk_states_df[risk_states_df[\"RiskStates\"].isin([\"remediated\", \"confirmedSafe\"])].UserPrincipalName.drop_duplicates()\n", + "\n", + "# Separate unmitigated from mitigated risk users\n", + "risk_users_df = signin_risk_users_df[~signin_risk_users_df[\"UserPrincipalName\"].isin(safe_users_df)]\n", + "mitigated_users_df = signin_risk_users_df[signin_risk_users_df[\"UserPrincipalName\"].isin(safe_users_df)]\n", + "\n", + "df_caption(risk_users_df[[\"UserPrincipalName\"]], \"Unmitigated risk users\")\n", + "df_caption(mitigated_users_df[[\"UserPrincipalName\"]], \"Mitigated risk users\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Retrieve the historical risk level for previous N days\n", + "\n", + "This is used to distinguish accounts that have a new \"At Risk\"\n", + "designation from those accounts that have a history of risk signins.\n", + "\n", + "> Note: \"N\" is the `baseline_period` parameter for the notebook - default is 28 days" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Signing Summary for users with unmitigated risk" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Sign-in risk summary - unmitigated
 UserPrincipalNameSignInsMeanAggRiskMeanSigninRiskRiskStatesRiskEventsRiskEventsV2SourceIPsRiskHistory
1tamuto@seccxpninja.onmicrosoft.com31.0000001.000000['atRisk']['unfamiliarFeatures', 'unlikelyTravel']['unfamiliarFeatures', 'unlikelyTravel']['20.25.98.192']Existing
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "_AADSIL_DISPLAY_COLUMNS = [\n", + " 'TimeGenerated', 'ResultType', 'ResultDescription', 'UserPrincipalName', 'UserId',\n", + " 'Location', 'IPAddress', 'AppDisplayName', 'ClientAppUsed', 'AppId',\n", + " 'AuthenticationDetails', 'AuthenticationMethodsUsed',\n", + " 'RiskEventTypes', 'RiskEventTypes_V2', 'RiskLevelAggregated',\n", + " 'RiskLevelDuringSignIn', 'RiskState', 'ResourceDisplayName',\n", + " 'LocationDetails', 'MfaDetail', 'NetworkLocationDetails',\n", + " 'UserAgent', 'UserDisplayName', 'UserType', 'IPAddressFromResourceProvider',\n", + " 'ResourceTenantId', 'HomeTenantId', 'AutonomousSystemNumber', 'Type'\n", + "]\n", + "\n", + "\n", + "# Function to summarize the history data\n", + "def weekly_signin_summary(data) -> pd.DataFrame:\n", + " \"\"\"Create signin summary from historical data.\"\"\"\n", + " return (\n", + " data\n", + " [_AADSIL_DISPLAY_COLUMNS]\n", + " .explode([\"RiskEventTypes\"])\n", + " .groupby([\"UserPrincipalName\", pd.Grouper(key=\"TimeGenerated\", freq=\"W\")])\n", + " .agg(\n", + " LoginCount=pd.NamedAgg(\"ResultType\", \"count\"),\n", + " ResultTypes=pd.NamedAgg(\"ResultType\", \"unique\"),\n", + " RiskEventTypes=pd.NamedAgg(\"RiskEventTypes\", \"unique\"),\n", + " RiskLevels=pd.NamedAgg(\"RiskLevelAggregated\", \"unique\"),\n", + " RiskLevelSignins=pd.NamedAgg(\"RiskLevelDuringSignIn\", \"unique\"),\n", + " IPs=pd.NamedAgg(\"IPAddress\", \"nunique\"),\n", + " Locations=pd.NamedAgg(\"Location\", \"nunique\"),\n", + " Apps=pd.NamedAgg(\"AppDisplayName\", \"nunique\"),\n", + " UserAgents=pd.NamedAgg(\"UserAgent\", \"nunique\"),\n", + " StartDate=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", + " EndDate=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", + " )\n", + " .sort_index()\n", + " )\n", + "\n", + "\n", + "# Get historical risk level for previous {period} days\n", + "risk_hist_query = \"\"\"\n", + "let q_end = datetime({start});\n", + "let q_start = datetime_add(\"day\", -{period}, q_end);\n", + "SigninLogs\n", + "| where TimeGenerated between (q_start .. q_end)\n", + "| where RiskState != \"none\"\n", + "| where UserPrincipalName in ({users})\n", + "| extend RiskEventTypes = parse_json(RiskEventTypes), RiskEventTypes_V2 = parse_json(RiskEventTypes_V2)\n", + "\"\"\"\n", + "\n", + "# Unmitigated risk users\n", + "risk_user_hist_df = qry_prov.exec_query(\n", + " risk_hist_query.format(\n", + " users=get_user_param(risk_users_df),\n", + " start=start,\n", + " period=baseline_period,\n", + " )\n", + ")\n", + "\n", + "risk_users_history = weekly_signin_summary(risk_user_hist_df).reset_index()\n", + "\n", + "# Isolate users that have no history of risk in previous period\n", + "users_with_past_risk_criteria = risk_users_df.UserPrincipalName.isin(risk_user_hist_df.UserPrincipalName.unique())\n", + "risk_users_df = risk_users_df.copy()\n", + "risk_users_df.loc[~users_with_past_risk_criteria, \"RiskHistory\"] = \"New\"\n", + "risk_users_df.loc[users_with_past_risk_criteria, \"RiskHistory\"] = \"Existing\"\n", + "\n", + "summary_report.add_summary_data(\n", + " data=risk_users_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Risk Users Summary\",\n", + ")\n", + "summary_report.add_summary_data(\n", + " data=risk_users_history,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Risk Users History\",\n", + ")\n", + "\n", + "df_caption(risk_users_df, \"Sign-in risk summary - unmitigated\")" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
UserPrincipalNameTimeGeneratedLoginCountResultTypesRiskEventTypesRiskLevelsRiskLevelSigninsIPsLocationsAppsUserAgentsStartDateEndDate
0tamuto@seccxpninja.onmicrosoft.com2023-01-22 00:00:00+00:001[0][mcasImpossibleTravel][low][none]11112023-01-18 01:27:19.258557600+00:002023-01-18 01:27:19.258557600+00:00
\n", + "
" + ], + "text/plain": [ + " UserPrincipalName TimeGenerated LoginCount \\\n", + "0 tamuto@seccxpninja.onmicrosoft.com 2023-01-22 00:00:00+00:00 1 \n", + "\n", + " ResultTypes RiskEventTypes RiskLevels RiskLevelSignins IPs \\\n", + "0 [0] [mcasImpossibleTravel] [low] [none] 1 \n", + "\n", + " Locations Apps UserAgents StartDate \\\n", + "0 1 1 1 2023-01-18 01:27:19.258557600+00:00 \n", + "\n", + " EndDate \n", + "0 2023-01-18 01:27:19.258557600+00:00 " + ] + }, + "execution_count": 54, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "not_mit_risk_history" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Signing Summary for users with mitigated risk\n", + "### [info only]" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Sign-in risk summary - mitigated
 UserPrincipalNameSignInsMeanAggRiskMeanSigninRiskRiskStatesRiskEventsRiskEventsV2SourceIPsRiskHistory
0pdemo@seccxpninja.onmicrosoft.com2681.6194034.276119['atRisk', 'dismissed', 'confirmedSafe']['unfamiliarFeatures', 'unlikelyTravel']['unfamiliarFeatures', 'unlikelyTravel']['84.59.133.96', '49.207.205.157', '182.48.225.204', '202.171.187.206', '83.6.102.205', '94.239.55.19', '51.142.235.76', '110.49.50.142', '49.37.163.128', '165.225.120.88', '89.211.239.104', '94.245.87.14', '50.208.71.66', '140.186.246.113', '201.191.218.23', '190.104.120.0', '109.48.220.202', '73.43.36.19', '76.67.108.134', '148.64.97.101', '51.142.111.1', '37.186.51.21', '76.184.244.1', '172.13.62.40', '107.129.128.34', '99.248.154.225', '87.187.23.105', '108.34.158.176', '80.187.114.167', '157.49.156.108', '194.107.2.82', '167.220.24.243', '187.56.121.165', '73.25.210.7', '90.146.97.205', '195.97.138.43', '14.187.179.30', '114.79.170.132', '147.161.199.96', '8.23.71.2', '20.122.92.1', '189.249.64.2', '168.149.166.14', '47.231.129.2', '212.180.224.82', '107.11.97.170', '96.234.155.228', '147.235.216.117', '39.9.193.150', '109.147.153.136', '168.149.166.78', '70.164.213.113', '147.161.199.101', '24.98.48.107', '180.218.164.251', '94.174.54.38', '79.107.37.34', '208.104.177.188', '24.15.125.185', '104.219.136.49', '159.196.229.180', '209.65.150.148', '61.68.47.232']Existing
2dwilliams@seccxp.ninja20.5000003.000000['atRisk', 'confirmedSafe']['unfamiliarFeatures']['unfamiliarFeatures']['20.227.3.22']New
3adm_pwatkins@seccxpninja.onmicrosoft.com240.0000004.833333['remediated']['anonymizedIPAddress']['anonymizedIPAddress']['185.220.100.247', '192.42.116.216']Existing
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# History of mitigated risk users\n", + "mit_risk_user_hist_df = qry_prov.exec_query(\n", + " risk_hist_query.format(\n", + " users=get_user_param(mitigated_users_df),\n", + " start=start,\n", + " period=baseline_period\n", + " )\n", + ")\n", + "\n", + "\n", + "# Isolate users that have no history of risk in previous period\n", + "users_with_past_risk_criteria = mitigated_users_df.UserPrincipalName.isin(mit_risk_user_hist_df.UserPrincipalName.unique())\n", + "mitigated_users_df = mitigated_users_df.copy()\n", + "mitigated_users_df.loc[~users_with_past_risk_criteria, \"RiskHistory\"] = \"New\"\n", + "mitigated_users_df.loc[users_with_past_risk_criteria, \"RiskHistory\"] = \"Existing\"\n", + "mitigated_users_df\n", + "df_caption(mitigated_users_df, \"Sign-in risk summary - mitigated\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Retrieve and Run UEBA hunting queries on risk-flagged users\n", + "\n", + "> UEBA = User Entity Behavior Analytics\n", + "\n", + "The next cell retrieves the current UEBA hunting\n", + "queries and runs them against the risk-flagged users.\n", + "\n", + "For more information see [Microsoft Sentinel UEBA](https://learn.microsoft.com/azure/sentinel/identify-threats-with-entity-behavior-analytics)" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 21/21 [00:04<00:00, 5.14it/s]\n" + ] + }, + { + "data": { + "text/html": [ + "
Anomalies on users tagged as VIPhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/anomaliesOnVIPUsers.yaml
Anomalous AAD Account Manipulationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20AAD%20Account%20Manipulation.yaml
Anomalous AAD Account Creationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Account%20Creation.yaml
Anomalous Activity Role Assignmenthttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Activity%20Role%20Assignment.yaml
Anomalous Code Executionhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Code%20Execution.yaml
Anomalous Data Accesshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Data%20Access.yaml
Anomalous Defensive Mechanism Modificationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Defensive%20Mechanism%20Modification.yaml
Anomalous Failed Logonhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Failed%20Logon.yaml
Anomalous Geo Location Logonhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Geo%20Location%20Logon.yaml
Anomalous Login to Deviceshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Login%20to%20Devices.yaml
Anomalous Password Resethttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Password%20Reset.yaml
Anomalous RDP Activityhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20RDP%20Activity.yaml
Anomalous Resource Accesshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Resource%20Access.yaml
Anomalous Role Assignmenthttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Role%20Assignment.yaml
Anomalous Sign-in Activityhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Sign-in%20Activity.yaml
Anomalous action performed in tenant by privileged userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/anomalousActionInTenant.yaml
Dormant account activity from uncommon countryhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/dormantAccountActivityFromUncommonCountry.yaml
Anomalous connection from highly privileged userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/firstConnectionFromGroup.yaml
Anomalous login activity originated from Botnet, Tor proxy or C2https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/loginActivityFromBotnet.yaml
New account added to admin grouphttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/newAccountAddedToAdminGroup.yaml
Anomalous update Key Vault activity by high blast radius userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/updateKeyVaultActivity.yaml
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Hunting Queries\n", + "_SENTINEL_REPO = \"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master\"\n", + "_SI_LOG_ROOT = f\"{_SENTINEL_REPO}/Hunting%20Queries/SigninLogs\"\n", + "_GEN_HUNTING_QRY = [\n", + " # \"AnomalousUserAppSigninLocationIncreaseDetail.yaml\",\n", + " # \"LegacyAuthAttempt.yaml\",\n", + " # \"Signins-From-VPS-Providers.yaml\",\n", + " # \"UserAccountsMeasurableincreaseofsuccessfulsignins.yaml\",\n", + " # \"riskSignInWithNewMFAMethod.yaml\",\n", + " # \"signinBurstFromMultipleLocations.yaml\",\n", + "]\n", + "\n", + "# UEBA Hunting Queries\n", + "_UEBA_HQ_ROOT = f\"{_SENTINEL_REPO}/Solutions/UEBA%20Essentials/Hunting%20Queries\"\n", + "_UEBA_HUNTING_QRY = [\n", + " \"anomaliesOnVIPUsers.yaml\",\n", + " \"Anomalous AAD Account Manipulation.yaml\",\n", + " \"Anomalous Account Creation.yaml\",\n", + " \"Anomalous Activity Role Assignment.yaml\",\n", + " \"Anomalous Code Execution.yaml\",\n", + " \"Anomalous Data Access.yaml\",\n", + " \"Anomalous Defensive Mechanism Modification.yaml\",\n", + " \"Anomalous Failed Logon.yaml\",\n", + " \"Anomalous Geo Location Logon.yaml\",\n", + " \"Anomalous Login to Devices.yaml\",\n", + " \"Anomalous Password Reset.yaml\",\n", + " \"Anomalous RDP Activity.yaml\",\n", + " \"Anomalous Resource Access.yaml\",\n", + " \"Anomalous Role Assignment.yaml\",\n", + " \"Anomalous Sign-in Activity.yaml\",\n", + " \"anomalousActionInTenant.yaml\",\n", + " \"dormantAccountActivityFromUncommonCountry.yaml\",\n", + " \"firstConnectionFromGroup.yaml\",\n", + " \"loginActivityFromBotnet.yaml\",\n", + " \"newAccountAddedToAdminGroup.yaml\",\n", + " # \"terminatedEmployeeAccessHVA.yaml\",\n", + " # \"terminatedEmployeeActivity.yaml\",\n", + " \"updateKeyVaultActivity.yaml\",\n", + "]\n", + "\n", + "ALL_QUERIES = {qry: _SI_LOG_ROOT for qry in _GEN_HUNTING_QRY}\n", + "ALL_QUERIES.update({qry: _UEBA_HQ_ROOT for qry in _UEBA_HUNTING_QRY})\n", + "\n", + "TIME_TOKEN = re.compile(r\"(\\{\\{StartTimeISO\\}\\}|\\{\\{EndTimeISO\\}\\})\")\n", + "_LEFT_BRACE = r\"[^{](\\{)[^{]\"\n", + "_RIGHT_BRACE = r\"[^}](\\})[^}]\"\n", + "_LB_TOKEN = \"%%~[~%%\"\n", + "_RB_TOKEN = \"%%~]~%%\"\n", + "\n", + "\n", + "def replace_time_params(query):\n", + " repl_query = re.sub(_LEFT_BRACE, _LB_TOKEN, query)\n", + " repl_query = re.sub(_RIGHT_BRACE, _RB_TOKEN, repl_query)\n", + " repl_query = repl_query.replace(\"{{StartTimeISO}}\", \"{start}\").replace(\"{{EndTimeISO}}\", \"{end}\")\n", + " return repl_query.replace(_LB_TOKEN, \"{{\").replace(_RB_TOKEN, \"}}\")\n", + "\n", + "\n", + "QueryProps = namedtuple(\"QueryProps\", \"name, query, req_time, description, url, raw_query\")\n", + "\n", + "\n", + "def fetch_queries(query_dict: Dict[str, str], verbose: bool = False) -> Dict[str, QueryProps]:\n", + " \"\"\"Fetch queries from Sentinel GitHub repo.\"\"\"\n", + " discover_queries: Dict[str, QueryProps] = {}\n", + " error_queries: Dict[str, str] = {}\n", + " for query, path in tqdm(query_dict.items()):\n", + " q_path = f\"{path}/{urllib.parse.quote(query)}\"\n", + " resp = httpx.get(q_path)\n", + " if resp.status_code != 200:\n", + " print(f\"invalid URL {path}\")\n", + " continue\n", + " try:\n", + " q_dict = yaml.safe_load(resp.content)\n", + " except yaml.scanner.ScannerError as err:\n", + " print(f\"could not parse query {query} at {q_path}\")\n", + " error_queries[query] = resp.content\n", + " continue\n", + "\n", + " query_text = q_dict.get(\"query\")\n", + " req_time = False\n", + " if re.search(TIME_TOKEN, query_text):\n", + " query_text = replace_time_params(query_text)\n", + " req_time = True\n", + "\n", + " if \"UEBA\" in path:\n", + " query_text = add_ueba_time_params(query_text)\n", + " if verbose:\n", + " print(f\"Query {query}, {q_dict['name']}, req time: {req_time}\")\n", + " discover_queries[query] = QueryProps(\n", + " name=q_dict.get(\"name\"),\n", + " query=query_text,\n", + " req_time=req_time,\n", + " description=q_dict.get(\"description\"),\n", + " url=q_path,\n", + " raw_query=q_dict.get(\"query\"),\n", + " )\n", + " return discover_queries\n", + "\n", + "\n", + "PRIM_TABLE_EXP = r\"(?P^|\\n)(?P(BehaviorAnalytics|AuditLogs|IdentityInfo|SigninLogs))(?=[\\s\\n\\)\\|])\"\n", + "PRIM_TABLE_REPL = r\"\\g\\g
\\n| where TimeGenerated > datetime({start})\"\n", + "JOIN_TABLE_EXP = r\"(?P\\|\\s+join[^(]*\\(\\s*[^\\s]+)(?=[\\s\\)\\|])\"\n", + "JOIN_TABLE_REPL = r\"\\g\\n| where TimeGenerated > datetime({start})\"\n", + "\n", + "\n", + "def add_ueba_time_params(query):\n", + " if isinstance(query, tuple):\n", + " if query.req_time:\n", + " return query.query\n", + " query = query.query\n", + " return re.sub(\n", + " JOIN_TABLE_EXP,\n", + " JOIN_TABLE_REPL,\n", + " re.sub(PRIM_TABLE_EXP, PRIM_TABLE_REPL, query)\n", + " )\n", + "\n", + "\n", + "def display_query_table(queries):\n", + " ht_table = \"
{rows}
\"\n", + " rows = [f\"{q.name}{q.url}\"\n", + " for q in queries.values()]\n", + " from IPython.display import HTML\n", + " display(HTML(ht_table.format(rows=\"\".join(rows))))\n", + "\n", + "\n", + "hunting_queries = fetch_queries(ALL_QUERIES)\n", + "\n", + "display_query_table(hunting_queries).head()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Browser for UEBA queries - not used in notebook" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": {}, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "7e8aa5f430fc41e491630200b6ef68dc", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "VBox(children=(Select(description='Query', layout=Layout(height='200px', padding='5pt', width='50%'), options=…" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import ipywidgets as widgets\n", + "import difflib\n", + "\n", + "def browse_queries(queries: Dict[str, QueryProps]):\n", + " \"\"\"\n", + " Browse Hunting queries.\n", + " \n", + " Notes\n", + " -----\n", + " T\n", + " \"\"\"\n", + " select_query = widgets.Select(\n", + " description=\"Query\",\n", + " options=[(qry.name, idx) for idx, qry in queries.items()],\n", + " layout=widgets.Layout(height=\"200px\", width=\"50%\", padding=\"5pt\")\n", + " )\n", + " layout_query = lambda x, y: widgets.Layout(height=x, width=y, padding=\"5pt\")\n", + " layout_w = lambda x: widgets.Layout(width=x, padding=\"5pt\")\n", + " qry_view = widgets.Textarea(layout=layout_query(\"200px\", \"95%\"))\n", + " qry_view_repl = widgets.Textarea(layout=layout_query(\"200px\", \"95%\"))\n", + " qry_view_diff = widgets.Textarea(layout=layout_query(\"150px\", \"50%\"))\n", + " qry_file = widgets.Label(layout=layout_w(\"60%\"))\n", + " orig_lbl = widgets.Label(value=\"Original query\", layout=layout_w(\"60%\"))\n", + " mod_lbl = widgets.Label(value=\"Modified query\", layout=layout_w(\"60%\"))\n", + " vbox = widgets.VBox([\n", + " select_query,\n", + " qry_file,\n", + " widgets.HBox([\n", + " widgets.VBox([orig_lbl, qry_view], layout=layout_query(\"250px\", \"45%\")),\n", + " widgets.VBox([mod_lbl, qry_view_repl], layout=layout_query(\"250px\", \"45%\"))\n", + " ]),\n", + " qry_view_diff\n", + " ])\n", + "\n", + " def update_query(change):\n", + " query = queries[select_query.value]\n", + " qry_file.value = query.url\n", + " qry_view.value = query.raw_query\n", + " qry_view_repl.value = query.query\n", + " qry_view_diff.value = \"\\n\".join(difflib.unified_diff(qry_view.value.splitlines(), qry_view_repl.value.splitlines()))\n", + "\n", + " select_query.observe(update_query, names=\"value\")\n", + " update_query(None)\n", + " return vbox\n", + "\n", + "# Uncomment the follow line to browse the hunting queries\n", + "# browse_queries(hunting_queries)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Run Hunting queries for time range on risky accounts" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 21/21 [00:27<00:00, 1.33s/it]\n" + ] + } + ], + "source": [ + "\n", + "def run_ueba_queries(queries, start, end) -> pd.DataFrame:\n", + " dfs = []\n", + " query_params = {\"end\": end, \"start\": start}\n", + " for query in tqdm(queries.values()):\n", + " if \"UEBA\" not in query.url:\n", + " continue\n", + " try:\n", + " repl_query = query.query\n", + " if \"{start}\" in repl_query or \"{end}\" in repl_query:\n", + " try:\n", + " repl_query = repl_query.format(**query_params)\n", + " except KeyError:\n", + " print(f\"Format error: {query.name}\")\n", + " result_df = qry_prov.exec_query(repl_query)\n", + " result_df[\"UEBAQuery\"] = query.name\n", + " dfs.append(result_df)\n", + " except Exception as err:\n", + " print(\"Exception:\", type(err), query.name)\n", + " return pd.concat(dfs)\n", + "\n", + "ueba_df = run_ueba_queries(hunting_queries, start=start, end=end)" + ] + }, + { + "cell_type": "code", + "execution_count": 62, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
UEBA entries for unmitigated risk users
  UEBAEventCountStartTimeEndTime
UserPrincipalNameUEBAQuery   
tamuto@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity842023-01-31 00:54:39+00:002023-02-01 13:04:04+00:00
Anomalous action performed in tenant by privileged user12023-02-01 06:00:31+00:002023-02-01 06:00:31+00:00
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "ueba_summary = (\n", + " ueba_df[ueba_df[\"UserPrincipalName\"].str.lower().isin(risk_users_df.UserPrincipalName)]\n", + " .groupby([\"UserPrincipalName\", \"UEBAQuery\"])\n", + " .agg(\n", + " UEBAEventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", + " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", + " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", + " )\n", + ")\n", + "summary_report.add_summary_data(\n", + " data=ueba_summary.reset_index(),\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"UEBA Summary\",\n", + ")\n", + "df_caption(\n", + " ueba_summary,\n", + " caption=\"UEBA entries for unmitigated risk users\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
UserPrincipalNameUEBAQueryUEBAEventCountStartTimeEndTime
0tamuto@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity842023-01-31 00:54:39+00:002023-02-01 13:04:04+00:00
1tamuto@seccxpninja.onmicrosoft.comAnomalous action performed in tenant by privileged user12023-02-01 06:00:31+00:002023-02-01 06:00:31+00:00
\n", + "
" + ], + "text/plain": [ + " UserPrincipalName \\\n", + "0 tamuto@seccxpninja.onmicrosoft.com \n", + "1 tamuto@seccxpninja.onmicrosoft.com \n", + "\n", + " UEBAQuery UEBAEventCount \\\n", + "0 Anomalous Sign-in Activity 84 \n", + "1 Anomalous action performed in tenant by privileged user 1 \n", + "\n", + " StartTime EndTime \n", + "0 2023-01-31 00:54:39+00:00 2023-02-01 13:04:04+00:00 \n", + "1 2023-02-01 06:00:31+00:00 2023-02-01 06:00:31+00:00 " + ] + }, + "execution_count": 61, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "ueba_summary.reset_index()" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
UEBA entries for mitigated risk users
  UEBAEventCountStartTimeEndTime
UserPrincipalNameUEBAQuery   
PDemo@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity10642023-01-31 06:41:25+00:002023-02-01 23:47:02+00:00
adm_pwatkins@seccxpninja.onmicrosoft.comAnomalies on users tagged as VIP332023-01-31 12:51:07+00:002023-01-31 13:17:32+00:00
Anomalous Sign-in Activity302023-01-31 12:59:44+00:002023-01-31 13:17:32+00:00
Anomalous login activity originated from Botnet, Tor proxy or C2252023-01-31 13:02:17+00:002023-01-31 13:17:32+00:00
dwilliams@seccxp.ninjaAnomalous Sign-in Activity32023-01-31 12:12:42+00:002023-02-01 00:53:23+00:00
pdemo@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity3692023-01-30 23:51:12+00:002023-01-31 16:27:02+00:00
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "df_caption(\n", + " ueba_df[ueba_df[\"UserPrincipalName\"].str.lower().isin(mitigated_users_df.UserPrincipalName)]\n", + " .groupby([\"UserPrincipalName\", \"UEBAQuery\"])\n", + " .agg(\n", + " UEBAEventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", + " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", + " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", + " ),\n", + " caption=\"UEBA entries for mitigated risk users\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Signin Summaries for prior week" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Sign-in summary for previous week
 ValuesNumUniqueValues
AttributeClientAppUserIPAddressLocationUserAgentClientAppUserIPAddressLocationUserAgent
UserPrincipalName        
tamuto@seccxpninja.onmicrosoft.com['Browser']['118.200.55.233' '20.25.98.192']['SG' 'US']['Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.70'\n", + " 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.61'\n", + " 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.70']1223
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "user_summary_query = \"\"\"\n", + "let si_history = SigninLogs\n", + "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", + "| where UserPrincipalName in~ ({users})\n", + "| summarize count() by UserPrincipalName, ResultType, RiskLevelAggregated, RiskLevelDuringSignIn, ClientAppUsed, UserAgent, IPAddress, Location;\n", + "si_history\n", + "| summarize OpCount=sum(count_) by UserPrincipalName, ClientAppUsed\n", + "| project UserPrincipalName, Attribute=\"ClientAppUser\", Value=ClientAppUsed, OpCount\n", + "| union ( \n", + "si_history\n", + "| summarize OpCount=sum(count_) by UserPrincipalName, IPAddress\n", + "| project UserPrincipalName, Attribute=\"IPAddress\", Value=IPAddress, OpCount\n", + ")\n", + "| union ( \n", + "si_history\n", + "| summarize OpCount=sum(count_) by UserPrincipalName, UserAgent\n", + "| project UserPrincipalName, Attribute=\"UserAgent\", Value=UserAgent, OpCount\n", + ")\n", + "| union ( \n", + "si_history\n", + "| summarize OpCount=sum(count_) by UserPrincipalName, Location\n", + "| project UserPrincipalName, Attribute=\"Location\", Value=Location, OpCount\n", + ")\n", + "\"\"\"\n", + "week_ago = (end - timedelta(7))\n", + "user_summary_df = qry_prov.exec_query(user_summary_query.format(\n", + " users=get_user_param(risk_users_df),\n", + " start=week_ago,\n", + " end=end\n", + "))\n", + "\n", + "summary_report.add_summary_data(\n", + " data=user_summary_df,\n", + " user_column=\"UserPrincipalName\",\n", + " report=\"Signin summary for previous week\"\n", + ")\n", + "df_caption(\n", + " user_summary_df.groupby([\"UserPrincipalName\", \"Attribute\"]).agg(\n", + " Values=pd.NamedAgg(\"Value\", \"unique\"),\n", + " NumUniqueValues=pd.NamedAgg(\"Value\", \"nunique\"),\n", + " OpCount=pd.NamedAgg(\"Value\", \"count\"),\n", + " )\n", + " .reset_index()\n", + " .pivot(index=['UserPrincipalName'], columns='Attribute', values=[\"Values\", \"NumUniqueValues\"]),\n", + " caption=\"Sign-in summary for previous week\"\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Related alerts\n", + "\n", + "## 1. Alerts that name the account explicitly" + ] + }, + { + "cell_type": "code", + "execution_count": 38, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 1/1 [00:02<00:00, 2.26s/it]\n" + ] + } + ], + "source": [ + "related_alerts_df = pd.concat([\n", + " (\n", + " qry_prov.SecurityAlert.list_related_alerts(account_name=acct)\n", + " .assign(UserPrincipalName=acct)\n", + " )\n", + " for acct in tqdm(risk_users_df.UserPrincipalName)\n", + "])\n", + "summary_report.add_summary_data(\n", + " data=related_alerts_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Related alerts for user\"\n", + ")\n", + "df_caption(related_alerts_df.drop(\n", + " columns=[\"Description\", \"RemediationSteps\", \"ExtendedProperties\"]),\n", + " caption=\"Related alerts for account\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Alerts related to signin-in IP addresses" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 3/3 [00:07<00:00, 2.35s/it]\n" + ] + } + ], + "source": [ + "related_alerts_ip_df = pd.concat([\n", + " (\n", + " qry_prov.SecurityAlert.list_alerts_for_ip(source_ip_list=ip_addr)\n", + " .assign(UserPrincipalName=acct, IPAddress=ip_addr)\n", + " )\n", + " for acct, ip_addr in tqdm(\n", + " risk_users_df.explode(\"SourceIPs\")[[\"UserPrincipalName\", \"SourceIPs\"]].apply(tuple, axis=1)\n", + " )\n", + "])\n", + "summary_report.add_summary_data(\n", + " data=related_alerts_ip_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Related alerts for user signin IP address\"\n", + ")\n", + "\n", + "df_caption(related_alerts_ip_df, caption=\"Related alerts for sign-in IP Address\")\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Threat Intelligence reports for sign-in IPs" + ] + }, + { + "cell_type": "code", + "execution_count": 82, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Observables processed: 100%|██████████| 6/6 [00:00<00:00, 600.07obs/s]\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Threat intel reports for risky sign-in IPs
 UserPrincipalNameSourceIPsQuerySubtypeResultDetailsRawResultReferenceStatusIocIocTypeSafeIocSeverityProvider
2tamuto@seccxpninja.onmicrosoft.com20.25.98.192NoneTrue{'summary': {'resolutions': 0, 'certificates': 0, 'malware_hashes': 0, 'projects': 0, 'articles': 30, 'total': 30, 'netblock': '20.0.0.0/11', 'os': 'n/a', 'asn': 'AS8075 - MICROSOFT-CORP-MSN-AS-BLOCK', 'hosting_provider': 'n/a', 'link': 'https://community.riskiq.com/search/20.25.98.192', 'links': {'resolutions': 'https://community.riskiq.com/search/20.25.98.192/resolutions', 'services': 'https://community.riskiq.com/search/20.25.98.192/services', 'certificates': 'https://community.riskiq.com/search/20.25.98.192/certificates', 'projects': 'https://community.riskiq.com/search/20.25.98.192/projects', 'articles': 'https://community.riskiq.com/research?query=20.25.98.192', 'trackers': 'https://community.riskiq.com/search/20.25.98.192/trackers', 'components': 'https://community.riskiq.com/search/20.25.98.192/components', 'host_pairs': 'https://community.riskiq.com/search/20.25.98.192/hostpairs', 'reverse_dns': 'https://community.riskiq.com/search/20.25.98.192/dns', 'cookies': 'https://community.riskiq.com/search/20.25.98.192/cookies', 'malware_hashes': 'https://community.riskiq.com/search/20.25.98.192/hashes'}, 'services': 0}, 'reputation': {'score': 4, 'classification': 'UNKNOWN', 'rules': []}}{'summary': {'resolutions': 0, 'certificates': 0, 'malware_hashes': 0, 'projects': 0, 'articles': 30, 'total': 30, 'netblock': '20.0.0.0/11', 'os': 'n/a', 'asn': 'AS8075 - MICROSOFT-CORP-MSN-AS-BLOCK', 'hosting_provider': 'n/a', 'link': 'https://community.riskiq.com/search/20.25.98.192', 'links': {'resolutions': 'https://community.riskiq.com/search/20.25.98.192/resolutions', 'services': 'https://community.riskiq.com/search/20.25.98.192/services', 'certificates': 'https://community.riskiq.com/search/20.25.98.192/certificates', 'projects': 'https://community.riskiq.com/search/20.25.98.192/projects', 'articles': 'https://community.riskiq.com/research?query=20.25.98.192', 'trackers': 'https://community.riskiq.com/search/20.25.98.192/trackers', 'components': 'https://community.riskiq.com/search/20.25.98.192/components', 'host_pairs': 'https://community.riskiq.com/search/20.25.98.192/hostpairs', 'reverse_dns': 'https://community.riskiq.com/search/20.25.98.192/dns', 'cookies': 'https://community.riskiq.com/search/20.25.98.192/cookies', 'malware_hashes': 'https://community.riskiq.com/search/20.25.98.192/hashes'}, 'services': 0}, 'reputation': {'score': 4, 'classification': 'UNKNOWN', 'rules': []}}https://community.riskiq.com020.25.98.192ipv420.25.98.192highRiskIQ
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# look up IP addresses - join UserPrincipalName from source DF to output\n", + "ti_user_ip = IpAddress.tilookup_ip(\n", + " risk_users_df.explode(\"SourceIPs\")[[\"UserPrincipalName\", \"SourceIPs\"]],\n", + " column=\"SourceIPs\",\n", + " join=\"left\"\n", + ").query(\"Severity != 'information'\")\n", + "\n", + "summary_report.add_summary_data(\n", + " data=ti_user_ip,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Threat intel reports for user sign-in IP address(es)\"\n", + ")\n", + "\n", + "df_caption(ti_user_ip, caption=\"Threat intel reports for risky sign-in IPs\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Unusual Azure Audit entries\n", + "\n", + "Look for operations in Azure audit for selected accounts\n", + "where account used operations type in the current time slot that\n", + "it had not used in the baseline period (default prior 30 days)" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Azure audit activity types not seen in baseline period.
 IdentityUserPrincipalNameOperationNameLoggedByServiceInitiatedByAdditionalDetailsTargetResources
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Azure Audit\n", + "# Find any operation types for current period that weren't seen for\n", + "# that user in previous baseline period\n", + "azure_audit_query = \"\"\"\n", + "let start = datetime(\"{start}\");\n", + "let end = datetime(\"{end}\");\n", + "let baseline_start = start - ({baseline_period} * 1d);\n", + "let bl_threshold = {threshold};\n", + "let operation_history = AuditLogs\n", + "| where TimeGenerated between(baseline_start .. start)\n", + "| where Identity !in (\"Azure AD Cloud Sync\", \"Managed Service Identity\", \"Microsoft.Azure.SyncFabric\")\n", + "| where bag_has_key(InitiatedBy, \"user\")\n", + "| extend UserPrincipalName = tostring(InitiatedBy[\"user\"][\"userPrincipalName\"])\n", + "| where UserPrincipalName in~ ({users})\n", + "| summarize EventCount=count() by UserPrincipalName, OperationName\n", + "| where EventCount > bl_threshold;\n", + "AuditLogs\n", + "| where TimeGenerated between(end .. start)\n", + "| where Identity !in (\"Azure AD Cloud Sync\", \"Managed Service Identity\", \"Microsoft.Azure.SyncFabric\")\n", + "| where bag_has_key(InitiatedBy, \"user\")\n", + "| extend UserPrincipalName = tostring(InitiatedBy[\"user\"][\"userPrincipalName\"]), IPAddress = InitiatedBy[\"user\"][\"ipAddress\"]\n", + "| where UserPrincipalName in~ ({users})\n", + "| join kind=leftanti (operation_history) on UserPrincipalName, OperationName\n", + "| project Identity, UserPrincipalName, OperationName, LoggedByService, InitiatedBy, AdditionalDetails, TargetResources\n", + "\"\"\"\n", + "\n", + "end = datetime.now(tz=timezone.utc)\n", + "start = end-timedelta(1)\n", + "from datetime import datetime, timezone, timedelta\n", + "fmt_query = azure_audit_query.format(\n", + " start=start,\n", + " end=end,\n", + " baseline_period=baseline_period,\n", + " threshold=0,\n", + " users=get_user_param(risk_users_df),\n", + ")\n", + "az_audit_df = qry_prov.exec_query(fmt_query)\n", + "summary_report.add_summary_data(\n", + " data=az_audit_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Unusual Azure Audit log entries for user\"\n", + ")\n", + "df_caption(az_audit_df, caption=\"Azure audit activity types not seen in baseline period.\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# New or unusual Office 365 activity\n", + "\n", + "Office operations occurring in the measured period that had\n", + "not occurred or rarely occurred in the baseline period." + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [], + "source": [ + "o365_baseline_activity_query = \"\"\"\n", + "let num_stddev = {std_dev_scale};\n", + "let bl_period = datetime_add(\"day\", -{baseline_period}, datetime({start}));\n", + "OfficeActivity\n", + "| where TimeGenerated between (bl_period .. datetime({start}))\n", + "| where UserId in~ ({users})\n", + "// count operations by user and op type per day\n", + "| summarize OpCount = count() by UserId, OfficeWorkload, Operation, bin(TimeGenerated, 1d)\n", + "// calculate mean and average values for the user/op combos\n", + "| summarize OpStdev = stdev(OpCount), OpMean = avg(OpCount) by UserId, OfficeWorkload, Operation\n", + "// Calculate a baseline score Mean + N StdDevs * StdDev (default to 1 if 0 variance)\n", + "| extend OpBase = OpMean + (num_stddev * iif(OpStdev > 0, OpStdev, 1.0))\n", + "| extend RecType=\"baseline\"\n", + "\"\"\"\n", + "\n", + "o365_current_activity_query = \"\"\"\n", + "OfficeActivity\n", + "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", + "| where UserId in~ ({users})\n", + "| summarize OpCount = count() by UserId, OfficeWorkload, Operation\n", + "| extend RecType=\"current\"\n", + "\"\"\"\n", + "\n", + "# set number of std deviations from mean to use as indicating\n", + "# anomalous activity\n", + "_STD_THRESHOLD = 2\n", + "\n", + "end = datetime.now(tz=timezone.utc)\n", + "start = end - timedelta(1)\n", + "office_baseline_df = qry_prov.exec_query(\n", + " o365_baseline_activity_query.format(\n", + " users=get_user_param(risk_users_df),\n", + " std_dev_scale=_STD_THRESHOLD,\n", + " start=start,\n", + " baseline_period=baseline_period,\n", + " )\n", + ")\n", + "office_current_df = qry_prov.exec_query(\n", + " o365_current_activity_query.format(\n", + " users=get_user_param(risk_users_df),\n", + " start=start,\n", + " end=end\n", + " )\n", + ")\n", + "\n", + "# Pull out any current activity that exceeds the baseline threshold (mean + N*stddev)\n", + "office_activity_df = (\n", + " office_current_df\n", + " .merge(office_baseline_df, on=[\"UserId\", \"OfficeWorkload\", \"Operation\"], how=\"left\")\n", + " .fillna({\"OpBase\": 0})\n", + " .query(\"OpCount > OpBase\")\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Office baseline operations.
 UserIdOfficeWorkloadOperationOpStdevOpMeanOpBaseRecType
0tamuto@seccxpninja.onmicrosoft.comSharePointFileUploaded0.1889822.9642863.342250baseline
1tamuto@seccxpninja.onmicrosoft.comExchangeMailItemsAccessed2.5726292.2500007.395258baseline
2tamuto@seccxpninja.onmicrosoft.comExchangeCreate0.0000001.0000003.000000baseline
3tamuto@seccxpninja.onmicrosoft.comSharePointFileAccessed0.0000002.0000004.000000baseline
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Office current operations.
 UserIdOfficeWorkloadOperationOpCountRecType
0tamuto@seccxpninja.onmicrosoft.comExchangeMailItemsAccessed2current
1tamuto@seccxpninja.onmicrosoft.comSharePointFileUploaded3current
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Office anomalous operations.
 UserIdOfficeWorkloadOperationOpCountRecType_xOpStdevOpMeanOpBaseRecType_y
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "df_caption(office_baseline_df, \"Office baseline operations.\")\n", + "df_caption(office_current_df, \"Office current operations.\")\n", + "df_caption(office_activity_df, \"Office anomalous operations.\")\n", + "summary_report.add_summary_data(\n", + " data=office_activity_df,\n", + " user_column=\"UserId\",\n", + " section=\"Unusual Office activity for user\"\n", + ")\n", + "summary_report.add_summary_data(\n", + " data=office_current_df,\n", + " user_column=\"UserId\",\n", + " section=\"Summarized current Office activity for user\"\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Unusual Azure activity\n", + "\n", + "Azure activity operations occurring in the measured period that had\n", + "not occurred in the baseline period.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Index(['TimeGenerated', 'UserPrincipalName', 'OperationNameValue', 'IPAddress',\n", + " 'EventDataId', 'ActivityStatusValue', 'ResourceGroup', 'SubscriptionId',\n", + " 'TenantId'],\n", + " dtype='object')" + ] + }, + "execution_count": 85, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "azure_activity_df.columns" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Azure activity operations not seen in baseline period.
 UserPrincipalNameOperationNameValueIPAddressResourceGroupSubscriptionIdTenantIdEventCountActivityStatusValueStartTimeEndTime
3tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/GETWORKERCOUNT/ACTION147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d10['Success' 'Start']2023-02-01 05:52:53.605463700+00:002023-02-01 06:00:53.770754+00:00
6tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/HYBRIDRUNBOOKWORKERS/WRITE147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 06:00:13.837572300+00:002023-02-01 06:00:15.603256700+00:00
12tamuto@seccxpninja.onmicrosoft.comMICROSOFT.COMPUTE/VIRTUALMACHINES/EXTENSIONS/WRITE147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Accept' 'Start']2023-02-01 06:00:15.803323200+00:002023-02-01 06:00:21.787995800+00:00
11tamuto@seccxpninja.onmicrosoft.comMICROSOFT.COMPUTE/VIRTUALMACHINES/EXTENSIONS/DELETE147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d7['Success' 'Accept' 'Start']2023-02-01 06:00:19.999565200+00:002023-02-01 06:12:47.627469900+00:00
5tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/HYBRIDRUNBOOKWORKERS/DELETE147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d4['Success' 'Start' 'Failure']2023-02-01 06:00:52.906841700+00:002023-02-01 06:01:17.421008100+00:00
20tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/NETWORKINTERFACES/EFFECTIVENETWORKSECURITYGROUPS/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d18['Accept' 'Start' 'Success']2023-02-01 06:02:56.308497800+00:002023-02-01 12:52:20.724925900+00:00
24tamuto@seccxpninja.onmicrosoft.comMICROSOFT.RECOVERYSERVICES/LOCATIONS/BACKUPSTATUS/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d4['Success' 'Start']2023-02-01 06:03:00.808491100+00:002023-02-01 06:04:01.636515100+00:00
4tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/GETWORKERCOUNT/ACTION147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d10['Success' 'Start']2023-02-01 07:05:00.205528100+00:002023-02-01 10:46:02.598749700+00:00
7tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/HYBRIDRUNBOOKWORKERS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 07:05:49.958148300+00:002023-02-01 07:05:52.100110600+00:00
13tamuto@seccxpninja.onmicrosoft.comMICROSOFT.COMPUTE/VIRTUALMACHINES/EXTENSIONS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d5['Success' 'Accept' 'Start']2023-02-01 07:05:52.301304600+00:002023-02-01 07:18:07.123937800+00:00
23tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/VIRTUALNETWORKS/SUBNETS/WRITE118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d3['Accept' 'Start' 'Success']2023-02-01 07:25:57.158919700+00:002023-02-01 07:26:03.513145200+00:00
25tamuto@seccxpninja.onmicrosoft.comMICROSOFT.RESOURCES/DEPLOYMENTS/VALIDATE/ACTION147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 07:26:03.099815300+00:002023-02-01 07:26:09.520994300+00:00
26tamuto@seccxpninja.onmicrosoft.comMICROSOFT.RESOURCES/DEPLOYMENTS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d3['Success' 'Accept' 'Start']2023-02-01 07:26:09.708168200+00:002023-02-01 07:37:00.044172400+00:00
22tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/PUBLICIPADDRESSES/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d5['Success' 'Accept' 'Start']2023-02-01 07:26:17.593352800+00:002023-02-01 07:38:34.764853600+00:00
19tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/BASTIONHOSTS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d10['Accept' 'Success' 'Start']2023-02-01 07:26:27.613222300+00:002023-02-01 07:48:47.545287400+00:00
29tamuto@seccxpninja.onmicrosoft.comMICROSOFT.SECURITYINSIGHTS/INCIDENTS/RUNPLAYBOOK/ACTION147.243.28.22SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d8['Success' 'Start']2023-02-01 07:51:57.616138800+00:002023-02-01 07:52:06.451008600+00:00
28tamuto@seccxpninja.onmicrosoft.comMICROSOFT.SECURITYINSIGHTS/INCIDENTS/RELATIONS/WRITE147.243.28.22SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d4['Success' 'Start']2023-02-01 07:52:29.689097400+00:002023-02-01 08:00:47.658756600+00:00
27tamuto@seccxpninja.onmicrosoft.comMICROSOFT.SECURITYINSIGHTS/INCIDENTS/RELATIONS/DELETE147.243.28.22SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d4['Success' 'Start']2023-02-01 07:55:50.427017+00:002023-02-01 07:59:57.879176500+00:00
18tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/BASTIONHOSTS/GETACTIVESESSIONS/ACTION118.200.55.233SOC-MDEd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d3['Accept' 'Start' 'Success']2023-02-01 09:15:05.655942+00:002023-02-01 09:15:12.692524100+00:00
17tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/BASTIONHOSTS/GETACTIVESESSIONS/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d14['Failure' 'Start' 'Success' 'Accept']2023-02-01 09:15:53.640419100+00:002023-02-01 12:51:56.844321800+00:00
9tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/RUNBOOKS/DRAFT/WRITE118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d27['Success' 'Accept' 'Start']2023-02-01 10:54:00.931669700+00:002023-02-01 11:03:34.187166400+00:00
10tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/RUNBOOKS/PUBLISH/ACTION147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d11['Accept' 'Start' 'Success']2023-02-01 10:54:01.203270+00:002023-02-01 11:04:06.589645400+00:00
14tamuto@seccxpninja.onmicrosoft.comMICROSOFT.LOGIC/WORKFLOWS/DISABLE/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 10:55:22.306755800+00:002023-02-01 10:55:24.103642+00:00
8tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/JOBS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d8['Accept' 'Start']2023-02-01 10:56:08.818195300+00:002023-02-01 11:19:46.546390100+00:00
15tamuto@seccxpninja.onmicrosoft.comMICROSOFT.LOGIC/WORKFLOWS/ENABLE/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 11:30:14.904068+00:002023-02-01 11:30:19.279050800+00:00
16tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/BASTIONHOSTS/DELETE118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d5['Success' 'Accept' 'Start']2023-02-01 12:51:54.313056100+00:002023-02-01 13:10:14.653217100+00:00
21tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/NETWORKSECURITYGROUPS/SECURITYRULES/WRITE118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d5['Success' 'Start' 'Accept']2023-02-01 12:52:39.782214+00:002023-02-01 12:52:43.287663600+00:00
2tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/WRITE147.243.27.237SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-02 03:31:40.786224200+00:002023-02-02 03:31:48.567780100+00:00
0tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/DELETE147.243.27.237SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-02 03:35:45.544154200+00:002023-02-02 03:35:50.481888+00:00
1tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/WRITE147.243.27.204SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-02 03:39:26.520506+00:002023-02-02 03:39:33.520717900+00:00
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Azure Activity\n", + "azure_activity_query = \"\"\"\n", + "let start = datetime(\"{start}\");\n", + "let end = datetime(\"{end}\");\n", + "let baseline_start = start - ({period} * 1d);\n", + "let bl_threshold = {threshold};\n", + "let operation_history = AzureActivity\n", + "| where TimeGenerated between(baseline_start .. start)\n", + "| where Caller in~ ({users})\n", + "| project UserPrincipalName=Caller, OperationNameValue\n", + "| summarize EventCount=count() by UserPrincipalName, OperationNameValue\n", + "| where EventCount > bl_threshold;\n", + "AzureActivity\n", + "| where TimeGenerated between(start .. end)\n", + "| where Caller in~ ({users})\n", + "| project-rename UserPrincipalName=Caller\n", + "| join kind=leftanti (operation_history) on UserPrincipalName, OperationNameValue\n", + "| project TimeGenerated, UserPrincipalName, OperationNameValue, IPAddress=CallerIpAddress,\n", + " EventDataId, ActivityStatusValue, ResourceGroup, SubscriptionId, TenantId\n", + "\"\"\"\n", + "\n", + "fmt_query = azure_activity_query.format(\n", + " end=datetime.now(tz=timezone.utc),\n", + " start=end-timedelta(1),\n", + " period=28,\n", + " threshold=0,\n", + " users=get_user_param(risk_users_df),\n", + ")\n", + "azure_activity_df = qry_prov.exec_query(fmt_query)\n", + "\n", + "aa_summary_cols = [\n", + " \"UserPrincipalName\",\n", + " \"OperationNameValue\",\n", + " \"IPAddress\",\n", + " \"ResourceGroup\",\n", + " \"SubscriptionId\",\n", + " \"TenantId\",\n", + "]\n", + "\n", + "azure_activity_summary_df = azure_activity_df.groupby(aa_summary_cols).agg(\n", + " EventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", + " ActivityStatusValue=pd.NamedAgg(\"ActivityStatusValue\", \"unique\"),\n", + " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", + " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", + ").reset_index().sort_values(\"StartTime\", ascending=True)\n", + "\n", + "summary_report.add_summary_data(\n", + " data=azure_activity_summary_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Unusual Azure activity for user\"\n", + ")\n", + "df_caption(azure_activity_summary_df, \"Azure activity operations not seen in baseline period.\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Summarizing data\n", + "\n", + "Create dynamic summaries for each user and upload to sentinel\n", + "\n", + "> Note: we could offer the option to group by report type instead\n", + "> of user. That would result in a Dynamic Summary entry for each\n", + "> report type (with consistent schema) but with data from (potentially)\n", + "> multiple users." + ] + }, + { + "cell_type": "code", + "execution_count": 110, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "AccountEvaluation - tamuto@seccxpninja.onmicrosoft.com\n" + ] + } + ], + "source": [ + "# Iterate through summary reports and create a summary for each user\n", + "\n", + "dynamic_summaries = []\n", + "for user, reports in summary_report._summary_reports.items():\n", + " # Create a summary for each user\n", + " user_ds = DynamicSummary(\n", + " summary_name=f\"AccountEvaluation - {user}\",\n", + " summary_description=\"Summary generated from AccountSignInEvaluation notebook.\",\n", + " source_info=\"AccountSignInEvaluation.ipynb\"\n", + " )\n", + "\n", + " for report_type, summary_item in reports.items():\n", + " ds_item_params = {\n", + " \"event_time_utc\": end,\n", + " \"search_key\": user,\n", + " \"observable_type\": \"report_type\",\n", + " \"observable_value\": report_type\n", + " }\n", + " user_ds.add_summary_items(\n", + " data=summary_item.data,\n", + " **ds_item_params\n", + " )\n", + " dynamic_summaries.append(user_ds)\n", + "\n", + "for dyn_summary in dynamic_summaries:\n", + " # Create or update the report\n", + " # sentinel.create_dynamic_summary(dyn_summary)\n", + " print(dyn_summary.summary_name)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Appendix - Pickling and restoring data" + ] + }, + { + "cell_type": "code", + "execution_count": 111, + "metadata": {}, + "outputs": [], + "source": [ + "import pickle\n", + "obj = pickle.dumps(dynamic_summaries)\n", + "\n", + "with open(\"acct_nb_summaries.pkl\", \"wb\") as pickle_file:\n", + " pickle_file.write(obj)" + ] + }, + { + "cell_type": "code", + "execution_count": 112, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['AccountEvaluation - tamuto@seccxpninja.onmicrosoft.com']\n" + ] + } + ], + "source": [ + "# note - you need to have the DynamicSummary class imported\n", + "from msticpy.context.azure.sentinel_dynamic_summary import DynamicSummary\n", + "# defined (see earlier in the notebook) to successfully restore\n", + "# the summary report\n", + "with open(\"acct_nb_summaries.pkl\", \"rb\") as pickle_file:\n", + " summary_obj = pickle_file.read()\n", + " dynamic_summaries_copy = pickle.loads(obj)\n", + "\n", + "print([ds.summary_name for ds in dynamic_summaries_copy])" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "msticpy", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.7" + }, + "orig_nbformat": 4, + "vscode": { + "interpreter": { + "hash": "0f1a8e166ce5c1ec1911a36e4fdbd34b2f623e2a3442791008b8ac429a1d6070" + } + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} From 48f0a691264e6a4d987f8d54f4a0ddcee0f2af9d Mon Sep 17 00:00:00 2001 From: Ian Hellen Date: Wed, 22 Mar 2023 12:51:24 -0700 Subject: [PATCH 2/2] Completed AccountSignInEvaluation.ipynb notebook. Updated Authoring automated notebooks readme to include reference to MSTICPy Dynamic Summaries implementation. --- AccountSignInEvaluation.ipynb | 2809 --------------- .../AccountSignInEvaluation.ipynb | 3123 +++++++++++++++++ .../Authoring automated notebooks.md | 31 +- 3 files changed, 3142 insertions(+), 2821 deletions(-) delete mode 100644 AccountSignInEvaluation.ipynb create mode 100644 scenario-notebooks/Automated-Notebooks/AccountSignInEvaluation.ipynb diff --git a/AccountSignInEvaluation.ipynb b/AccountSignInEvaluation.ipynb deleted file mode 100644 index cd5bb661..00000000 --- a/AccountSignInEvaluation.ipynb +++ /dev/null @@ -1,2809 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Unusual Account Activity\n", - "
\n", - "  Notebook Details...\n", - "\n", - " **Notebook Version:** 2.0
\n", - " **Python Version:** Python 3.8+
\n", - " **Required Packages**: msticpy, msticnb
\n", - "\n", - " **Data Sources Required**:\n", - " - Sentinel - SecurityAlert, SecurityEvent, HuntingBookmark, Syslog, AAD SigninLogs, AzureActivity, OfficeActivity, ThreatIndicator\n", - " - (Optional) - VirusTotal, AlienVault OTX, IBM XForce, Open Page Rank, (all require accounts and API keys)\n", - "
\n", - "\n", - "## TBD" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "toc": true - }, - "source": [ - "\n", - "\n", - " \n", - " \n", - " \n", - "

Contents

\n", - "
\n", - "
    \n", - "
  • TBD
    • \n", - "
  • TBD
    • \n", - "
\n", - "
\n", - " \n", - " \n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Hunting Hypothesis\n", - "TBD\n", - "\n", - "\n", - "Flow:\n", - "- Query - risk-flagged sign-ins\n", - "- Add supplemental queries\n", - "- Query alerts for related accounts\n", - "- TI lookup for source IP (other?)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "---\n", - "# Notebook initialization\n", - "This should complete without errors. If you encounter errors or warnings look at the following notebooks:\n", - "\n", - "- Getting Started Notebook\n", - "- [TroubleShootingNotebooks](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/TroubleShootingNotebooks.ipynb)\n", - "- [ConfiguringNotebookEnvironment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)\n", - "\n", - "
\n", - "  Details...\n", - "The next cell:\n", - "- Checks for the correct Python version\n", - "- Checks versions and optionally installs required packages\n", - "- Imports the required packages into the notebook\n", - "- Sets a number of configuration options.\n", - "\n", - "If you are running in the Azure Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:\n", - "- [Getting Started](./A Getting Started Guide For Azure Sentinel ML Notebooks.ipynb)\n", - "- [Run TroubleShootingNotebooks](./TroubleShootingNotebooks.ipynb)\n", - "- [Run ConfiguringNotebookEnvironment](./ConfiguringNotebookEnvironment.ipynb)\n", - "\n", - "You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup. \n", - "There are more details about this in the `ConfiguringNotebookEnvironment` notebook and in these documents:\n", - "- [msticpy configuration](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)\n", - "- [Threat intelligence provider configuration](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file)\n", - "
" - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "This product includes GeoLite2 data created by MaxMind, available from\n", - "https://www.maxmind.com.\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "Attempting connection to Key Vault using cli credentials..." - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "done
" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "\n", - "This library uses services provided by ipstack.\n", - "https://ipstack.com" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "from datetime import datetime, timedelta, timezone\n", - "\n", - "REQ_PYTHON_VER = \"3.8\"\n", - "REQ_MSTICPY_VER = \"2.3.0\"\n", - "\n", - "# %pip install --upgrade msticpy\n", - "\n", - "import msticpy as mp\n", - "mp.init_notebook()\n" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "tags": [ - "parameters" - ] - }, - "outputs": [], - "source": [ - "# papermill default parameters\n", - "ws_name = \"Default\"\n", - "end = datetime.now(timezone.utc)\n", - "start = end - timedelta(days=2)\n", - "baseline_period = 28\n", - "run_date = end" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Get Workspace and Authenticate\n", - "\n", - "
\n", - " Authentication help...\n", - " If you want to use a workspace other than one you have defined in your
\n", - "msticpyconfig.yaml create a connection string with your AAD TENANT_ID and
\n", - "your WORKSPACE_ID (these should both be quoted UUID strings).\n", - "\n", - "```python\n", - " workspace_cs = \"loganalytics://code().tenant('TENANT_ID').workspace('WORKSPACE_ID')\"\n", - "```\n", - "e.g.\n", - "```python\n", - " workspace_cs = \"loganalytics://code().tenant('c3de0f06-dcb8-40fb-9d1a-b62faea29d9d').workspace('c62d3dc5-11e6-4e29-aa67-eac88d5e6cf6')\"\n", - "```\n", - "Then in the Authentication cell replace\n", - "the call to `qry_prov.connect` with the following:\n", - "```python\n", - " qry_prov.connect(connect_str=workspace_cs)\n", - "```\n", - "The cell should now look like this:\n", - "\n", - "```python\n", - "...\n", - " # Authentication\n", - " qry_prov = QueryProvider(data_environment=\"MSSentinel\")\n", - " qry_prov.connect(connect_str=workspace_cs)\n", - "...\n", - "```\n", - "\n", - "On successful authentication you should see a ```popup schema``` button.\n", - "To find your Workspace Id go to [Log Analytics](https://ms.portal.azure.com/#blade/HubsExtension/Resources/resourceType/Microsoft.OperationalInsights%2Fworkspaces). Look at the workspace properties to find the ID.\n", - "
" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Configured workspaces: ASIHuntOMSWorkspaceV4, CCIS, Centrica, CyberSecuritySoc, Default, GovCyberSecuritySOC, NationalGrid, RedmondSentinelDemoEnvironment\n" - ] - }, - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "0576052186894a1a8969969bd4a8eb9f", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "Combobox(value='Default', description='Workspace Name', options=('ASIHuntOMSWorkspaceV4', 'CCIS', 'Centrica', …" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "print(\"Configured workspaces: \", \", \".join(msticpy.settings.get_config(\"AzureSentinel.Workspaces\").keys()))\n", - "import ipywidgets as widgets\n", - "ws_param = widgets.Combobox(\n", - " description=\"Workspace Name\",\n", - " value=ws_name,\n", - " options=list(msticpy.settings.get_config(\"AzureSentinel.Workspaces\").keys())\n", - ")\n", - "ws_param" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Connecting... connected\n" - ] - }, - { - "data": { - "text/html": [ - "

" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "

Confirm time range to search

" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "5421cbd9aa08477fa1502bb6201aa14a", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "VBox(children=(HTML(value='

Set query time boundaries

'), HBox(children=(DatePicker(value=datetime.date…" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "from msticpy.common.timespan import TimeSpan\n", - "from msticpy.context.azure import MicrosoftSentinel\n", - "\n", - "# Authentication\n", - "qry_prov = mp.QueryProvider(data_environment=\"MSSentinel\")\n", - "qry_prov.connect(workspace=ws_param.value)\n", - "\n", - "sentinel = MicrosoftSentinel(workspace=ws_param.value, connect=True)\n", - "\n", - "nb_timespan = TimeSpan(start, end)\n", - "qry_prov.query_time.timespan = nb_timespan\n", - "md(\"
\")\n", - "md(\"Confirm time range to search\", \"bold\")\n", - "qry_prov.query_time" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Notebook Logic\n", - "\n", - "{period} = query time period\n", - "\n", - "{baseline} = {period}.start - 28 days ... {period}.start\n", - "\n", - "1. Find users with high risk and unmitigated signin for {period}\n", - "2. Find users with high risk signins for {baseline}\n", - "3. Divide 1 into:\n", - " a. Users with on-going high risk - for triage\n", - " b. Users with new high risk status\n", - "4. For users in 3.a, check:\n", - " - Azure activity - any activity types in {period} not in baseline {baseline}\n", - " - Azure audit - any activity types in {period} not in baseline {baseline}\n", - "\n", - "Output (dynamic summary):\n", - "- List of ongoing high risk users\n", - "- New high risk users:\n", - " - Signin types and locations\n", - " - Novel Azure Activity and Audit types" - ] - }, - { - "cell_type": "code", - "execution_count": 50, - "metadata": {}, - "outputs": [], - "source": [ - "import re\n", - "import urllib\n", - "from collections import namedtuple, defaultdict\n", - "from datetime import datetime, timedelta, timezone\n", - "from typing import Dict, NamedTuple, Optional\n", - "\n", - "import httpx\n", - "import pandas as pd\n", - "import yaml\n", - "from tqdm.auto import tqdm\n", - "\n", - "from msticpy.context.azure.sentinel_dynamic_summary import DynamicSummary, DynamicSummaryItem\n", - "\n", - "\n", - "# Summary report classes\n", - "class SummaryItem(NamedTuple):\n", - " \"\"\"Data report collection for summary.\"\"\"\n", - " key: str\n", - " data: pd.DataFrame\n", - " properties: Dict[str, Any]\n", - "\n", - "\n", - "class SummaryReport:\n", - " \"\"\"Class to hold summary reports during exec of notebook.\"\"\"\n", - " def __init__(self):\n", - " self._summary_reports: Dict[str, Dict[str, SummaryItem]] = defaultdict(dict)\n", - "\n", - " def add_summary_data(self, data: pd.DataFrame, user_column: str, section: str, **kwargs):\n", - " \"\"\"Add data for users to the summary report\"\"\"\n", - " for user, user_data in data.groupby(user_column):\n", - " summary = SummaryItem(\n", - " key=user,\n", - " data=user_data,\n", - " properties=kwargs\n", - " )\n", - " self._summary_reports[user.casefold()][section] = summary\n", - "\n", - " @property\n", - " def users(self):\n", - " return sorted(self._summary_reports)\n", - "\n", - " @property\n", - " def report_types(self):\n", - " return sorted({\n", - " report for user_reports in self._summary_reports.values()\n", - " for report in user_reports\n", - " })\n", - "\n", - "\n", - "summary_report = SummaryReport()\n", - "\n", - "\n", - "# DF display function\n", - "def df_caption(data: pd.DataFrame, caption: str):\n", - " \"\"\"Display dataframe with a caption.\"\"\"\n", - " caption_css = \"; \".join([\n", - " \"caption-side: top\",\n", - " \"text-align: left\",\n", - " \"font-size: 15pt\",\n", - " \"font-weight: bold\",\n", - " \"padding: 5pt\",\n", - " ])\n", - " display(\n", - " data.style.set_caption(f\"{caption}\").set_table_styles(\n", - " [\n", - " {\n", - " \"selector\": \"caption\",\n", - " \"props\": caption_css,\n", - " }\n", - " ]\n", - " )\n", - " )\n", - "\n", - "\n", - "def get_user_param(data: pd.DataFrame) -> str:\n", - " \"\"\"Return user names from DataFrame as comma-sep string.\"\"\"\n", - " return \",\".join([\n", - " f\"'{user}'\" for user\n", - " in data.UserPrincipalName.values\n", - " ])\n", - "\n", - "\n", - "# update any changes to start/end datetimes\n", - "start = qry_prov.query_time.start\n", - "end = qry_prov.query_time.end" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Get risk-flagged sign-ins\n", - "\n", - "This query retrieves user signins that have been flagged by Azure Identity Protection\n", - "as at risk. See [Azure Identity Protection](https://learn.microsoft.com/azure/active-directory/identity-protection/overview-identity-protection)\n", - "for more background." - ] - }, - { - "cell_type": "code", - "execution_count": 51, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Unmitigated risk users
 UserPrincipalName
1tamuto@seccxpninja.onmicrosoft.com
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Mitigated risk users
 UserPrincipalName
0pdemo@seccxpninja.onmicrosoft.com
2dwilliams@seccxp.ninja
3adm_pwatkins@seccxpninja.onmicrosoft.com
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "signing_risk_query = \"\"\"\n", - "SigninLogs\n", - "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", - "| where RiskState != \"none\"\n", - "| project UserPrincipalName, ResultDescription, RiskState, RiskDetail, RiskEventTypes,\n", - " RiskEventTypes_V2, RiskLevelAggregated, RiskLevelDuringSignIn, IPAddress\n", - "| extend SigninRisk = case(\n", - " RiskLevelDuringSignIn == \"high\", 5,\n", - " RiskLevelDuringSignIn == \"medium\", 3,\n", - " RiskLevelDuringSignIn == \"low\", 1,\n", - " 0\n", - " ),\n", - " AggRisk = case(\n", - " RiskLevelAggregated == \"high\", 5,\n", - " RiskLevelAggregated == \"medium\", 3,\n", - " RiskLevelAggregated == \"low\", 1,\n", - " 0\n", - " )\n", - "| extend RiskEventDyn = parse_json(RiskEventTypes), RiskEventV2Dyn = parse_json(RiskEventTypes_V2)\n", - "| mv-expand RiskEventDyn, RiskEventV2Dyn\n", - "| summarize SignIns=count(AggRisk), MeanAggRisk=avg(AggRisk), MeanSigninRisk=avg(SigninRisk), \n", - " RiskStates=make_set(RiskState), RiskEvents=make_set(RiskEventDyn), RiskEventsV2=make_set(RiskEventV2Dyn),\n", - " SourceIPs=make_set(IPAddress)\n", - " by UserPrincipalName\n", - "| order by MeanAggRisk, MeanSigninRisk asc nulls last\n", - "\"\"\"\n", - "\n", - "# run the query\n", - "signin_risk_users_df = qry_prov.exec_query(\n", - " signing_risk_query.format(start=start, end=end)\n", - ")\n", - "# expand RiskStates (list)\n", - "risk_states_df = signin_risk_users_df.explode(\"RiskStates\")\n", - "# Extract list of users where risk was mitigated \n", - "safe_users_df = risk_states_df[risk_states_df[\"RiskStates\"].isin([\"remediated\", \"confirmedSafe\"])].UserPrincipalName.drop_duplicates()\n", - "\n", - "# Separate unmitigated from mitigated risk users\n", - "risk_users_df = signin_risk_users_df[~signin_risk_users_df[\"UserPrincipalName\"].isin(safe_users_df)]\n", - "mitigated_users_df = signin_risk_users_df[signin_risk_users_df[\"UserPrincipalName\"].isin(safe_users_df)]\n", - "\n", - "df_caption(risk_users_df[[\"UserPrincipalName\"]], \"Unmitigated risk users\")\n", - "df_caption(mitigated_users_df[[\"UserPrincipalName\"]], \"Mitigated risk users\")" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Retrieve the historical risk level for previous N days\n", - "\n", - "This is used to distinguish accounts that have a new \"At Risk\"\n", - "designation from those accounts that have a history of risk signins.\n", - "\n", - "> Note: \"N\" is the `baseline_period` parameter for the notebook - default is 28 days" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Signing Summary for users with unmitigated risk" - ] - }, - { - "cell_type": "code", - "execution_count": 53, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Sign-in risk summary - unmitigated
 UserPrincipalNameSignInsMeanAggRiskMeanSigninRiskRiskStatesRiskEventsRiskEventsV2SourceIPsRiskHistory
1tamuto@seccxpninja.onmicrosoft.com31.0000001.000000['atRisk']['unfamiliarFeatures', 'unlikelyTravel']['unfamiliarFeatures', 'unlikelyTravel']['20.25.98.192']Existing
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "_AADSIL_DISPLAY_COLUMNS = [\n", - " 'TimeGenerated', 'ResultType', 'ResultDescription', 'UserPrincipalName', 'UserId',\n", - " 'Location', 'IPAddress', 'AppDisplayName', 'ClientAppUsed', 'AppId',\n", - " 'AuthenticationDetails', 'AuthenticationMethodsUsed',\n", - " 'RiskEventTypes', 'RiskEventTypes_V2', 'RiskLevelAggregated',\n", - " 'RiskLevelDuringSignIn', 'RiskState', 'ResourceDisplayName',\n", - " 'LocationDetails', 'MfaDetail', 'NetworkLocationDetails',\n", - " 'UserAgent', 'UserDisplayName', 'UserType', 'IPAddressFromResourceProvider',\n", - " 'ResourceTenantId', 'HomeTenantId', 'AutonomousSystemNumber', 'Type'\n", - "]\n", - "\n", - "\n", - "# Function to summarize the history data\n", - "def weekly_signin_summary(data) -> pd.DataFrame:\n", - " \"\"\"Create signin summary from historical data.\"\"\"\n", - " return (\n", - " data\n", - " [_AADSIL_DISPLAY_COLUMNS]\n", - " .explode([\"RiskEventTypes\"])\n", - " .groupby([\"UserPrincipalName\", pd.Grouper(key=\"TimeGenerated\", freq=\"W\")])\n", - " .agg(\n", - " LoginCount=pd.NamedAgg(\"ResultType\", \"count\"),\n", - " ResultTypes=pd.NamedAgg(\"ResultType\", \"unique\"),\n", - " RiskEventTypes=pd.NamedAgg(\"RiskEventTypes\", \"unique\"),\n", - " RiskLevels=pd.NamedAgg(\"RiskLevelAggregated\", \"unique\"),\n", - " RiskLevelSignins=pd.NamedAgg(\"RiskLevelDuringSignIn\", \"unique\"),\n", - " IPs=pd.NamedAgg(\"IPAddress\", \"nunique\"),\n", - " Locations=pd.NamedAgg(\"Location\", \"nunique\"),\n", - " Apps=pd.NamedAgg(\"AppDisplayName\", \"nunique\"),\n", - " UserAgents=pd.NamedAgg(\"UserAgent\", \"nunique\"),\n", - " StartDate=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", - " EndDate=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", - " )\n", - " .sort_index()\n", - " )\n", - "\n", - "\n", - "# Get historical risk level for previous {period} days\n", - "risk_hist_query = \"\"\"\n", - "let q_end = datetime({start});\n", - "let q_start = datetime_add(\"day\", -{period}, q_end);\n", - "SigninLogs\n", - "| where TimeGenerated between (q_start .. q_end)\n", - "| where RiskState != \"none\"\n", - "| where UserPrincipalName in ({users})\n", - "| extend RiskEventTypes = parse_json(RiskEventTypes), RiskEventTypes_V2 = parse_json(RiskEventTypes_V2)\n", - "\"\"\"\n", - "\n", - "# Unmitigated risk users\n", - "risk_user_hist_df = qry_prov.exec_query(\n", - " risk_hist_query.format(\n", - " users=get_user_param(risk_users_df),\n", - " start=start,\n", - " period=baseline_period,\n", - " )\n", - ")\n", - "\n", - "risk_users_history = weekly_signin_summary(risk_user_hist_df).reset_index()\n", - "\n", - "# Isolate users that have no history of risk in previous period\n", - "users_with_past_risk_criteria = risk_users_df.UserPrincipalName.isin(risk_user_hist_df.UserPrincipalName.unique())\n", - "risk_users_df = risk_users_df.copy()\n", - "risk_users_df.loc[~users_with_past_risk_criteria, \"RiskHistory\"] = \"New\"\n", - "risk_users_df.loc[users_with_past_risk_criteria, \"RiskHistory\"] = \"Existing\"\n", - "\n", - "summary_report.add_summary_data(\n", - " data=risk_users_df,\n", - " user_column=\"UserPrincipalName\",\n", - " section=\"Risk Users Summary\",\n", - ")\n", - "summary_report.add_summary_data(\n", - " data=risk_users_history,\n", - " user_column=\"UserPrincipalName\",\n", - " section=\"Risk Users History\",\n", - ")\n", - "\n", - "df_caption(risk_users_df, \"Sign-in risk summary - unmitigated\")" - ] - }, - { - "cell_type": "code", - "execution_count": 54, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
UserPrincipalNameTimeGeneratedLoginCountResultTypesRiskEventTypesRiskLevelsRiskLevelSigninsIPsLocationsAppsUserAgentsStartDateEndDate
0tamuto@seccxpninja.onmicrosoft.com2023-01-22 00:00:00+00:001[0][mcasImpossibleTravel][low][none]11112023-01-18 01:27:19.258557600+00:002023-01-18 01:27:19.258557600+00:00
\n", - "
" - ], - "text/plain": [ - " UserPrincipalName TimeGenerated LoginCount \\\n", - "0 tamuto@seccxpninja.onmicrosoft.com 2023-01-22 00:00:00+00:00 1 \n", - "\n", - " ResultTypes RiskEventTypes RiskLevels RiskLevelSignins IPs \\\n", - "0 [0] [mcasImpossibleTravel] [low] [none] 1 \n", - "\n", - " Locations Apps UserAgents StartDate \\\n", - "0 1 1 1 2023-01-18 01:27:19.258557600+00:00 \n", - "\n", - " EndDate \n", - "0 2023-01-18 01:27:19.258557600+00:00 " - ] - }, - "execution_count": 54, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "not_mit_risk_history" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Signing Summary for users with mitigated risk\n", - "### [info only]" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Sign-in risk summary - mitigated
 UserPrincipalNameSignInsMeanAggRiskMeanSigninRiskRiskStatesRiskEventsRiskEventsV2SourceIPsRiskHistory
0pdemo@seccxpninja.onmicrosoft.com2681.6194034.276119['atRisk', 'dismissed', 'confirmedSafe']['unfamiliarFeatures', 'unlikelyTravel']['unfamiliarFeatures', 'unlikelyTravel']['84.59.133.96', '49.207.205.157', '182.48.225.204', '202.171.187.206', '83.6.102.205', '94.239.55.19', '51.142.235.76', '110.49.50.142', '49.37.163.128', '165.225.120.88', '89.211.239.104', '94.245.87.14', '50.208.71.66', '140.186.246.113', '201.191.218.23', '190.104.120.0', '109.48.220.202', '73.43.36.19', '76.67.108.134', '148.64.97.101', '51.142.111.1', '37.186.51.21', '76.184.244.1', '172.13.62.40', '107.129.128.34', '99.248.154.225', '87.187.23.105', '108.34.158.176', '80.187.114.167', '157.49.156.108', '194.107.2.82', '167.220.24.243', '187.56.121.165', '73.25.210.7', '90.146.97.205', '195.97.138.43', '14.187.179.30', '114.79.170.132', '147.161.199.96', '8.23.71.2', '20.122.92.1', '189.249.64.2', '168.149.166.14', '47.231.129.2', '212.180.224.82', '107.11.97.170', '96.234.155.228', '147.235.216.117', '39.9.193.150', '109.147.153.136', '168.149.166.78', '70.164.213.113', '147.161.199.101', '24.98.48.107', '180.218.164.251', '94.174.54.38', '79.107.37.34', '208.104.177.188', '24.15.125.185', '104.219.136.49', '159.196.229.180', '209.65.150.148', '61.68.47.232']Existing
2dwilliams@seccxp.ninja20.5000003.000000['atRisk', 'confirmedSafe']['unfamiliarFeatures']['unfamiliarFeatures']['20.227.3.22']New
3adm_pwatkins@seccxpninja.onmicrosoft.com240.0000004.833333['remediated']['anonymizedIPAddress']['anonymizedIPAddress']['185.220.100.247', '192.42.116.216']Existing
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "# History of mitigated risk users\n", - "mit_risk_user_hist_df = qry_prov.exec_query(\n", - " risk_hist_query.format(\n", - " users=get_user_param(mitigated_users_df),\n", - " start=start,\n", - " period=baseline_period\n", - " )\n", - ")\n", - "\n", - "\n", - "# Isolate users that have no history of risk in previous period\n", - "users_with_past_risk_criteria = mitigated_users_df.UserPrincipalName.isin(mit_risk_user_hist_df.UserPrincipalName.unique())\n", - "mitigated_users_df = mitigated_users_df.copy()\n", - "mitigated_users_df.loc[~users_with_past_risk_criteria, \"RiskHistory\"] = \"New\"\n", - "mitigated_users_df.loc[users_with_past_risk_criteria, \"RiskHistory\"] = \"Existing\"\n", - "mitigated_users_df\n", - "df_caption(mitigated_users_df, \"Sign-in risk summary - mitigated\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Retrieve and Run UEBA hunting queries on risk-flagged users\n", - "\n", - "> UEBA = User Entity Behavior Analytics\n", - "\n", - "The next cell retrieves the current UEBA hunting\n", - "queries and runs them against the risk-flagged users.\n", - "\n", - "For more information see [Microsoft Sentinel UEBA](https://learn.microsoft.com/azure/sentinel/identify-threats-with-entity-behavior-analytics)" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 21/21 [00:04<00:00, 5.14it/s]\n" - ] - }, - { - "data": { - "text/html": [ - "
Anomalies on users tagged as VIPhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/anomaliesOnVIPUsers.yaml
Anomalous AAD Account Manipulationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20AAD%20Account%20Manipulation.yaml
Anomalous AAD Account Creationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Account%20Creation.yaml
Anomalous Activity Role Assignmenthttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Activity%20Role%20Assignment.yaml
Anomalous Code Executionhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Code%20Execution.yaml
Anomalous Data Accesshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Data%20Access.yaml
Anomalous Defensive Mechanism Modificationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Defensive%20Mechanism%20Modification.yaml
Anomalous Failed Logonhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Failed%20Logon.yaml
Anomalous Geo Location Logonhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Geo%20Location%20Logon.yaml
Anomalous Login to Deviceshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Login%20to%20Devices.yaml
Anomalous Password Resethttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Password%20Reset.yaml
Anomalous RDP Activityhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20RDP%20Activity.yaml
Anomalous Resource Accesshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Resource%20Access.yaml
Anomalous Role Assignmenthttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Role%20Assignment.yaml
Anomalous Sign-in Activityhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Sign-in%20Activity.yaml
Anomalous action performed in tenant by privileged userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/anomalousActionInTenant.yaml
Dormant account activity from uncommon countryhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/dormantAccountActivityFromUncommonCountry.yaml
Anomalous connection from highly privileged userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/firstConnectionFromGroup.yaml
Anomalous login activity originated from Botnet, Tor proxy or C2https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/loginActivityFromBotnet.yaml
New account added to admin grouphttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/newAccountAddedToAdminGroup.yaml
Anomalous update Key Vault activity by high blast radius userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/updateKeyVaultActivity.yaml
" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "# Hunting Queries\n", - "_SENTINEL_REPO = \"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master\"\n", - "_SI_LOG_ROOT = f\"{_SENTINEL_REPO}/Hunting%20Queries/SigninLogs\"\n", - "_GEN_HUNTING_QRY = [\n", - " # \"AnomalousUserAppSigninLocationIncreaseDetail.yaml\",\n", - " # \"LegacyAuthAttempt.yaml\",\n", - " # \"Signins-From-VPS-Providers.yaml\",\n", - " # \"UserAccountsMeasurableincreaseofsuccessfulsignins.yaml\",\n", - " # \"riskSignInWithNewMFAMethod.yaml\",\n", - " # \"signinBurstFromMultipleLocations.yaml\",\n", - "]\n", - "\n", - "# UEBA Hunting Queries\n", - "_UEBA_HQ_ROOT = f\"{_SENTINEL_REPO}/Solutions/UEBA%20Essentials/Hunting%20Queries\"\n", - "_UEBA_HUNTING_QRY = [\n", - " \"anomaliesOnVIPUsers.yaml\",\n", - " \"Anomalous AAD Account Manipulation.yaml\",\n", - " \"Anomalous Account Creation.yaml\",\n", - " \"Anomalous Activity Role Assignment.yaml\",\n", - " \"Anomalous Code Execution.yaml\",\n", - " \"Anomalous Data Access.yaml\",\n", - " \"Anomalous Defensive Mechanism Modification.yaml\",\n", - " \"Anomalous Failed Logon.yaml\",\n", - " \"Anomalous Geo Location Logon.yaml\",\n", - " \"Anomalous Login to Devices.yaml\",\n", - " \"Anomalous Password Reset.yaml\",\n", - " \"Anomalous RDP Activity.yaml\",\n", - " \"Anomalous Resource Access.yaml\",\n", - " \"Anomalous Role Assignment.yaml\",\n", - " \"Anomalous Sign-in Activity.yaml\",\n", - " \"anomalousActionInTenant.yaml\",\n", - " \"dormantAccountActivityFromUncommonCountry.yaml\",\n", - " \"firstConnectionFromGroup.yaml\",\n", - " \"loginActivityFromBotnet.yaml\",\n", - " \"newAccountAddedToAdminGroup.yaml\",\n", - " # \"terminatedEmployeeAccessHVA.yaml\",\n", - " # \"terminatedEmployeeActivity.yaml\",\n", - " \"updateKeyVaultActivity.yaml\",\n", - "]\n", - "\n", - "ALL_QUERIES = {qry: _SI_LOG_ROOT for qry in _GEN_HUNTING_QRY}\n", - "ALL_QUERIES.update({qry: _UEBA_HQ_ROOT for qry in _UEBA_HUNTING_QRY})\n", - "\n", - "TIME_TOKEN = re.compile(r\"(\\{\\{StartTimeISO\\}\\}|\\{\\{EndTimeISO\\}\\})\")\n", - "_LEFT_BRACE = r\"[^{](\\{)[^{]\"\n", - "_RIGHT_BRACE = r\"[^}](\\})[^}]\"\n", - "_LB_TOKEN = \"%%~[~%%\"\n", - "_RB_TOKEN = \"%%~]~%%\"\n", - "\n", - "\n", - "def replace_time_params(query):\n", - " repl_query = re.sub(_LEFT_BRACE, _LB_TOKEN, query)\n", - " repl_query = re.sub(_RIGHT_BRACE, _RB_TOKEN, repl_query)\n", - " repl_query = repl_query.replace(\"{{StartTimeISO}}\", \"{start}\").replace(\"{{EndTimeISO}}\", \"{end}\")\n", - " return repl_query.replace(_LB_TOKEN, \"{{\").replace(_RB_TOKEN, \"}}\")\n", - "\n", - "\n", - "QueryProps = namedtuple(\"QueryProps\", \"name, query, req_time, description, url, raw_query\")\n", - "\n", - "\n", - "def fetch_queries(query_dict: Dict[str, str], verbose: bool = False) -> Dict[str, QueryProps]:\n", - " \"\"\"Fetch queries from Sentinel GitHub repo.\"\"\"\n", - " discover_queries: Dict[str, QueryProps] = {}\n", - " error_queries: Dict[str, str] = {}\n", - " for query, path in tqdm(query_dict.items()):\n", - " q_path = f\"{path}/{urllib.parse.quote(query)}\"\n", - " resp = httpx.get(q_path)\n", - " if resp.status_code != 200:\n", - " print(f\"invalid URL {path}\")\n", - " continue\n", - " try:\n", - " q_dict = yaml.safe_load(resp.content)\n", - " except yaml.scanner.ScannerError as err:\n", - " print(f\"could not parse query {query} at {q_path}\")\n", - " error_queries[query] = resp.content\n", - " continue\n", - "\n", - " query_text = q_dict.get(\"query\")\n", - " req_time = False\n", - " if re.search(TIME_TOKEN, query_text):\n", - " query_text = replace_time_params(query_text)\n", - " req_time = True\n", - "\n", - " if \"UEBA\" in path:\n", - " query_text = add_ueba_time_params(query_text)\n", - " if verbose:\n", - " print(f\"Query {query}, {q_dict['name']}, req time: {req_time}\")\n", - " discover_queries[query] = QueryProps(\n", - " name=q_dict.get(\"name\"),\n", - " query=query_text,\n", - " req_time=req_time,\n", - " description=q_dict.get(\"description\"),\n", - " url=q_path,\n", - " raw_query=q_dict.get(\"query\"),\n", - " )\n", - " return discover_queries\n", - "\n", - "\n", - "PRIM_TABLE_EXP = r\"(?P^|\\n)(?P(BehaviorAnalytics|AuditLogs|IdentityInfo|SigninLogs))(?=[\\s\\n\\)\\|])\"\n", - "PRIM_TABLE_REPL = r\"\\g\\g
\\n| where TimeGenerated > datetime({start})\"\n", - "JOIN_TABLE_EXP = r\"(?P\\|\\s+join[^(]*\\(\\s*[^\\s]+)(?=[\\s\\)\\|])\"\n", - "JOIN_TABLE_REPL = r\"\\g\\n| where TimeGenerated > datetime({start})\"\n", - "\n", - "\n", - "def add_ueba_time_params(query):\n", - " if isinstance(query, tuple):\n", - " if query.req_time:\n", - " return query.query\n", - " query = query.query\n", - " return re.sub(\n", - " JOIN_TABLE_EXP,\n", - " JOIN_TABLE_REPL,\n", - " re.sub(PRIM_TABLE_EXP, PRIM_TABLE_REPL, query)\n", - " )\n", - "\n", - "\n", - "def display_query_table(queries):\n", - " ht_table = \"
{rows}
\"\n", - " rows = [f\"{q.name}{q.url}\"\n", - " for q in queries.values()]\n", - " from IPython.display import HTML\n", - " display(HTML(ht_table.format(rows=\"\".join(rows))))\n", - "\n", - "\n", - "hunting_queries = fetch_queries(ALL_QUERIES)\n", - "\n", - "display_query_table(hunting_queries).head()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Browser for UEBA queries - not used in notebook" - ] - }, - { - "cell_type": "code", - "execution_count": 35, - "metadata": {}, - "outputs": [ - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "7e8aa5f430fc41e491630200b6ef68dc", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "VBox(children=(Select(description='Query', layout=Layout(height='200px', padding='5pt', width='50%'), options=…" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "import ipywidgets as widgets\n", - "import difflib\n", - "\n", - "def browse_queries(queries: Dict[str, QueryProps]):\n", - " \"\"\"\n", - " Browse Hunting queries.\n", - " \n", - " Notes\n", - " -----\n", - " T\n", - " \"\"\"\n", - " select_query = widgets.Select(\n", - " description=\"Query\",\n", - " options=[(qry.name, idx) for idx, qry in queries.items()],\n", - " layout=widgets.Layout(height=\"200px\", width=\"50%\", padding=\"5pt\")\n", - " )\n", - " layout_query = lambda x, y: widgets.Layout(height=x, width=y, padding=\"5pt\")\n", - " layout_w = lambda x: widgets.Layout(width=x, padding=\"5pt\")\n", - " qry_view = widgets.Textarea(layout=layout_query(\"200px\", \"95%\"))\n", - " qry_view_repl = widgets.Textarea(layout=layout_query(\"200px\", \"95%\"))\n", - " qry_view_diff = widgets.Textarea(layout=layout_query(\"150px\", \"50%\"))\n", - " qry_file = widgets.Label(layout=layout_w(\"60%\"))\n", - " orig_lbl = widgets.Label(value=\"Original query\", layout=layout_w(\"60%\"))\n", - " mod_lbl = widgets.Label(value=\"Modified query\", layout=layout_w(\"60%\"))\n", - " vbox = widgets.VBox([\n", - " select_query,\n", - " qry_file,\n", - " widgets.HBox([\n", - " widgets.VBox([orig_lbl, qry_view], layout=layout_query(\"250px\", \"45%\")),\n", - " widgets.VBox([mod_lbl, qry_view_repl], layout=layout_query(\"250px\", \"45%\"))\n", - " ]),\n", - " qry_view_diff\n", - " ])\n", - "\n", - " def update_query(change):\n", - " query = queries[select_query.value]\n", - " qry_file.value = query.url\n", - " qry_view.value = query.raw_query\n", - " qry_view_repl.value = query.query\n", - " qry_view_diff.value = \"\\n\".join(difflib.unified_diff(qry_view.value.splitlines(), qry_view_repl.value.splitlines()))\n", - "\n", - " select_query.observe(update_query, names=\"value\")\n", - " update_query(None)\n", - " return vbox\n", - "\n", - "# Uncomment the follow line to browse the hunting queries\n", - "# browse_queries(hunting_queries)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Run Hunting queries for time range on risky accounts" - ] - }, - { - "cell_type": "code", - "execution_count": 15, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 21/21 [00:27<00:00, 1.33s/it]\n" - ] - } - ], - "source": [ - "\n", - "def run_ueba_queries(queries, start, end) -> pd.DataFrame:\n", - " dfs = []\n", - " query_params = {\"end\": end, \"start\": start}\n", - " for query in tqdm(queries.values()):\n", - " if \"UEBA\" not in query.url:\n", - " continue\n", - " try:\n", - " repl_query = query.query\n", - " if \"{start}\" in repl_query or \"{end}\" in repl_query:\n", - " try:\n", - " repl_query = repl_query.format(**query_params)\n", - " except KeyError:\n", - " print(f\"Format error: {query.name}\")\n", - " result_df = qry_prov.exec_query(repl_query)\n", - " result_df[\"UEBAQuery\"] = query.name\n", - " dfs.append(result_df)\n", - " except Exception as err:\n", - " print(\"Exception:\", type(err), query.name)\n", - " return pd.concat(dfs)\n", - "\n", - "ueba_df = run_ueba_queries(hunting_queries, start=start, end=end)" - ] - }, - { - "cell_type": "code", - "execution_count": 62, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
UEBA entries for unmitigated risk users
  UEBAEventCountStartTimeEndTime
UserPrincipalNameUEBAQuery   
tamuto@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity842023-01-31 00:54:39+00:002023-02-01 13:04:04+00:00
Anomalous action performed in tenant by privileged user12023-02-01 06:00:31+00:002023-02-01 06:00:31+00:00
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "ueba_summary = (\n", - " ueba_df[ueba_df[\"UserPrincipalName\"].str.lower().isin(risk_users_df.UserPrincipalName)]\n", - " .groupby([\"UserPrincipalName\", \"UEBAQuery\"])\n", - " .agg(\n", - " UEBAEventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", - " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", - " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", - " )\n", - ")\n", - "summary_report.add_summary_data(\n", - " data=ueba_summary.reset_index(),\n", - " user_column=\"UserPrincipalName\",\n", - " section=\"UEBA Summary\",\n", - ")\n", - "df_caption(\n", - " ueba_summary,\n", - " caption=\"UEBA entries for unmitigated risk users\"\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 61, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
UserPrincipalNameUEBAQueryUEBAEventCountStartTimeEndTime
0tamuto@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity842023-01-31 00:54:39+00:002023-02-01 13:04:04+00:00
1tamuto@seccxpninja.onmicrosoft.comAnomalous action performed in tenant by privileged user12023-02-01 06:00:31+00:002023-02-01 06:00:31+00:00
\n", - "
" - ], - "text/plain": [ - " UserPrincipalName \\\n", - "0 tamuto@seccxpninja.onmicrosoft.com \n", - "1 tamuto@seccxpninja.onmicrosoft.com \n", - "\n", - " UEBAQuery UEBAEventCount \\\n", - "0 Anomalous Sign-in Activity 84 \n", - "1 Anomalous action performed in tenant by privileged user 1 \n", - "\n", - " StartTime EndTime \n", - "0 2023-01-31 00:54:39+00:00 2023-02-01 13:04:04+00:00 \n", - "1 2023-02-01 06:00:31+00:00 2023-02-01 06:00:31+00:00 " - ] - }, - "execution_count": 61, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "ueba_summary.reset_index()" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
UEBA entries for mitigated risk users
  UEBAEventCountStartTimeEndTime
UserPrincipalNameUEBAQuery   
PDemo@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity10642023-01-31 06:41:25+00:002023-02-01 23:47:02+00:00
adm_pwatkins@seccxpninja.onmicrosoft.comAnomalies on users tagged as VIP332023-01-31 12:51:07+00:002023-01-31 13:17:32+00:00
Anomalous Sign-in Activity302023-01-31 12:59:44+00:002023-01-31 13:17:32+00:00
Anomalous login activity originated from Botnet, Tor proxy or C2252023-01-31 13:02:17+00:002023-01-31 13:17:32+00:00
dwilliams@seccxp.ninjaAnomalous Sign-in Activity32023-01-31 12:12:42+00:002023-02-01 00:53:23+00:00
pdemo@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity3692023-01-30 23:51:12+00:002023-01-31 16:27:02+00:00
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "df_caption(\n", - " ueba_df[ueba_df[\"UserPrincipalName\"].str.lower().isin(mitigated_users_df.UserPrincipalName)]\n", - " .groupby([\"UserPrincipalName\", \"UEBAQuery\"])\n", - " .agg(\n", - " UEBAEventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", - " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", - " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", - " ),\n", - " caption=\"UEBA entries for mitigated risk users\"\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Signin Summaries for prior week" - ] - }, - { - "cell_type": "code", - "execution_count": 18, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Sign-in summary for previous week
 ValuesNumUniqueValues
AttributeClientAppUserIPAddressLocationUserAgentClientAppUserIPAddressLocationUserAgent
UserPrincipalName        
tamuto@seccxpninja.onmicrosoft.com['Browser']['118.200.55.233' '20.25.98.192']['SG' 'US']['Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.70'\n", - " 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.61'\n", - " 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.70']1223
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "user_summary_query = \"\"\"\n", - "let si_history = SigninLogs\n", - "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", - "| where UserPrincipalName in~ ({users})\n", - "| summarize count() by UserPrincipalName, ResultType, RiskLevelAggregated, RiskLevelDuringSignIn, ClientAppUsed, UserAgent, IPAddress, Location;\n", - "si_history\n", - "| summarize OpCount=sum(count_) by UserPrincipalName, ClientAppUsed\n", - "| project UserPrincipalName, Attribute=\"ClientAppUser\", Value=ClientAppUsed, OpCount\n", - "| union ( \n", - "si_history\n", - "| summarize OpCount=sum(count_) by UserPrincipalName, IPAddress\n", - "| project UserPrincipalName, Attribute=\"IPAddress\", Value=IPAddress, OpCount\n", - ")\n", - "| union ( \n", - "si_history\n", - "| summarize OpCount=sum(count_) by UserPrincipalName, UserAgent\n", - "| project UserPrincipalName, Attribute=\"UserAgent\", Value=UserAgent, OpCount\n", - ")\n", - "| union ( \n", - "si_history\n", - "| summarize OpCount=sum(count_) by UserPrincipalName, Location\n", - "| project UserPrincipalName, Attribute=\"Location\", Value=Location, OpCount\n", - ")\n", - "\"\"\"\n", - "week_ago = (end - timedelta(7))\n", - "user_summary_df = qry_prov.exec_query(user_summary_query.format(\n", - " users=get_user_param(risk_users_df),\n", - " start=week_ago,\n", - " end=end\n", - "))\n", - "\n", - "summary_report.add_summary_data(\n", - " data=user_summary_df,\n", - " user_column=\"UserPrincipalName\",\n", - " report=\"Signin summary for previous week\"\n", - ")\n", - "df_caption(\n", - " user_summary_df.groupby([\"UserPrincipalName\", \"Attribute\"]).agg(\n", - " Values=pd.NamedAgg(\"Value\", \"unique\"),\n", - " NumUniqueValues=pd.NamedAgg(\"Value\", \"nunique\"),\n", - " OpCount=pd.NamedAgg(\"Value\", \"count\"),\n", - " )\n", - " .reset_index()\n", - " .pivot(index=['UserPrincipalName'], columns='Attribute', values=[\"Values\", \"NumUniqueValues\"]),\n", - " caption=\"Sign-in summary for previous week\"\n", - ")" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Related alerts\n", - "\n", - "## 1. Alerts that name the account explicitly" - ] - }, - { - "cell_type": "code", - "execution_count": 38, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 1/1 [00:02<00:00, 2.26s/it]\n" - ] - } - ], - "source": [ - "related_alerts_df = pd.concat([\n", - " (\n", - " qry_prov.SecurityAlert.list_related_alerts(account_name=acct)\n", - " .assign(UserPrincipalName=acct)\n", - " )\n", - " for acct in tqdm(risk_users_df.UserPrincipalName)\n", - "])\n", - "summary_report.add_summary_data(\n", - " data=related_alerts_df,\n", - " user_column=\"UserPrincipalName\",\n", - " section=\"Related alerts for user\"\n", - ")\n", - "df_caption(related_alerts_df.drop(\n", - " columns=[\"Description\", \"RemediationSteps\", \"ExtendedProperties\"]),\n", - " caption=\"Related alerts for account\")" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## 2. Alerts related to signin-in IP addresses" - ] - }, - { - "cell_type": "code", - "execution_count": 88, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "100%|██████████| 3/3 [00:07<00:00, 2.35s/it]\n" - ] - } - ], - "source": [ - "related_alerts_ip_df = pd.concat([\n", - " (\n", - " qry_prov.SecurityAlert.list_alerts_for_ip(source_ip_list=ip_addr)\n", - " .assign(UserPrincipalName=acct, IPAddress=ip_addr)\n", - " )\n", - " for acct, ip_addr in tqdm(\n", - " risk_users_df.explode(\"SourceIPs\")[[\"UserPrincipalName\", \"SourceIPs\"]].apply(tuple, axis=1)\n", - " )\n", - "])\n", - "summary_report.add_summary_data(\n", - " data=related_alerts_ip_df,\n", - " user_column=\"UserPrincipalName\",\n", - " section=\"Related alerts for user signin IP address\"\n", - ")\n", - "\n", - "df_caption(related_alerts_ip_df, caption=\"Related alerts for sign-in IP Address\")\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Threat Intelligence reports for sign-in IPs" - ] - }, - { - "cell_type": "code", - "execution_count": 82, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Observables processed: 100%|██████████| 6/6 [00:00<00:00, 600.07obs/s]\n" - ] - }, - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Threat intel reports for risky sign-in IPs
 UserPrincipalNameSourceIPsQuerySubtypeResultDetailsRawResultReferenceStatusIocIocTypeSafeIocSeverityProvider
2tamuto@seccxpninja.onmicrosoft.com20.25.98.192NoneTrue{'summary': {'resolutions': 0, 'certificates': 0, 'malware_hashes': 0, 'projects': 0, 'articles': 30, 'total': 30, 'netblock': '20.0.0.0/11', 'os': 'n/a', 'asn': 'AS8075 - MICROSOFT-CORP-MSN-AS-BLOCK', 'hosting_provider': 'n/a', 'link': 'https://community.riskiq.com/search/20.25.98.192', 'links': {'resolutions': 'https://community.riskiq.com/search/20.25.98.192/resolutions', 'services': 'https://community.riskiq.com/search/20.25.98.192/services', 'certificates': 'https://community.riskiq.com/search/20.25.98.192/certificates', 'projects': 'https://community.riskiq.com/search/20.25.98.192/projects', 'articles': 'https://community.riskiq.com/research?query=20.25.98.192', 'trackers': 'https://community.riskiq.com/search/20.25.98.192/trackers', 'components': 'https://community.riskiq.com/search/20.25.98.192/components', 'host_pairs': 'https://community.riskiq.com/search/20.25.98.192/hostpairs', 'reverse_dns': 'https://community.riskiq.com/search/20.25.98.192/dns', 'cookies': 'https://community.riskiq.com/search/20.25.98.192/cookies', 'malware_hashes': 'https://community.riskiq.com/search/20.25.98.192/hashes'}, 'services': 0}, 'reputation': {'score': 4, 'classification': 'UNKNOWN', 'rules': []}}{'summary': {'resolutions': 0, 'certificates': 0, 'malware_hashes': 0, 'projects': 0, 'articles': 30, 'total': 30, 'netblock': '20.0.0.0/11', 'os': 'n/a', 'asn': 'AS8075 - MICROSOFT-CORP-MSN-AS-BLOCK', 'hosting_provider': 'n/a', 'link': 'https://community.riskiq.com/search/20.25.98.192', 'links': {'resolutions': 'https://community.riskiq.com/search/20.25.98.192/resolutions', 'services': 'https://community.riskiq.com/search/20.25.98.192/services', 'certificates': 'https://community.riskiq.com/search/20.25.98.192/certificates', 'projects': 'https://community.riskiq.com/search/20.25.98.192/projects', 'articles': 'https://community.riskiq.com/research?query=20.25.98.192', 'trackers': 'https://community.riskiq.com/search/20.25.98.192/trackers', 'components': 'https://community.riskiq.com/search/20.25.98.192/components', 'host_pairs': 'https://community.riskiq.com/search/20.25.98.192/hostpairs', 'reverse_dns': 'https://community.riskiq.com/search/20.25.98.192/dns', 'cookies': 'https://community.riskiq.com/search/20.25.98.192/cookies', 'malware_hashes': 'https://community.riskiq.com/search/20.25.98.192/hashes'}, 'services': 0}, 'reputation': {'score': 4, 'classification': 'UNKNOWN', 'rules': []}}https://community.riskiq.com020.25.98.192ipv420.25.98.192highRiskIQ
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "# look up IP addresses - join UserPrincipalName from source DF to output\n", - "ti_user_ip = IpAddress.tilookup_ip(\n", - " risk_users_df.explode(\"SourceIPs\")[[\"UserPrincipalName\", \"SourceIPs\"]],\n", - " column=\"SourceIPs\",\n", - " join=\"left\"\n", - ").query(\"Severity != 'information'\")\n", - "\n", - "summary_report.add_summary_data(\n", - " data=ti_user_ip,\n", - " user_column=\"UserPrincipalName\",\n", - " section=\"Threat intel reports for user sign-in IP address(es)\"\n", - ")\n", - "\n", - "df_caption(ti_user_ip, caption=\"Threat intel reports for risky sign-in IPs\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Unusual Azure Audit entries\n", - "\n", - "Look for operations in Azure audit for selected accounts\n", - "where account used operations type in the current time slot that\n", - "it had not used in the baseline period (default prior 30 days)" - ] - }, - { - "cell_type": "code", - "execution_count": 20, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Azure audit activity types not seen in baseline period.
 IdentityUserPrincipalNameOperationNameLoggedByServiceInitiatedByAdditionalDetailsTargetResources
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "# Azure Audit\n", - "# Find any operation types for current period that weren't seen for\n", - "# that user in previous baseline period\n", - "azure_audit_query = \"\"\"\n", - "let start = datetime(\"{start}\");\n", - "let end = datetime(\"{end}\");\n", - "let baseline_start = start - ({baseline_period} * 1d);\n", - "let bl_threshold = {threshold};\n", - "let operation_history = AuditLogs\n", - "| where TimeGenerated between(baseline_start .. start)\n", - "| where Identity !in (\"Azure AD Cloud Sync\", \"Managed Service Identity\", \"Microsoft.Azure.SyncFabric\")\n", - "| where bag_has_key(InitiatedBy, \"user\")\n", - "| extend UserPrincipalName = tostring(InitiatedBy[\"user\"][\"userPrincipalName\"])\n", - "| where UserPrincipalName in~ ({users})\n", - "| summarize EventCount=count() by UserPrincipalName, OperationName\n", - "| where EventCount > bl_threshold;\n", - "AuditLogs\n", - "| where TimeGenerated between(end .. start)\n", - "| where Identity !in (\"Azure AD Cloud Sync\", \"Managed Service Identity\", \"Microsoft.Azure.SyncFabric\")\n", - "| where bag_has_key(InitiatedBy, \"user\")\n", - "| extend UserPrincipalName = tostring(InitiatedBy[\"user\"][\"userPrincipalName\"]), IPAddress = InitiatedBy[\"user\"][\"ipAddress\"]\n", - "| where UserPrincipalName in~ ({users})\n", - "| join kind=leftanti (operation_history) on UserPrincipalName, OperationName\n", - "| project Identity, UserPrincipalName, OperationName, LoggedByService, InitiatedBy, AdditionalDetails, TargetResources\n", - "\"\"\"\n", - "\n", - "end = datetime.now(tz=timezone.utc)\n", - "start = end-timedelta(1)\n", - "from datetime import datetime, timezone, timedelta\n", - "fmt_query = azure_audit_query.format(\n", - " start=start,\n", - " end=end,\n", - " baseline_period=baseline_period,\n", - " threshold=0,\n", - " users=get_user_param(risk_users_df),\n", - ")\n", - "az_audit_df = qry_prov.exec_query(fmt_query)\n", - "summary_report.add_summary_data(\n", - " data=az_audit_df,\n", - " user_column=\"UserPrincipalName\",\n", - " section=\"Unusual Azure Audit log entries for user\"\n", - ")\n", - "df_caption(az_audit_df, caption=\"Azure audit activity types not seen in baseline period.\")" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# New or unusual Office 365 activity\n", - "\n", - "Office operations occurring in the measured period that had\n", - "not occurred or rarely occurred in the baseline period." - ] - }, - { - "cell_type": "code", - "execution_count": 31, - "metadata": {}, - "outputs": [], - "source": [ - "o365_baseline_activity_query = \"\"\"\n", - "let num_stddev = {std_dev_scale};\n", - "let bl_period = datetime_add(\"day\", -{baseline_period}, datetime({start}));\n", - "OfficeActivity\n", - "| where TimeGenerated between (bl_period .. datetime({start}))\n", - "| where UserId in~ ({users})\n", - "// count operations by user and op type per day\n", - "| summarize OpCount = count() by UserId, OfficeWorkload, Operation, bin(TimeGenerated, 1d)\n", - "// calculate mean and average values for the user/op combos\n", - "| summarize OpStdev = stdev(OpCount), OpMean = avg(OpCount) by UserId, OfficeWorkload, Operation\n", - "// Calculate a baseline score Mean + N StdDevs * StdDev (default to 1 if 0 variance)\n", - "| extend OpBase = OpMean + (num_stddev * iif(OpStdev > 0, OpStdev, 1.0))\n", - "| extend RecType=\"baseline\"\n", - "\"\"\"\n", - "\n", - "o365_current_activity_query = \"\"\"\n", - "OfficeActivity\n", - "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", - "| where UserId in~ ({users})\n", - "| summarize OpCount = count() by UserId, OfficeWorkload, Operation\n", - "| extend RecType=\"current\"\n", - "\"\"\"\n", - "\n", - "# set number of std deviations from mean to use as indicating\n", - "# anomalous activity\n", - "_STD_THRESHOLD = 2\n", - "\n", - "end = datetime.now(tz=timezone.utc)\n", - "start = end - timedelta(1)\n", - "office_baseline_df = qry_prov.exec_query(\n", - " o365_baseline_activity_query.format(\n", - " users=get_user_param(risk_users_df),\n", - " std_dev_scale=_STD_THRESHOLD,\n", - " start=start,\n", - " baseline_period=baseline_period,\n", - " )\n", - ")\n", - "office_current_df = qry_prov.exec_query(\n", - " o365_current_activity_query.format(\n", - " users=get_user_param(risk_users_df),\n", - " start=start,\n", - " end=end\n", - " )\n", - ")\n", - "\n", - "# Pull out any current activity that exceeds the baseline threshold (mean + N*stddev)\n", - "office_activity_df = (\n", - " office_current_df\n", - " .merge(office_baseline_df, on=[\"UserId\", \"OfficeWorkload\", \"Operation\"], how=\"left\")\n", - " .fillna({\"OpBase\": 0})\n", - " .query(\"OpCount > OpBase\")\n", - ")" - ] - }, - { - "cell_type": "code", - "execution_count": 83, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Office baseline operations.
 UserIdOfficeWorkloadOperationOpStdevOpMeanOpBaseRecType
0tamuto@seccxpninja.onmicrosoft.comSharePointFileUploaded0.1889822.9642863.342250baseline
1tamuto@seccxpninja.onmicrosoft.comExchangeMailItemsAccessed2.5726292.2500007.395258baseline
2tamuto@seccxpninja.onmicrosoft.comExchangeCreate0.0000001.0000003.000000baseline
3tamuto@seccxpninja.onmicrosoft.comSharePointFileAccessed0.0000002.0000004.000000baseline
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Office current operations.
 UserIdOfficeWorkloadOperationOpCountRecType
0tamuto@seccxpninja.onmicrosoft.comExchangeMailItemsAccessed2current
1tamuto@seccxpninja.onmicrosoft.comSharePointFileUploaded3current
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - }, - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Office anomalous operations.
 UserIdOfficeWorkloadOperationOpCountRecType_xOpStdevOpMeanOpBaseRecType_y
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "df_caption(office_baseline_df, \"Office baseline operations.\")\n", - "df_caption(office_current_df, \"Office current operations.\")\n", - "df_caption(office_activity_df, \"Office anomalous operations.\")\n", - "summary_report.add_summary_data(\n", - " data=office_activity_df,\n", - " user_column=\"UserId\",\n", - " section=\"Unusual Office activity for user\"\n", - ")\n", - "summary_report.add_summary_data(\n", - " data=office_current_df,\n", - " user_column=\"UserId\",\n", - " section=\"Summarized current Office activity for user\"\n", - ")" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Unusual Azure activity\n", - "\n", - "Azure activity operations occurring in the measured period that had\n", - "not occurred in the baseline period.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 85, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "Index(['TimeGenerated', 'UserPrincipalName', 'OperationNameValue', 'IPAddress',\n", - " 'EventDataId', 'ActivityStatusValue', 'ResourceGroup', 'SubscriptionId',\n", - " 'TenantId'],\n", - " dtype='object')" - ] - }, - "execution_count": 85, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "azure_activity_df.columns" - ] - }, - { - "cell_type": "code", - "execution_count": 90, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
Azure activity operations not seen in baseline period.
 UserPrincipalNameOperationNameValueIPAddressResourceGroupSubscriptionIdTenantIdEventCountActivityStatusValueStartTimeEndTime
3tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/GETWORKERCOUNT/ACTION147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d10['Success' 'Start']2023-02-01 05:52:53.605463700+00:002023-02-01 06:00:53.770754+00:00
6tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/HYBRIDRUNBOOKWORKERS/WRITE147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 06:00:13.837572300+00:002023-02-01 06:00:15.603256700+00:00
12tamuto@seccxpninja.onmicrosoft.comMICROSOFT.COMPUTE/VIRTUALMACHINES/EXTENSIONS/WRITE147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Accept' 'Start']2023-02-01 06:00:15.803323200+00:002023-02-01 06:00:21.787995800+00:00
11tamuto@seccxpninja.onmicrosoft.comMICROSOFT.COMPUTE/VIRTUALMACHINES/EXTENSIONS/DELETE147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d7['Success' 'Accept' 'Start']2023-02-01 06:00:19.999565200+00:002023-02-01 06:12:47.627469900+00:00
5tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/HYBRIDRUNBOOKWORKERS/DELETE147.243.27.203SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d4['Success' 'Start' 'Failure']2023-02-01 06:00:52.906841700+00:002023-02-01 06:01:17.421008100+00:00
20tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/NETWORKINTERFACES/EFFECTIVENETWORKSECURITYGROUPS/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d18['Accept' 'Start' 'Success']2023-02-01 06:02:56.308497800+00:002023-02-01 12:52:20.724925900+00:00
24tamuto@seccxpninja.onmicrosoft.comMICROSOFT.RECOVERYSERVICES/LOCATIONS/BACKUPSTATUS/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d4['Success' 'Start']2023-02-01 06:03:00.808491100+00:002023-02-01 06:04:01.636515100+00:00
4tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/GETWORKERCOUNT/ACTION147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d10['Success' 'Start']2023-02-01 07:05:00.205528100+00:002023-02-01 10:46:02.598749700+00:00
7tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/HYBRIDRUNBOOKWORKERGROUPS/HYBRIDRUNBOOKWORKERS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 07:05:49.958148300+00:002023-02-01 07:05:52.100110600+00:00
13tamuto@seccxpninja.onmicrosoft.comMICROSOFT.COMPUTE/VIRTUALMACHINES/EXTENSIONS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d5['Success' 'Accept' 'Start']2023-02-01 07:05:52.301304600+00:002023-02-01 07:18:07.123937800+00:00
23tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/VIRTUALNETWORKS/SUBNETS/WRITE118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d3['Accept' 'Start' 'Success']2023-02-01 07:25:57.158919700+00:002023-02-01 07:26:03.513145200+00:00
25tamuto@seccxpninja.onmicrosoft.comMICROSOFT.RESOURCES/DEPLOYMENTS/VALIDATE/ACTION147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 07:26:03.099815300+00:002023-02-01 07:26:09.520994300+00:00
26tamuto@seccxpninja.onmicrosoft.comMICROSOFT.RESOURCES/DEPLOYMENTS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d3['Success' 'Accept' 'Start']2023-02-01 07:26:09.708168200+00:002023-02-01 07:37:00.044172400+00:00
22tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/PUBLICIPADDRESSES/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d5['Success' 'Accept' 'Start']2023-02-01 07:26:17.593352800+00:002023-02-01 07:38:34.764853600+00:00
19tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/BASTIONHOSTS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d10['Accept' 'Success' 'Start']2023-02-01 07:26:27.613222300+00:002023-02-01 07:48:47.545287400+00:00
29tamuto@seccxpninja.onmicrosoft.comMICROSOFT.SECURITYINSIGHTS/INCIDENTS/RUNPLAYBOOK/ACTION147.243.28.22SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d8['Success' 'Start']2023-02-01 07:51:57.616138800+00:002023-02-01 07:52:06.451008600+00:00
28tamuto@seccxpninja.onmicrosoft.comMICROSOFT.SECURITYINSIGHTS/INCIDENTS/RELATIONS/WRITE147.243.28.22SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d4['Success' 'Start']2023-02-01 07:52:29.689097400+00:002023-02-01 08:00:47.658756600+00:00
27tamuto@seccxpninja.onmicrosoft.comMICROSOFT.SECURITYINSIGHTS/INCIDENTS/RELATIONS/DELETE147.243.28.22SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d4['Success' 'Start']2023-02-01 07:55:50.427017+00:002023-02-01 07:59:57.879176500+00:00
18tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/BASTIONHOSTS/GETACTIVESESSIONS/ACTION118.200.55.233SOC-MDEd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d3['Accept' 'Start' 'Success']2023-02-01 09:15:05.655942+00:002023-02-01 09:15:12.692524100+00:00
17tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/BASTIONHOSTS/GETACTIVESESSIONS/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d14['Failure' 'Start' 'Success' 'Accept']2023-02-01 09:15:53.640419100+00:002023-02-01 12:51:56.844321800+00:00
9tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/RUNBOOKS/DRAFT/WRITE118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d27['Success' 'Accept' 'Start']2023-02-01 10:54:00.931669700+00:002023-02-01 11:03:34.187166400+00:00
10tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/RUNBOOKS/PUBLISH/ACTION147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d11['Accept' 'Start' 'Success']2023-02-01 10:54:01.203270+00:002023-02-01 11:04:06.589645400+00:00
14tamuto@seccxpninja.onmicrosoft.comMICROSOFT.LOGIC/WORKFLOWS/DISABLE/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 10:55:22.306755800+00:002023-02-01 10:55:24.103642+00:00
8tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTOMATION/AUTOMATIONACCOUNTS/JOBS/WRITE147.243.28.22SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d8['Accept' 'Start']2023-02-01 10:56:08.818195300+00:002023-02-01 11:19:46.546390100+00:00
15tamuto@seccxpninja.onmicrosoft.comMICROSOFT.LOGIC/WORKFLOWS/ENABLE/ACTION118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-01 11:30:14.904068+00:002023-02-01 11:30:19.279050800+00:00
16tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/BASTIONHOSTS/DELETE118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d5['Success' 'Accept' 'Start']2023-02-01 12:51:54.313056100+00:002023-02-01 13:10:14.653217100+00:00
21tamuto@seccxpninja.onmicrosoft.comMICROSOFT.NETWORK/NETWORKSECURITYGROUPS/SECURITYRULES/WRITE118.200.55.233SIMULANDd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d5['Success' 'Start' 'Accept']2023-02-01 12:52:39.782214+00:002023-02-01 12:52:43.287663600+00:00
2tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/WRITE147.243.27.237SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-02 03:31:40.786224200+00:002023-02-02 03:31:48.567780100+00:00
0tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/DELETE147.243.27.237SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-02 03:35:45.544154200+00:002023-02-02 03:35:50.481888+00:00
1tamuto@seccxpninja.onmicrosoft.comMICROSOFT.AUTHORIZATION/ROLEASSIGNMENTS/WRITE147.243.27.204SOCd1d8779d-38d7-4f06-91db-9cbc8de0176f8ecf8077-cf51-4820-aadd-14040956f35d2['Success' 'Start']2023-02-02 03:39:26.520506+00:002023-02-02 03:39:33.520717900+00:00
\n" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "# Azure Activity\n", - "azure_activity_query = \"\"\"\n", - "let start = datetime(\"{start}\");\n", - "let end = datetime(\"{end}\");\n", - "let baseline_start = start - ({period} * 1d);\n", - "let bl_threshold = {threshold};\n", - "let operation_history = AzureActivity\n", - "| where TimeGenerated between(baseline_start .. start)\n", - "| where Caller in~ ({users})\n", - "| project UserPrincipalName=Caller, OperationNameValue\n", - "| summarize EventCount=count() by UserPrincipalName, OperationNameValue\n", - "| where EventCount > bl_threshold;\n", - "AzureActivity\n", - "| where TimeGenerated between(start .. end)\n", - "| where Caller in~ ({users})\n", - "| project-rename UserPrincipalName=Caller\n", - "| join kind=leftanti (operation_history) on UserPrincipalName, OperationNameValue\n", - "| project TimeGenerated, UserPrincipalName, OperationNameValue, IPAddress=CallerIpAddress,\n", - " EventDataId, ActivityStatusValue, ResourceGroup, SubscriptionId, TenantId\n", - "\"\"\"\n", - "\n", - "fmt_query = azure_activity_query.format(\n", - " end=datetime.now(tz=timezone.utc),\n", - " start=end-timedelta(1),\n", - " period=28,\n", - " threshold=0,\n", - " users=get_user_param(risk_users_df),\n", - ")\n", - "azure_activity_df = qry_prov.exec_query(fmt_query)\n", - "\n", - "aa_summary_cols = [\n", - " \"UserPrincipalName\",\n", - " \"OperationNameValue\",\n", - " \"IPAddress\",\n", - " \"ResourceGroup\",\n", - " \"SubscriptionId\",\n", - " \"TenantId\",\n", - "]\n", - "\n", - "azure_activity_summary_df = azure_activity_df.groupby(aa_summary_cols).agg(\n", - " EventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", - " ActivityStatusValue=pd.NamedAgg(\"ActivityStatusValue\", \"unique\"),\n", - " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", - " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", - ").reset_index().sort_values(\"StartTime\", ascending=True)\n", - "\n", - "summary_report.add_summary_data(\n", - " data=azure_activity_summary_df,\n", - " user_column=\"UserPrincipalName\",\n", - " section=\"Unusual Azure activity for user\"\n", - ")\n", - "df_caption(azure_activity_summary_df, \"Azure activity operations not seen in baseline period.\")" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Summarizing data\n", - "\n", - "Create dynamic summaries for each user and upload to sentinel\n", - "\n", - "> Note: we could offer the option to group by report type instead\n", - "> of user. That would result in a Dynamic Summary entry for each\n", - "> report type (with consistent schema) but with data from (potentially)\n", - "> multiple users." - ] - }, - { - "cell_type": "code", - "execution_count": 110, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "AccountEvaluation - tamuto@seccxpninja.onmicrosoft.com\n" - ] - } - ], - "source": [ - "# Iterate through summary reports and create a summary for each user\n", - "\n", - "dynamic_summaries = []\n", - "for user, reports in summary_report._summary_reports.items():\n", - " # Create a summary for each user\n", - " user_ds = DynamicSummary(\n", - " summary_name=f\"AccountEvaluation - {user}\",\n", - " summary_description=\"Summary generated from AccountSignInEvaluation notebook.\",\n", - " source_info=\"AccountSignInEvaluation.ipynb\"\n", - " )\n", - "\n", - " for report_type, summary_item in reports.items():\n", - " ds_item_params = {\n", - " \"event_time_utc\": end,\n", - " \"search_key\": user,\n", - " \"observable_type\": \"report_type\",\n", - " \"observable_value\": report_type\n", - " }\n", - " user_ds.add_summary_items(\n", - " data=summary_item.data,\n", - " **ds_item_params\n", - " )\n", - " dynamic_summaries.append(user_ds)\n", - "\n", - "for dyn_summary in dynamic_summaries:\n", - " # Create or update the report\n", - " # sentinel.create_dynamic_summary(dyn_summary)\n", - " print(dyn_summary.summary_name)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Appendix - Pickling and restoring data" - ] - }, - { - "cell_type": "code", - "execution_count": 111, - "metadata": {}, - "outputs": [], - "source": [ - "import pickle\n", - "obj = pickle.dumps(dynamic_summaries)\n", - "\n", - "with open(\"acct_nb_summaries.pkl\", \"wb\") as pickle_file:\n", - " pickle_file.write(obj)" - ] - }, - { - "cell_type": "code", - "execution_count": 112, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "['AccountEvaluation - tamuto@seccxpninja.onmicrosoft.com']\n" - ] - } - ], - "source": [ - "# note - you need to have the DynamicSummary class imported\n", - "from msticpy.context.azure.sentinel_dynamic_summary import DynamicSummary\n", - "# defined (see earlier in the notebook) to successfully restore\n", - "# the summary report\n", - "with open(\"acct_nb_summaries.pkl\", \"rb\") as pickle_file:\n", - " summary_obj = pickle_file.read()\n", - " dynamic_summaries_copy = pickle.loads(obj)\n", - "\n", - "print([ds.summary_name for ds in dynamic_summaries_copy])" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "msticpy", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.9.7" - }, - "orig_nbformat": 4, - "vscode": { - "interpreter": { - "hash": "0f1a8e166ce5c1ec1911a36e4fdbd34b2f623e2a3442791008b8ac429a1d6070" - } - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/scenario-notebooks/Automated-Notebooks/AccountSignInEvaluation.ipynb b/scenario-notebooks/Automated-Notebooks/AccountSignInEvaluation.ipynb new file mode 100644 index 00000000..1bcd7d2d --- /dev/null +++ b/scenario-notebooks/Automated-Notebooks/AccountSignInEvaluation.ipynb @@ -0,0 +1,3123 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Unusual Account Activity\n", + "
\n", + "  Notebook Details...\n", + "\n", + " **Notebook Version:** 2.0
\n", + " **Python Version:** Python 3.8+
\n", + " **Required Packages**: msticpy, msticnb
\n", + "\n", + " **Data Sources Required**:\n", + " - Sentinel - SecurityAlert, SecurityEvent, HuntingBookmark, Syslog, AAD SigninLogs, AzureActivity, OfficeActivity, ThreatIndicator\n", + " - (Optional) - VirusTotal, AlienVault OTX, IBM XForce, Open Page Rank, (all require accounts and API keys)\n", + "
\n" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Notebook Overview\n", + "\n", + "The notebook uses Azure Active Directory risk assignment to identify\n", + "potentially risky user sign-ins. \n", + "\n", + "For each high risk sign-in that was not later marked as safe/mitigated,\n", + "additional data about that user account is collected and uploaded to an MS Sentinel \n", + "Dynamic Summary\n", + "\n", + "## Time ranges for the notebook\n", + "\n", + "The investigation time range is the previous 2 days using the notebook\n", + "run time as the origin time.\n", + "This can be overridden by notebook parameters.\n", + "The default baseline period is the 28 days prior to the investigation\n", + "time range.\n", + "\n", + "## Notebook Contents\n", + "\n", + "1. Notebook initialization and Connection\n", + "2. Get risk-flagged sign-ins (for primary period)\n", + "3. Get login risk level for baseline period\n", + "4. Retrieve and Run UEBA hunting queries on risk-flagged users \n", + "5. Get related alerts for users and user IPs\n", + "6. Get Threat Intelligence reports for sign-in IPs\n", + "7. Look for unusual Azure Audit entries\n", + "8. Look for unusual Office 365 activity\n", + "9. Look for unusual Azure activity\n", + "10. Summarize and upload data\n", + "\n", + "\n", + "## Output (dynamic summary):\n", + "- Dynamic Summary item for each user with additional data as item 3 above.\n", + "\n", + "\n", + "## Notebook parameters\n", + "\n", + "- `ws_name`: str - The MS Sentinel workspace name to query, default is \"Default\"\n", + "- `start`: datetime/datetime string - the start time of the investigation period\n", + "- `end`: datetime/datetime string - the start time of the investigation period\n", + "- `baseline_period`: int (days) - the number of days before `start` to use to\n", + " use for a baseline (comparison of current with previous behavior)\n" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "---\n", + "# 1. Notebook initialization\n", + "This should complete without errors. If you encounter errors or warnings look at the following notebooks:\n", + "\n", + "- Getting Started Notebook\n", + "- [TroubleShootingNotebooks](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/TroubleShootingNotebooks.ipynb)\n", + "- [ConfiguringNotebookEnvironment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)\n", + "\n", + "
\n", + "  Details...\n", + "The next cell:\n", + "- Checks for the correct Python version\n", + "- Checks versions and optionally installs required packages\n", + "- Imports the required packages into the notebook\n", + "- Sets a number of configuration options.\n", + "\n", + "If you are running in the Azure Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:\n", + "- [Getting Started](./A Getting Started Guide For Azure Sentinel ML Notebooks.ipynb)\n", + "- [Run TroubleShootingNotebooks](./TroubleShootingNotebooks.ipynb)\n", + "- [Run ConfiguringNotebookEnvironment](./ConfiguringNotebookEnvironment.ipynb)\n", + "\n", + "You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup. \n", + "There are more details about this in the `ConfiguringNotebookEnvironment` notebook and in these documents:\n", + "- [msticpy configuration](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)\n", + "- [Threat intelligence provider configuration](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file)\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "Attempting connection to Key Vault using cli credentials..." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "done
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from datetime import datetime, timedelta, timezone\n", + "\n", + "REQ_PYTHON_VER = \"3.8\"\n", + "REQ_MSTICPY_VER = \"2.3.0\"\n", + "\n", + "# %pip install --upgrade msticpy\n", + "\n", + "import msticpy as mp\n", + "mp.init_notebook()\n" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "tags": [ + "parameters" + ] + }, + "outputs": [], + "source": [ + "# papermill default parameters\n", + "ws_name = \"Default\"\n", + "end = datetime.now(timezone.utc)\n", + "start = end - timedelta(days=2)\n", + "baseline_period = 28\n", + "run_date = end" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Get Workspace and Authenticate\n", + "\n", + "
\n", + " Authentication help...\n", + " If you want to use a workspace other than one you have defined in your
\n", + "msticpyconfig.yaml create a connection string with your AAD TENANT_ID and
\n", + "your WORKSPACE_ID (these should both be quoted UUID strings).\n", + "\n", + "```python\n", + " workspace_cs = \"loganalytics://code().tenant('TENANT_ID').workspace('WORKSPACE_ID')\"\n", + "```\n", + "e.g.\n", + "```python\n", + " workspace_cs = \"loganalytics://code().tenant('c3de0f06-dcb8-40fb-9d1a-b62faea29d9d').workspace('c62d3dc5-11e6-4e29-aa67-eac88d5e6cf6')\"\n", + "```\n", + "Then in the Authentication cell replace\n", + "the call to `qry_prov.connect` with the following:\n", + "```python\n", + " qry_prov.connect(connect_str=workspace_cs)\n", + "```\n", + "The cell should now look like this:\n", + "\n", + "```python\n", + "...\n", + " # Authentication\n", + " qry_prov = QueryProvider(data_environment=\"MSSentinel\")\n", + " qry_prov.connect(connect_str=workspace_cs)\n", + "...\n", + "```\n", + "\n", + "On successful authentication you should see a ```popup schema``` button.\n", + "To find your Workspace Id go to [Log Analytics](https://ms.portal.azure.com/#blade/HubsExtension/Resources/resourceType/Microsoft.OperationalInsights%2Fworkspaces). Look at the workspace properties to find the ID.\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Configured workspaces: ASIHuntOMSWorkspaceV4, CCIS, Centrica, CyberSecuritySoc, Default, GovCyberSecuritySOC, NationalGrid, RedmondSentinelDemoEnvironment\n" + ] + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "1b48df6e81084894a51de0382633f31d", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Combobox(value='Default', description='Workspace Name', options=('ASIHuntOMSWorkspaceV4', 'CCIS', 'Centrica', …" + ] + }, + "execution_count": 3, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "print(\"Configured workspaces: \", \", \".join(msticpy.settings.get_config(\"AzureSentinel.Workspaces\").keys()))\n", + "import ipywidgets as widgets\n", + "ws_param = widgets.Combobox(\n", + " description=\"Workspace Name\",\n", + " value=ws_name,\n", + " options=list(msticpy.settings.get_config(\"AzureSentinel.Workspaces\").keys())\n", + ")\n", + "ws_param" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Please wait. Loading Kqlmagic extension...done\n", + "Connecting... " + ] + }, + { + "data": { + "text/html": [ + "\n", + " \n", + " \n", + " " + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "connected\n" + ] + }, + { + "data": { + "text/html": [ + "

" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "

Confirm time range to search

" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "03d5a74ade514734a1e4050baa4681db", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "VBox(children=(HTML(value='

Set query time boundaries

'), HBox(children=(DatePicker(value=datetime.date…" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "from msticpy.common.timespan import TimeSpan\n", + "from msticpy.context.azure import MicrosoftSentinel\n", + "\n", + "# Authentication\n", + "qry_prov = mp.QueryProvider(data_environment=\"MSSentinel\")\n", + "qry_prov.connect(workspace=ws_param.value)\n", + "\n", + "sentinel = MicrosoftSentinel(workspace=ws_param.value, connect=True)\n", + "\n", + "nb_timespan = TimeSpan(start, end)\n", + "qry_prov.query_time.timespan = nb_timespan\n", + "md(\"
\")\n", + "md(\"Confirm time range to search\", \"large, bold\")\n", + "qry_prov.query_time" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Function and class defintions used by the notebook" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [], + "source": [ + "import re\n", + "import urllib\n", + "from collections import namedtuple, defaultdict\n", + "from datetime import datetime, timedelta, timezone\n", + "from typing import Any, Dict, NamedTuple, Optional\n", + "\n", + "import httpx\n", + "import pandas as pd\n", + "import yaml\n", + "from tqdm.auto import tqdm\n", + "\n", + "from msticpy.context.azure.sentinel_dynamic_summary import DynamicSummary, DynamicSummaryItem\n", + "\n", + "\n", + "# Summary report classes\n", + "class SummaryItem(NamedTuple):\n", + " \"\"\"Data report collection for summary.\"\"\"\n", + " key: str\n", + " data: pd.DataFrame\n", + " properties: Dict[str, Any]\n", + "\n", + "\n", + "class SummaryReport:\n", + " \"\"\"Class to hold summary reports during exec of notebook.\"\"\"\n", + " def __init__(self):\n", + " self.summary_reports: Dict[str, Dict[str, SummaryItem]] = defaultdict(dict)\n", + "\n", + " def add_summary_data(self, data: pd.DataFrame, user_column: str, section: str, **kwargs):\n", + " \"\"\"Add data for users to the summary report\"\"\"\n", + " for user, user_data in data.groupby(user_column):\n", + " summary = SummaryItem(\n", + " key=user,\n", + " data=user_data,\n", + " properties=kwargs\n", + " )\n", + " self.summary_reports[user.casefold()][section] = summary\n", + "\n", + " @property\n", + " def users(self):\n", + " return sorted(self.summary_reports)\n", + "\n", + " @property\n", + " def report_types(self):\n", + " return sorted({\n", + " report for user_reports in self.summary_reports.values()\n", + " for report in user_reports\n", + " })\n", + "\n", + "\n", + "summary_report = SummaryReport()\n", + "\n", + "\n", + "# DF display function\n", + "def df_caption(data: pd.DataFrame, caption: str):\n", + " \"\"\"Display dataframe with a caption.\"\"\"\n", + " caption_css = \"; \".join([\n", + " \"caption-side: top\",\n", + " \"text-align: left\",\n", + " \"font-size: 15pt\",\n", + " \"font-weight: bold\",\n", + " \"padding: 5pt\",\n", + " ])\n", + " display(\n", + " data.style.set_caption(f\"{caption}\").set_table_styles(\n", + " [\n", + " {\n", + " \"selector\": \"caption\",\n", + " \"props\": caption_css,\n", + " }\n", + " ]\n", + " )\n", + " )\n", + "\n", + "\n", + "def get_user_param(data: pd.DataFrame) -> str:\n", + " \"\"\"Return user names from DataFrame as comma-sep string.\"\"\"\n", + " return \",\".join([\n", + " f\"'{user}'\" for user\n", + " in data.UserPrincipalName.values\n", + " ])\n", + "\n", + "\n", + "# update any changes to start/end datetimes\n", + "start = qry_prov.query_time.start\n", + "end = qry_prov.query_time.end" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 2. Get risk-flagged sign-ins\n", + "\n", + "This query retrieves user signins that have been flagged by Azure Identity Protection\n", + "as at risk. See [Azure Identity Protection](https://learn.microsoft.com/azure/active-directory/identity-protection/overview-identity-protection)\n", + "for more background." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Unmitigated risk users
 UserPrincipalName
0jank@seccxpninja.onmicrosoft.com
2aguruswamy@contosohotels.com
3suzanac@contosohotels.com
4asekstee@microsoft.com
5takuyaot@microsoft.com
6ragomeri@microsoft.com
7elsherif@microsoft.com
8aweinkopf@microsoft.com
9rickkotlarz@microsoft.com
10aauvinen@microsoft.com
11adstadel@microsoft.com
12malarkin@microsoft.com
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Mitigated risk users
 UserPrincipalName
1pdemo@seccxpninja.onmicrosoft.com
13adm_pwatkins@seccxpninja.onmicrosoft.com
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "signing_risk_query = \"\"\"\n", + "SigninLogs\n", + "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", + "| where RiskState != \"none\"\n", + "| project UserPrincipalName, ResultDescription, RiskState, RiskDetail, RiskEventTypes,\n", + " RiskEventTypes_V2, RiskLevelAggregated, RiskLevelDuringSignIn, IPAddress\n", + "| extend SigninRisk = case(\n", + " RiskLevelDuringSignIn == \"high\", 5,\n", + " RiskLevelDuringSignIn == \"medium\", 3,\n", + " RiskLevelDuringSignIn == \"low\", 1,\n", + " 0\n", + " ),\n", + " AggRisk = case(\n", + " RiskLevelAggregated == \"high\", 5,\n", + " RiskLevelAggregated == \"medium\", 3,\n", + " RiskLevelAggregated == \"low\", 1,\n", + " 0\n", + " )\n", + "| extend RiskEventDyn = parse_json(RiskEventTypes), RiskEventV2Dyn = parse_json(RiskEventTypes_V2)\n", + "| mv-expand RiskEventDyn, RiskEventV2Dyn\n", + "| summarize SignIns=count(AggRisk), MeanAggRisk=avg(AggRisk), MeanSigninRisk=avg(SigninRisk), \n", + " RiskStates=make_set(RiskState), RiskEvents=make_set(RiskEventDyn), RiskEventsV2=make_set(RiskEventV2Dyn),\n", + " SourceIPs=make_set(IPAddress)\n", + " by UserPrincipalName\n", + "| order by MeanAggRisk, MeanSigninRisk asc nulls last\n", + "\"\"\"\n", + "\n", + "# run the query\n", + "signin_risk_users_df = qry_prov.exec_query(\n", + " signing_risk_query.format(start=start, end=end)\n", + ")\n", + "# expand RiskStates (list)\n", + "risk_states_df = signin_risk_users_df.explode(\"RiskStates\")\n", + "# Extract list of users where risk was mitigated \n", + "safe_users_df = risk_states_df[risk_states_df[\"RiskStates\"].isin([\"remediated\", \"confirmedSafe\"])].UserPrincipalName.drop_duplicates()\n", + "\n", + "# Separate unmitigated from mitigated risk users\n", + "risk_users_df = signin_risk_users_df[~signin_risk_users_df[\"UserPrincipalName\"].isin(safe_users_df)]\n", + "mitigated_users_df = signin_risk_users_df[signin_risk_users_df[\"UserPrincipalName\"].isin(safe_users_df)]\n", + "\n", + "df_caption(risk_users_df[[\"UserPrincipalName\"]], \"Unmitigated risk users\")\n", + "df_caption(mitigated_users_df[[\"UserPrincipalName\"]], \"Mitigated risk users\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 3. Retrieve login risk level for baseline period\n", + "\n", + "This is used to distinguish accounts that have a new \"At Risk\"\n", + "designation from those accounts that have a history of risk signins.\n", + "\n", + "> Note: The period used is the `baseline_period` parameter for the notebook - default is 28 days" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Signing Summary for users with unmitigated risk" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Sign-in risk summary - unmitigated
 UserPrincipalNameSignInsMeanAggRiskMeanSigninRiskRiskStatesRiskEventsRiskEventsV2SourceIPsRiskHistory
0jank@seccxpninja.onmicrosoft.com13.0000000.000000['atRisk'][]['newCountry']['73.109.22.203']New
2aguruswamy@contosohotels.com11.0000000.000000['atRisk'][]['newCountry']['67.168.169.80']New
3suzanac@contosohotels.com11.0000000.000000['atRisk'][]['newCountry']['50.47.87.74']New
4asekstee@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['144.134.106.54']New
5takuyaot@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['153.240.206.142']New
6ragomeri@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['167.220.197.42']New
7elsherif@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['94.128.105.93']New
8aweinkopf@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['167.220.196.39']New
9rickkotlarz@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['45.22.1.222']New
10aauvinen@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['167.220.197.91']New
11adstadel@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['85.7.181.19']New
12malarkin@microsoft.com10.0000000.000000['dismissed'][]['newCountry']['75.136.202.123']New
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "_AADSIL_DISPLAY_COLUMNS = [\n", + " 'TimeGenerated', 'ResultType', 'ResultDescription', 'UserPrincipalName', 'UserId',\n", + " 'Location', 'IPAddress', 'AppDisplayName', 'ClientAppUsed', 'AppId',\n", + " 'AuthenticationDetails', 'AuthenticationMethodsUsed',\n", + " 'RiskEventTypes', 'RiskEventTypes_V2', 'RiskLevelAggregated',\n", + " 'RiskLevelDuringSignIn', 'RiskState', 'ResourceDisplayName',\n", + " 'LocationDetails', 'MfaDetail', 'NetworkLocationDetails',\n", + " 'UserAgent', 'UserDisplayName', 'UserType', 'IPAddressFromResourceProvider',\n", + " 'ResourceTenantId', 'HomeTenantId', 'AutonomousSystemNumber', 'Type'\n", + "]\n", + "\n", + "\n", + "# Function to summarize the history data\n", + "def weekly_signin_summary(data) -> pd.DataFrame:\n", + " \"\"\"Create signin summary from historical data.\"\"\"\n", + " return (\n", + " data\n", + " [_AADSIL_DISPLAY_COLUMNS]\n", + " .explode([\"RiskEventTypes\"])\n", + " .groupby([\"UserPrincipalName\", pd.Grouper(key=\"TimeGenerated\", freq=\"W\")])\n", + " .agg(\n", + " LoginCount=pd.NamedAgg(\"ResultType\", \"count\"),\n", + " ResultTypes=pd.NamedAgg(\"ResultType\", \"unique\"),\n", + " RiskEventTypes=pd.NamedAgg(\"RiskEventTypes\", \"unique\"),\n", + " RiskLevels=pd.NamedAgg(\"RiskLevelAggregated\", \"unique\"),\n", + " RiskLevelSignins=pd.NamedAgg(\"RiskLevelDuringSignIn\", \"unique\"),\n", + " IPs=pd.NamedAgg(\"IPAddress\", \"nunique\"),\n", + " Locations=pd.NamedAgg(\"Location\", \"nunique\"),\n", + " Apps=pd.NamedAgg(\"AppDisplayName\", \"nunique\"),\n", + " UserAgents=pd.NamedAgg(\"UserAgent\", \"nunique\"),\n", + " StartDate=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", + " EndDate=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", + " )\n", + " .sort_index()\n", + " )\n", + "\n", + "\n", + "# Get historical risk level for previous {period} days\n", + "risk_hist_query = \"\"\"\n", + "let q_end = datetime({start});\n", + "let q_start = datetime_add(\"day\", -{baseline_period}, q_end);\n", + "SigninLogs\n", + "| where TimeGenerated between (q_start .. q_end)\n", + "| where RiskState != \"none\"\n", + "| where UserPrincipalName in ({users})\n", + "| extend RiskEventTypes = parse_json(RiskEventTypes),\n", + " RiskEventTypes_V2 = parse_json(RiskEventTypes_V2)\n", + "\"\"\"\n", + "\n", + "if risk_users_df.empty:\n", + " raise LookupError(\n", + " \"No user logins with unmitigated risk flag found for period.\",\n", + " \"Exiting notebook.\"\n", + " )\n", + "\n", + "# Unmitigated risk users\n", + "risk_user_hist_df = qry_prov.exec_query(\n", + " risk_hist_query.format(\n", + " users=get_user_param(risk_users_df),\n", + " start=start,\n", + " baseline_period=baseline_period,\n", + " )\n", + ")\n", + "\n", + "risk_users_history = weekly_signin_summary(risk_user_hist_df).reset_index()\n", + "\n", + "# Isolate users that have no history of risk in previous period\n", + "users_with_past_risk_criteria = risk_users_df.UserPrincipalName.isin(risk_user_hist_df.UserPrincipalName.unique())\n", + "risk_users_df = risk_users_df.copy()\n", + "risk_users_df.loc[~users_with_past_risk_criteria, \"RiskHistory\"] = \"New\"\n", + "risk_users_df.loc[users_with_past_risk_criteria, \"RiskHistory\"] = \"Existing\"\n", + "\n", + "summary_report.add_summary_data(\n", + " data=risk_users_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Risk Users Summary\",\n", + ")\n", + "summary_report.add_summary_data(\n", + " data=risk_users_history,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Risk Users History\",\n", + ")\n", + "\n", + "df_caption(risk_users_df, \"Sign-in risk summary - unmitigated\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Signing Summary for users with mitigated risk\n", + "### [info only]" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Sign-in risk summary - mitigated
 UserPrincipalNameSignInsMeanAggRiskMeanSigninRiskRiskStatesRiskEventsRiskEventsV2SourceIPsRiskHistory
1pdemo@seccxpninja.onmicrosoft.com1481.3851353.878378['confirmedSafe', 'atRisk', 'dismissed', 'confirmedCompromised']['unfamiliarFeatures', 'unlikelyTravel']['unfamiliarFeatures', 'unlikelyTravel']['195.200.70.38', '182.1.122.187', '174.49.144.40', '187.145.130.238', '213.180.18.170', '86.188.84.35', '178.15.174.19', '114.4.214.32', '180.178.100.70', '89.64.60.74', '201.191.51.20', '140.186.246.113', '86.136.20.241', '212.81.187.84', '96.234.155.228', '81.228.197.31', '194.69.103.247', '147.161.137.90', '91.37.88.113', '194.69.103.19', '112.201.164.139', '165.225.112.139', '101.180.77.136', '122.161.73.80', '46.223.162.163', '95.130.222.34', '31.223.2.253', '167.220.197.43', '108.218.142.243', '136.226.252.97', '134.238.224.48', '169.159.144.109', '194.69.103.115', '13.79.0.98', '31.160.80.18', '194.69.103.94', '167.220.197.108', '92.97.154.99', '94.15.56.164', '195.146.138.78', '31.168.52.224', '213.123.211.228', '189.128.102.186', '136.35.205.71', '167.220.197.233', '194.69.103.30', '194.69.103.141', '194.69.103.187']Existing
13adm_pwatkins@seccxpninja.onmicrosoft.com90.0000005.000000['remediated']['anonymizedIPAddress']['anonymizedIPAddress']['185.220.102.251', '192.42.116.17', '171.25.193.78']Existing
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# History of mitigated risk users\n", + "if not mitigated_users_df.empty:\n", + " mit_risk_user_hist_df = qry_prov.exec_query(\n", + " risk_hist_query.format(\n", + " users=get_user_param(mitigated_users_df),\n", + " start=start,\n", + " baseline_period=baseline_period\n", + " )\n", + " )\n", + "\n", + "\n", + " # Isolate users that have no history of risk in previous period\n", + " users_with_past_risk_criteria = mitigated_users_df.UserPrincipalName.isin(mit_risk_user_hist_df.UserPrincipalName.unique())\n", + " mitigated_users_df = mitigated_users_df.copy()\n", + " mitigated_users_df.loc[~users_with_past_risk_criteria, \"RiskHistory\"] = \"New\"\n", + " mitigated_users_df.loc[users_with_past_risk_criteria, \"RiskHistory\"] = \"Existing\"\n", + " mitigated_users_df\n", + " df_caption(mitigated_users_df, \"Sign-in risk summary - mitigated\")\n", + "else:\n", + " md(\"No users with mitigated risk in this period.\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 4. Retrieve and Run UEBA hunting queries on risk-flagged users\n", + "\n", + "> UEBA = User Entity Behavior Analytics\n", + "\n", + "The next cell retrieves the current UEBA hunting\n", + "queries and runs them against the risk-flagged users.\n", + "\n", + "For more information see [Microsoft Sentinel UEBA](https://learn.microsoft.com/azure/sentinel/identify-threats-with-entity-behavior-analytics)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 21/21 [00:04<00:00, 4.61it/s]\n" + ] + }, + { + "data": { + "text/html": [ + "
Anomalies on users tagged as VIPhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/anomaliesOnVIPUsers.yaml
Anomalous AAD Account Manipulationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20AAD%20Account%20Manipulation.yaml
Anomalous AAD Account Creationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Account%20Creation.yaml
Anomalous Activity Role Assignmenthttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Activity%20Role%20Assignment.yaml
Anomalous Code Executionhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Code%20Execution.yaml
Anomalous Data Accesshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Data%20Access.yaml
Anomalous Defensive Mechanism Modificationhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Defensive%20Mechanism%20Modification.yaml
Anomalous Failed Logonhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Failed%20Logon.yaml
Anomalous Geo Location Logonhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Geo%20Location%20Logon.yaml
Anomalous Login to Deviceshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Login%20to%20Devices.yaml
Anomalous Password Resethttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Password%20Reset.yaml
Anomalous RDP Activityhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20RDP%20Activity.yaml
Anomalous Resource Accesshttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Resource%20Access.yaml
Anomalous Role Assignmenthttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Role%20Assignment.yaml
Anomalous Sign-in Activityhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/Anomalous%20Sign-in%20Activity.yaml
Anomalous action performed in tenant by privileged userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/anomalousActionInTenant.yaml
Dormant account activity from uncommon countryhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/dormantAccountActivityFromUncommonCountry.yaml
Anomalous connection from highly privileged userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/firstConnectionFromGroup.yaml
Anomalous login activity originated from Botnet, Tor proxy or C2https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/loginActivityFromBotnet.yaml
New account added to admin grouphttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/newAccountAddedToAdminGroup.yaml
Anomalous update Key Vault activity by high blast radius userhttps://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Solutions/UEBA%20Essentials/Hunting%20Queries/updateKeyVaultActivity.yaml
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Hunting Queries\n", + "_SENTINEL_REPO = \"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master\"\n", + "_SI_LOG_ROOT = f\"{_SENTINEL_REPO}/Hunting%20Queries/SigninLogs\"\n", + "_GEN_HUNTING_QRY = [\n", + " # \"AnomalousUserAppSigninLocationIncreaseDetail.yaml\",\n", + " # \"LegacyAuthAttempt.yaml\",\n", + " # \"Signins-From-VPS-Providers.yaml\",\n", + " # \"UserAccountsMeasurableincreaseofsuccessfulsignins.yaml\",\n", + " # \"riskSignInWithNewMFAMethod.yaml\",\n", + " # \"signinBurstFromMultipleLocations.yaml\",\n", + "]\n", + "\n", + "# UEBA Hunting Queries\n", + "_UEBA_HQ_ROOT = f\"{_SENTINEL_REPO}/Solutions/UEBA%20Essentials/Hunting%20Queries\"\n", + "_UEBA_HUNTING_QRY = [\n", + " \"anomaliesOnVIPUsers.yaml\",\n", + " \"Anomalous AAD Account Manipulation.yaml\",\n", + " \"Anomalous Account Creation.yaml\",\n", + " \"Anomalous Activity Role Assignment.yaml\",\n", + " \"Anomalous Code Execution.yaml\",\n", + " \"Anomalous Data Access.yaml\",\n", + " \"Anomalous Defensive Mechanism Modification.yaml\",\n", + " \"Anomalous Failed Logon.yaml\",\n", + " \"Anomalous Geo Location Logon.yaml\",\n", + " \"Anomalous Login to Devices.yaml\",\n", + " \"Anomalous Password Reset.yaml\",\n", + " \"Anomalous RDP Activity.yaml\",\n", + " \"Anomalous Resource Access.yaml\",\n", + " \"Anomalous Role Assignment.yaml\",\n", + " \"Anomalous Sign-in Activity.yaml\",\n", + " \"anomalousActionInTenant.yaml\",\n", + " \"dormantAccountActivityFromUncommonCountry.yaml\",\n", + " \"firstConnectionFromGroup.yaml\",\n", + " \"loginActivityFromBotnet.yaml\",\n", + " \"newAccountAddedToAdminGroup.yaml\",\n", + " # \"terminatedEmployeeAccessHVA.yaml\",\n", + " # \"terminatedEmployeeActivity.yaml\",\n", + " \"updateKeyVaultActivity.yaml\",\n", + "]\n", + "\n", + "ALL_QUERIES = {qry: _SI_LOG_ROOT for qry in _GEN_HUNTING_QRY}\n", + "ALL_QUERIES.update({qry: _UEBA_HQ_ROOT for qry in _UEBA_HUNTING_QRY})\n", + "\n", + "TIME_TOKEN = re.compile(r\"(\\{\\{StartTimeISO\\}\\}|\\{\\{EndTimeISO\\}\\})\")\n", + "_LEFT_BRACE = r\"[^{](\\{)[^{]\"\n", + "_RIGHT_BRACE = r\"[^}](\\})[^}]\"\n", + "_LB_TOKEN = \"%%~[~%%\"\n", + "_RB_TOKEN = \"%%~]~%%\"\n", + "\n", + "\n", + "def replace_time_params(query):\n", + " repl_query = re.sub(_LEFT_BRACE, _LB_TOKEN, query)\n", + " repl_query = re.sub(_RIGHT_BRACE, _RB_TOKEN, repl_query)\n", + " repl_query = repl_query.replace(\"{{StartTimeISO}}\", \"{start}\").replace(\"{{EndTimeISO}}\", \"{end}\")\n", + " return repl_query.replace(_LB_TOKEN, \"{{\").replace(_RB_TOKEN, \"}}\")\n", + "\n", + "\n", + "QueryProps = namedtuple(\"QueryProps\", \"name, query, req_time, description, url, raw_query\")\n", + "\n", + "\n", + "def fetch_queries(query_dict: Dict[str, str], verbose: bool = False) -> Dict[str, QueryProps]:\n", + " \"\"\"Fetch queries from Sentinel GitHub repo.\"\"\"\n", + " discover_queries: Dict[str, QueryProps] = {}\n", + " error_queries: Dict[str, str] = {}\n", + " for query, path in tqdm(query_dict.items()):\n", + " q_path = f\"{path}/{urllib.parse.quote(query)}\"\n", + " resp = httpx.get(q_path)\n", + " if resp.status_code != 200:\n", + " print(f\"invalid URL {path}\")\n", + " continue\n", + " try:\n", + " q_dict = yaml.safe_load(resp.content)\n", + " except yaml.scanner.ScannerError as err:\n", + " print(f\"could not parse query {query} at {q_path}\")\n", + " error_queries[query] = resp.content\n", + " continue\n", + "\n", + " query_text = q_dict.get(\"query\")\n", + " req_time = False\n", + " if re.search(TIME_TOKEN, query_text):\n", + " query_text = replace_time_params(query_text)\n", + " req_time = True\n", + "\n", + " if \"UEBA\" in path:\n", + " query_text = add_ueba_time_params(query_text)\n", + " if verbose:\n", + " print(f\"Query {query}, {q_dict['name']}, req time: {req_time}\")\n", + " discover_queries[query] = QueryProps(\n", + " name=q_dict.get(\"name\"),\n", + " query=query_text,\n", + " req_time=req_time,\n", + " description=q_dict.get(\"description\"),\n", + " url=q_path,\n", + " raw_query=q_dict.get(\"query\"),\n", + " )\n", + " return discover_queries\n", + "\n", + "\n", + "PRIM_TABLE_EXP = r\"(?P^|\\n)(?P(BehaviorAnalytics|AuditLogs|IdentityInfo|SigninLogs))(?=[\\s\\n\\)\\|])\"\n", + "PRIM_TABLE_REPL = r\"\\g\\g
\\n| where TimeGenerated > datetime({start})\"\n", + "JOIN_TABLE_EXP = r\"(?P\\|\\s+join[^(]*\\(\\s*[^\\s]+)(?=[\\s\\)\\|])\"\n", + "JOIN_TABLE_REPL = r\"\\g\\n| where TimeGenerated > datetime({start})\"\n", + "\n", + "\n", + "def add_ueba_time_params(query):\n", + " if isinstance(query, tuple):\n", + " if query.req_time:\n", + " return query.query\n", + " query = query.query\n", + " return re.sub(\n", + " JOIN_TABLE_EXP,\n", + " JOIN_TABLE_REPL,\n", + " re.sub(PRIM_TABLE_EXP, PRIM_TABLE_REPL, query)\n", + " )\n", + "\n", + "\n", + "def display_query_table(queries):\n", + " ht_table = \"
{rows}
\"\n", + " rows = [f\"{q.name}{q.url}\"\n", + " for q in queries.values()]\n", + " from IPython.display import HTML\n", + " display(HTML(ht_table.format(rows=\"\".join(rows))))\n", + "\n", + "\n", + "hunting_queries = fetch_queries(ALL_QUERIES)\n", + "\n", + "display_query_table(hunting_queries)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Browser for UEBA queries - not used in notebook" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "import ipywidgets as widgets\n", + "import difflib\n", + "\n", + "def browse_queries(queries: Dict[str, QueryProps]):\n", + " \"\"\"\n", + " Browse Hunting queries.\n", + " \n", + " Notes\n", + " -----\n", + " T\n", + " \"\"\"\n", + " select_query = widgets.Select(\n", + " description=\"Query\",\n", + " options=[(qry.name, idx) for idx, qry in queries.items()],\n", + " layout=widgets.Layout(height=\"200px\", width=\"50%\", padding=\"5pt\")\n", + " )\n", + " layout_query = lambda x, y: widgets.Layout(height=x, width=y, padding=\"5pt\")\n", + " layout_w = lambda x: widgets.Layout(width=x, padding=\"5pt\")\n", + " qry_view = widgets.Textarea(layout=layout_query(\"200px\", \"95%\"))\n", + " qry_view_repl = widgets.Textarea(layout=layout_query(\"200px\", \"95%\"))\n", + " qry_view_diff = widgets.Textarea(layout=layout_query(\"150px\", \"50%\"))\n", + " qry_file = widgets.Label(layout=layout_w(\"60%\"))\n", + " orig_lbl = widgets.Label(value=\"Original query\", layout=layout_w(\"60%\"))\n", + " mod_lbl = widgets.Label(value=\"Modified query\", layout=layout_w(\"60%\"))\n", + " vbox = widgets.VBox([\n", + " select_query,\n", + " qry_file,\n", + " widgets.HBox([\n", + " widgets.VBox([orig_lbl, qry_view], layout=layout_query(\"250px\", \"45%\")),\n", + " widgets.VBox([mod_lbl, qry_view_repl], layout=layout_query(\"250px\", \"45%\"))\n", + " ]),\n", + " qry_view_diff\n", + " ])\n", + "\n", + " def update_query(change):\n", + " query = queries[select_query.value]\n", + " qry_file.value = query.url\n", + " qry_view.value = query.raw_query\n", + " qry_view_repl.value = query.query\n", + " qry_view_diff.value = \"\\n\".join(difflib.unified_diff(qry_view.value.splitlines(), qry_view_repl.value.splitlines()))\n", + "\n", + " select_query.observe(update_query, names=\"value\")\n", + " update_query(None)\n", + " return vbox\n", + "\n", + "# Uncomment the follow line to browse the hunting queries\n", + "# browse_queries(hunting_queries)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Run Hunting queries for time range on risky accounts" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Running 21 queries...\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 21/21 [00:32<00:00, 1.55s/it]\n" + ] + } + ], + "source": [ + "\n", + "def run_ueba_queries(queries, start, end) -> pd.DataFrame:\n", + " dfs = []\n", + " query_params = {\"end\": end, \"start\": start}\n", + " print(f\"Running {len(queries)} queries...\")\n", + " for query in tqdm(queries.values()):\n", + " if \"UEBA\" not in query.url:\n", + " continue\n", + " try:\n", + " repl_query = query.query\n", + " if \"{start}\" in repl_query or \"{end}\" in repl_query:\n", + " try:\n", + " repl_query = repl_query.format(**query_params)\n", + " except KeyError:\n", + " print(f\"Format error: {query.name}\")\n", + " result_df = qry_prov.exec_query(repl_query)\n", + " result_df[\"UEBAQuery\"] = query.name\n", + " dfs.append(result_df)\n", + " except Exception as err:\n", + " print(\"Exception:\", type(err), query.name)\n", + " return pd.concat(dfs)\n", + "\n", + "ueba_df = run_ueba_queries(hunting_queries, start=start, end=end)" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
UEBA entries for unmitigated risk users
  UEBAEventCountStartTimeEndTime
UserPrincipalNameUEBAQuery   
aguruswamy@contosohotels.comAnomalous Sign-in Activity102023-03-21 19:25:49+00:002023-03-21 19:29:54+00:00
elsherif@microsoft.comAnomalous Sign-in Activity22023-03-22 08:27:19+00:002023-03-22 08:27:19+00:00
jank@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity42023-03-21 23:09:09+00:002023-03-21 23:26:52+00:00
suzanac@contosohotels.comAnomalous Sign-in Activity132023-03-22 17:35:16+00:002023-03-22 17:41:53+00:00
takuyaot@microsoft.comAnomalous Sign-in Activity12023-03-22 07:19:31+00:002023-03-22 07:19:31+00:00
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "ueba_summary = (\n", + " ueba_df[ueba_df[\"UserPrincipalName\"].str.lower().isin(risk_users_df.UserPrincipalName)]\n", + " .groupby([\"UserPrincipalName\", \"UEBAQuery\"])\n", + " .agg(\n", + " UEBAEventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", + " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", + " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", + " )\n", + ")\n", + "summary_report.add_summary_data(\n", + " data=ueba_summary.reset_index(),\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"UEBA Summary\",\n", + ")\n", + "df_caption(\n", + " ueba_summary,\n", + " caption=\"UEBA entries for unmitigated risk users\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
UEBA entries for mitigated risk users
  UEBAEventCountStartTimeEndTime
UserPrincipalNameUEBAQuery   
PDemo@seccxpninja.onmicrosoft.comAnomalous Sign-in Activity14762023-03-20 19:48:55+00:002023-03-22 18:48:58+00:00
adm_pwatkins@seccxpninja.onmicrosoft.comAnomalies on users tagged as VIP92023-03-22 13:48:17.600813400+00:002023-03-22 13:50:40.154887600+00:00
Anomalous Sign-in Activity92023-03-22 13:48:17.600813400+00:002023-03-22 13:50:40.154887600+00:00
Anomalous login activity originated from Botnet, Tor proxy or C232023-03-22 13:48:35.564164900+00:002023-03-22 13:50:10.714615400+00:00
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "df_caption(\n", + " ueba_df[ueba_df[\"UserPrincipalName\"].str.lower().isin(mitigated_users_df.UserPrincipalName)]\n", + " .groupby([\"UserPrincipalName\", \"UEBAQuery\"])\n", + " .agg(\n", + " UEBAEventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", + " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", + " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", + " ),\n", + " caption=\"UEBA entries for mitigated risk users\"\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Signin Summaries for prior week\n", + "\n", + "Collect distinct locations, IP addresses, client apps \n", + "and User agent strings used in user sign-ins" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
IdUserPrincipalNameIsInteractive
021e3cf35-1191-44a2-922d-927718699800v-rniteesh@microsoft.comTrue
11d52279e-d326-471a-a761-a5b59cab3a00joanne.sensitive@contosohotels.comTrue
250644068-10ed-4dd3-94db-6778ed4a3200pdemo@seccxpninja.onmicrosoft.comTrue
3bb29c897-581e-4d60-9db2-08eeb032b200pdemo@seccxpninja.onmicrosoft.comTrue
475438e12-2502-41fd-93b5-c26d532e2100pdemo@seccxpninja.onmicrosoft.comTrue
............
98616cbdc387-bf42-474f-9a0e-eb6914d13b00sync_ninja-dc_9d913db9dfd8@seccxpninja.onmicrosoft.comTrue
9862ef4c9ef1-d46c-43a2-b65e-53051a246900sync_ninja-dc_9d913db9dfd8@seccxpninja.onmicrosoft.comTrue
9863a4454e8e-c501-470d-a2a8-9c2102971500sync_ninja-dc_9d913db9dfd8@seccxpninja.onmicrosoft.comTrue
986449affc23-1178-40db-b537-423e6d5b8900joanne.sensitive@contosohotels.comTrue
98653f1cce83-90d1-46c7-8a89-35a4dcce7c00chstelze@microsoft.comTrue
\n", + "

9866 rows × 3 columns

\n", + "
" + ], + "text/plain": [ + " Id \\\n", + "0 21e3cf35-1191-44a2-922d-927718699800 \n", + "1 1d52279e-d326-471a-a761-a5b59cab3a00 \n", + "2 50644068-10ed-4dd3-94db-6778ed4a3200 \n", + "3 bb29c897-581e-4d60-9db2-08eeb032b200 \n", + "4 75438e12-2502-41fd-93b5-c26d532e2100 \n", + "... ... \n", + "9861 6cbdc387-bf42-474f-9a0e-eb6914d13b00 \n", + "9862 ef4c9ef1-d46c-43a2-b65e-53051a246900 \n", + "9863 a4454e8e-c501-470d-a2a8-9c2102971500 \n", + "9864 49affc23-1178-40db-b537-423e6d5b8900 \n", + "9865 3f1cce83-90d1-46c7-8a89-35a4dcce7c00 \n", + "\n", + " UserPrincipalName IsInteractive \n", + "0 v-rniteesh@microsoft.com True \n", + "1 joanne.sensitive@contosohotels.com True \n", + "2 pdemo@seccxpninja.onmicrosoft.com True \n", + "3 pdemo@seccxpninja.onmicrosoft.com True \n", + "4 pdemo@seccxpninja.onmicrosoft.com True \n", + "... ... ... \n", + "9861 sync_ninja-dc_9d913db9dfd8@seccxpninja.onmicrosoft.com True \n", + "9862 sync_ninja-dc_9d913db9dfd8@seccxpninja.onmicrosoft.com True \n", + "9863 sync_ninja-dc_9d913db9dfd8@seccxpninja.onmicrosoft.com True \n", + "9864 joanne.sensitive@contosohotels.com True \n", + "9865 chstelze@microsoft.com True \n", + "\n", + "[9866 rows x 3 columns]" + ] + }, + "execution_count": 14, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "query = \"\"\"\n", + "SigninLogs\n", + "| where ResultType in (0, 50055, 50126)\n", + "| where TimeGenerated > ago(5d)\n", + "| project Id, UserPrincipalName, IsInteractive\n", + "\"\"\"\n", + "qry_prov.exec_query(query)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Sign-in summary for previous week
 ValuesNumUniqueValues
AttributeClientAppUserIPAddressLocationUserAgentClientAppUserIPAddressLocationUserAgent
UserPrincipalName        
aauvinen@microsoft.com['Browser']['167.220.197.91']['GB']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.44']1111
adstadel@microsoft.com['Browser']['85.7.181.19']['CH']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36']1111
aguruswamy@contosohotels.com['Browser']['67.168.169.80']['US']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.43']1111
asekstee@microsoft.com['Browser']['144.134.106.54']['AU']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.44']1111
aweinkopf@microsoft.com['Browser']['84.75.253.35' '167.220.196.39']['CH' 'GB']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.44']1221
elsherif@microsoft.com['Browser']['94.128.105.93']['KW']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.51']1111
jank@seccxpninja.onmicrosoft.com['Browser' 'Mobile Apps and Desktop clients']['73.109.22.203']['US']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.44']2111
malarkin@microsoft.com['Browser']['75.136.202.123']['US']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.43']1111
ragomeri@microsoft.com['Browser']['167.220.197.42']['GB']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.51']1111
rickkotlarz@microsoft.com['Browser']['45.22.1.222']['US']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.44']1111
suzanac@contosohotels.com['Browser']['50.47.87.74']['US']['Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.51']1111
takuyaot@microsoft.com['Browser']['153.240.206.142']['JP']['Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/111.0.1661.44']1111
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "user_summary_query = \"\"\"\n", + "let si_history = SigninLogs\n", + "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", + "| where UserPrincipalName in~ ({users})\n", + "| summarize count() by UserPrincipalName, ResultType, RiskLevelAggregated, RiskLevelDuringSignIn, ClientAppUsed, UserAgent, IPAddress, Location;\n", + "si_history\n", + "| summarize OpCount=sum(count_) by UserPrincipalName, ClientAppUsed\n", + "| project UserPrincipalName, Attribute=\"ClientAppUser\", Value=ClientAppUsed, OpCount\n", + "| union ( \n", + "si_history\n", + "| summarize OpCount=sum(count_) by UserPrincipalName, IPAddress\n", + "| project UserPrincipalName, Attribute=\"IPAddress\", Value=IPAddress, OpCount\n", + ")\n", + "| union ( \n", + "si_history\n", + "| summarize OpCount=sum(count_) by UserPrincipalName, UserAgent\n", + "| project UserPrincipalName, Attribute=\"UserAgent\", Value=UserAgent, OpCount\n", + ")\n", + "| union ( \n", + "si_history\n", + "| summarize OpCount=sum(count_) by UserPrincipalName, Location\n", + "| project UserPrincipalName, Attribute=\"Location\", Value=Location, OpCount\n", + ")\n", + "\"\"\"\n", + "week_ago = (end - timedelta(7))\n", + "user_summary_df = qry_prov.exec_query(user_summary_query.format(\n", + " users=get_user_param(risk_users_df),\n", + " start=week_ago,\n", + " end=end\n", + "))\n", + "\n", + "summary_report.add_summary_data(\n", + " data=user_summary_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Signin summary for previous week\"\n", + ")\n", + "df_caption(\n", + " user_summary_df.groupby([\"UserPrincipalName\", \"Attribute\"]).agg(\n", + " Values=pd.NamedAgg(\"Value\", \"unique\"),\n", + " NumUniqueValues=pd.NamedAgg(\"Value\", \"nunique\"),\n", + " OpCount=pd.NamedAgg(\"Value\", \"count\"),\n", + " )\n", + " .reset_index()\n", + " .pivot(index=['UserPrincipalName'], columns='Attribute', values=[\"Values\", \"NumUniqueValues\"]),\n", + " caption=\"Sign-in summary for previous week\"\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 5. Get related alerts for users and user IPs\n", + "\n", + "## Alerts that name the account explicitly" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 12/12 [00:20<00:00, 1.75s/it]\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Related alerts for account
 TenantIdTimeGeneratedAlertDisplayNameAlertNameSeverityProviderNameVendorNameVendorOriginalIdSystemAlertIdResourceIdSourceComputerIdAlertTypeConfidenceLevelConfidenceScoreIsIncidentStartTimeUtcEndTimeUtcProcessingEndTimeEntitiesSourceSystemWorkspaceSubscriptionIdWorkspaceResourceGroupExtendedLinksProductNameProductComponentNameAlertLinkStatusCompromisedEntityTacticsTechniquesTypeComputersrc_hostnamesrc_accountnamesrc_procnamehost_matchacct_matchproc_matchUserPrincipalName
08ecf8077-cf51-4820-aadd-14040956f35d2023-03-21 23:15:18.679918+00:00Activity from infrequent countryActivity from infrequent countryMediumMCASMicrosoft641a3a797612f527f42ed3d4abf4cd24-8737-861d-fb55-2b9d52757ec4MCAS_ALERT_ANUBIS_DETECTION_NEW_COUNTRYnanFalse2023-03-21 23:09:09.916000+00:002023-03-21 23:09:09.916000+00:002023-03-21 23:15:18.678815600+00:00[{\"$id\":\"2\",\"Address\":\"73.109.22.203\",\"Location\":{\"CountryCode\":\"US\"},\"Asset\":false,\"Roles\":[\"Attacker\"],\"Type\":\"ip\"},{\"$id\":\"3\",\"AppId\":11161,\"SaasId\":11161,\"Name\":\"Office 365\",\"InstanceId\":0,\"Type\":\"cloud-application\"},{\"$id\":\"4\",\"Name\":\"jank\",\"UPNSuffix\":\"seccxpninja.onmicrosoft.com\",\"AadUserId\":\"0cecc121-df78-4350-8e73-d81d9925bcb6\",\"CloudAppAccountId\":\"11161|0|0cecc121-df78-4350-8e73-d81d9925bcb6\",\"Type\":\"account\"}]Detection[{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/policy/?id=eq(5b2405b8185459b631739047,)\",\"Category\":null,\"Label\":\"Defender for Cloud Apps policy ID\",\"Type\":\"webLink\"},{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/alerts/641a3a797612f527f42ed3d4\",\"Category\":null,\"Label\":\"Defender for Cloud Apps alert ID\",\"Type\":\"webLink\"}]Microsoft Cloud App Securityhttps://seccxpninja.portal.cloudappsecurity.com/#/alerts/641a3a797612f527f42ed3d4Newjank@seccxpninja.onmicrosoft.comDefenseEvasion[\"T1078\"]SecurityAlertjankFalseTrueFalsejank@seccxpninja.onmicrosoft.com
18ecf8077-cf51-4820-aadd-14040956f35d2023-03-22 00:58:03.079763900+00:00Activity from infrequent countryActivity from infrequent countryMediumMCASMicrosoft64109eb2d28315667fdec3c9323bdd2e-45a9-9ed5-5c82-d9d785d954f4MCAS_ALERT_ANUBIS_DETECTION_NEW_COUNTRYnanFalse2023-03-14 16:16:26.380000+00:002023-03-14 16:16:26.380000+00:002023-03-22 00:58:03.078047300+00:00[{\"$id\":\"2\",\"Address\":\"181.214.93.55\",\"Location\":{\"CountryCode\":\"BR\"},\"Asset\":false,\"Roles\":[\"Attacker\"],\"Type\":\"ip\"},{\"$id\":\"3\",\"AppId\":11161,\"SaasId\":11161,\"Name\":\"Office 365\",\"InstanceName\":\"Office 365\",\"InstanceId\":0,\"Type\":\"cloud-application\"},{\"$id\":\"4\",\"Name\":\"jank\",\"UPNSuffix\":\"seccxpninja.onmicrosoft.com\",\"AadUserId\":\"0cecc121-df78-4350-8e73-d81d9925bcb6\",\"CloudAppAccountId\":\"11161|0|0cecc121-df78-4350-8e73-d81d9925bcb6\",\"Type\":\"account\"}]Detection[{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/policy/?id=eq(5b2405b8185459b631739047,)\",\"Category\":null,\"Label\":\"Defender for Cloud Apps policy ID\",\"Type\":\"webLink\"},{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/alerts/64109eb2d28315667fdec3c9\",\"Category\":null,\"Label\":\"Defender for Cloud Apps alert ID\",\"Type\":\"webLink\"}]Microsoft Cloud App Securityhttps://seccxpninja.portal.cloudappsecurity.com/#/alerts/64109eb2d28315667fdec3c9ResolvedDefenseEvasion[\"T1078\"]SecurityAlertjankFalseTrueFalsejank@seccxpninja.onmicrosoft.com
08ecf8077-cf51-4820-aadd-14040956f35d2023-03-21 19:27:18.303824600+00:00Activity from infrequent countryActivity from infrequent countryMediumMCASMicrosoft641a0510c6011d2c37ce2bbc204e5a06-5560-20e0-6e7f-95795f988c46MCAS_ALERT_ANUBIS_DETECTION_NEW_COUNTRYnanFalse2023-03-21 19:25:49.784000+00:002023-03-21 19:25:49.784000+00:002023-03-21 19:27:18.301896400+00:00[{\"$id\":\"2\",\"Name\":\"aguruswamy\",\"UPNSuffix\":\"contosohotels.com\",\"AadUserId\":\"4d10cc91-8c99-4f23-9470-7070cb2eaf4b\",\"CloudAppAccountId\":\"11161|0|4d10cc91-8c99-4f23-9470-7070cb2eaf4b\",\"Type\":\"account\"},{\"$id\":\"3\",\"Address\":\"67.168.169.80\",\"Location\":{\"CountryCode\":\"US\"},\"Asset\":false,\"Roles\":[\"Attacker\"],\"Type\":\"ip\"},{\"$id\":\"4\",\"AppId\":51040,\"SaasId\":51040,\"Name\":\"Microsoft MyAccount\",\"InstanceId\":2047,\"Type\":\"cloud-application\"}]Detection[{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/policy/?id=eq(5b2405b8185459b631739047,)\",\"Category\":null,\"Label\":\"Defender for Cloud Apps policy ID\",\"Type\":\"webLink\"},{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/alerts/641a0510c6011d2c37ce2bbc\",\"Category\":null,\"Label\":\"Defender for Cloud Apps alert ID\",\"Type\":\"webLink\"}]Microsoft Cloud App Securityhttps://seccxpninja.portal.cloudappsecurity.com/#/alerts/641a0510c6011d2c37ce2bbcNewaguruswamy@contosohotels.comDefenseEvasion[\"T1078\"]SecurityAlertaguruswamyFalseTrueFalseaguruswamy@contosohotels.com
08ecf8077-cf51-4820-aadd-14040956f35d2023-03-22 17:37:12.607978+00:00Activity from infrequent countryActivity from infrequent countryMediumMCASMicrosoft641b3cc03261b058ed4ce34832c9582c-419d-c77d-1140-55142b386500MCAS_ALERT_ANUBIS_DETECTION_NEW_COUNTRYnanFalse2023-03-22 17:35:17.280000+00:002023-03-22 17:35:17.280000+00:002023-03-22 17:37:12.606159400+00:00[{\"$id\":\"2\",\"Address\":\"50.47.87.74\",\"Location\":{\"CountryCode\":\"US\"},\"Asset\":false,\"Roles\":[\"Attacker\"],\"Type\":\"ip\"},{\"$id\":\"3\",\"Name\":\"suzanac\",\"UPNSuffix\":\"contosohotels.com\",\"AadUserId\":\"53ea4395-cd1b-4c86-a6ad-370aae4ce1b7\",\"CloudAppAccountId\":\"11161|0|53ea4395-cd1b-4c86-a6ad-370aae4ce1b7\",\"Type\":\"account\"},{\"$id\":\"4\",\"AppId\":51040,\"SaasId\":51040,\"Name\":\"Microsoft MyAccount\",\"InstanceId\":2047,\"Type\":\"cloud-application\"}]Detection[{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/policy/?id=eq(5b2405b8185459b631739047,)\",\"Category\":null,\"Label\":\"Defender for Cloud Apps policy ID\",\"Type\":\"webLink\"},{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/alerts/641b3cc03261b058ed4ce348\",\"Category\":null,\"Label\":\"Defender for Cloud Apps alert ID\",\"Type\":\"webLink\"}]Microsoft Cloud App Securityhttps://seccxpninja.portal.cloudappsecurity.com/#/alerts/641b3cc03261b058ed4ce348Newsuzanac@contosohotels.comDefenseEvasion[\"T1078\"]SecurityAlertsuzanacFalseTrueFalsesuzanac@contosohotels.com
08ecf8077-cf51-4820-aadd-14040956f35d2023-03-22 07:21:11.969415700+00:00Activity from infrequent countryActivity from infrequent countryMediumMCASMicrosoft641aac60525cb17b1013eae5bf1d5ca9-7e77-2a82-fa59-2715886aa153MCAS_ALERT_ANUBIS_DETECTION_NEW_COUNTRYnanFalse2023-03-22 07:19:32.246000+00:002023-03-22 07:19:32.246000+00:002023-03-22 07:21:11.968285200+00:00[{\"$id\":\"2\",\"AppId\":35931,\"SaasId\":35931,\"Name\":\"Microsoft 365 security center\",\"InstanceId\":2047,\"Type\":\"cloud-application\"},{\"$id\":\"3\",\"Address\":\"153.240.206.142\",\"Location\":{\"CountryCode\":\"JP\"},\"Asset\":false,\"Roles\":[\"Attacker\"],\"Type\":\"ip\"},{\"$id\":\"4\",\"Name\":\"takuyaot\",\"UPNSuffix\":\"microsoft.com\",\"AadUserId\":\"01b09ab3-09f3-485b-b55c-cc96d5d2ab9d\",\"CloudAppAccountId\":\"11161|0|01b09ab3-09f3-485b-b55c-cc96d5d2ab9d\",\"Type\":\"account\"}]Detection[{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/policy/?id=eq(5b2405b8185459b631739047,)\",\"Category\":null,\"Label\":\"Defender for Cloud Apps policy ID\",\"Type\":\"webLink\"},{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/alerts/641aac60525cb17b1013eae5\",\"Category\":null,\"Label\":\"Defender for Cloud Apps alert ID\",\"Type\":\"webLink\"}]Microsoft Cloud App Securityhttps://seccxpninja.portal.cloudappsecurity.com/#/alerts/641aac60525cb17b1013eae5Newtakuyaot@microsoft.comDefenseEvasion[\"T1078\"]SecurityAlerttakuyaotFalseTrueFalsetakuyaot@microsoft.com
08ecf8077-cf51-4820-aadd-14040956f35d2023-03-22 08:29:07.284193+00:00Activity from infrequent countryActivity from infrequent countryMediumMCASMicrosoft641abc4e08e2065d7dca9b11e61a55fc-acc5-cdde-d0d0-43f73c74b7d8MCAS_ALERT_ANUBIS_DETECTION_NEW_COUNTRYnanFalse2023-03-22 08:27:20.870000+00:002023-03-22 08:27:20.870000+00:002023-03-22 08:29:07.282423500+00:00[{\"$id\":\"2\",\"Address\":\"94.128.105.93\",\"Location\":{\"CountryCode\":\"KW\"},\"Asset\":false,\"Roles\":[\"Attacker\"],\"Type\":\"ip\"},{\"$id\":\"3\",\"Name\":\"elsherif\",\"UPNSuffix\":\"microsoft.com\",\"AadUserId\":\"b045505f-b621-4273-a434-7c83268a268e\",\"CloudAppAccountId\":\"11161|0|b045505f-b621-4273-a434-7c83268a268e\",\"Type\":\"account\"},{\"$id\":\"4\",\"AppId\":35931,\"SaasId\":35931,\"Name\":\"Microsoft 365 security center\",\"InstanceId\":2047,\"Type\":\"cloud-application\"}]Detection[{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/policy/?id=eq(5b2405b8185459b631739047,)\",\"Category\":null,\"Label\":\"Defender for Cloud Apps policy ID\",\"Type\":\"webLink\"},{\"Href\":\"https://seccxpninja.portal.cloudappsecurity.com/#/alerts/641abc4e08e2065d7dca9b11\",\"Category\":null,\"Label\":\"Defender for Cloud Apps alert ID\",\"Type\":\"webLink\"}]Microsoft Cloud App Securityhttps://seccxpninja.portal.cloudappsecurity.com/#/alerts/641abc4e08e2065d7dca9b11Newelsherif@microsoft.comDefenseEvasion[\"T1078\"]SecurityAlertelsherifFalseTrueFalseelsherif@microsoft.com
18ecf8077-cf51-4820-aadd-14040956f35d2023-03-22 11:51:43.988203400+00:00Authentication Attempt from New CountryAuthentication Attempt from New CountryMediumASI Scheduled AlertsMicrosoft96ad6b0e-ecda-4366-beae-3e292adbfc435f16ecd7-1788-64d3-9815-8882091bbb5e8ecf8077-cf51-4820-aadd-14040956f35d_70a3191f-d92a-42e1-b305-de1d4d2becd0nanFalse2023-03-08 11:46:38.822000+00:002023-03-22 11:46:38.822000+00:002023-03-22 11:51:43.918945400+00:00[{\"$id\":\"2\",\"Name\":\"pdemo\",\"UPNSuffix\":\"seccxpninja.onmicrosoft.com\",\"IsDomainJoined\":true,\"DisplayName\":\"pdemo@seccxpninja.onmicrosoft.com\",\"Type\":\"account\"},{\"$id\":\"3\",\"Name\":\"hesaad\",\"UPNSuffix\":\"microsoft.com\",\"IsDomainJoined\":true,\"DisplayName\":\"hesaad@microsoft.com\",\"Type\":\"account\"},{\"$id\":\"4\",\"Name\":\"elsherif\",\"UPNSuffix\":\"microsoft.com\",\"IsDomainJoined\":true,\"DisplayName\":\"elsherif@microsoft.com\",\"Type\":\"account\"}]Detectiond1d8779d-38d7-4f06-91db-9cbc8de0176fsocAzure SentinelScheduled AlertsNewInitialAccess, Reconnaissance[\"T1594\",\"T1078\"]SecurityAlertelsherifFalseTrueFalseelsherif@microsoft.com
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "related_alerts_df = pd.concat([\n", + " (\n", + " qry_prov.SecurityAlert.list_related_alerts(account_name=acct)\n", + " .assign(UserPrincipalName=acct)\n", + " )\n", + " for acct in tqdm(risk_users_df.UserPrincipalName)\n", + "])\n", + "summary_report.add_summary_data(\n", + " data=related_alerts_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Related alerts for user\"\n", + ")\n", + "df_caption(related_alerts_df.drop(\n", + " columns=[\"Description\", \"RemediationSteps\", \"ExtendedProperties\"]),\n", + " caption=\"Related alerts for account\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## 2. Alerts related to signin-in IP addresses" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "100%|██████████| 12/12 [00:24<00:00, 2.08s/it]\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Related alerts for sign-in IP Address
 TenantIdTimeGeneratedAlertDisplayNameAlertNameSeverityDescriptionProviderNameVendorNameVendorOriginalIdSystemAlertIdResourceIdSourceComputerIdAlertTypeConfidenceLevelConfidenceScoreIsIncidentStartTimeUtcEndTimeUtcProcessingEndTimeRemediationStepsExtendedPropertiesEntitiesSourceSystemWorkspaceSubscriptionIdWorkspaceResourceGroupExtendedLinksProductNameProductComponentNameAlertLinkStatusCompromisedEntityTacticsTechniquesTypeSystemAlertId1ExtendedProperties1Entities1MatchingIpsUserPrincipalNameIPAddress
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "related_alerts_ip_df = pd.concat([\n", + " (\n", + " qry_prov.SecurityAlert.list_alerts_for_ip(source_ip_list=ip_addr)\n", + " .assign(UserPrincipalName=acct, IPAddress=ip_addr)\n", + " )\n", + " for acct, ip_addr in tqdm(\n", + " risk_users_df.explode(\"SourceIPs\")[[\"UserPrincipalName\", \"SourceIPs\"]].apply(tuple, axis=1)\n", + " )\n", + "])\n", + "summary_report.add_summary_data(\n", + " data=related_alerts_ip_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Related alerts for user signin IP address\"\n", + ")\n", + "\n", + "df_caption(related_alerts_ip_df, caption=\"Related alerts for sign-in IP Address\")\n" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 6. Get Threat Intelligence reports for sign-in IPs" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Observables processed: 100%|██████████| 72/72 [00:45<00:00, 1.57obs/s]\n" + ] + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Threat intel reports for risky sign-in IPs
 UserPrincipalNameSourceIPsQuerySubtypeResultDetailsRawResultReferenceStatusIocIocTypeSafeIocSeverityProvider
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# look up IP addresses - join UserPrincipalName from source DF to output\n", + "ti_user_ip = IpAddress.tilookup_ip(\n", + " risk_users_df.explode(\"SourceIPs\")[[\"UserPrincipalName\", \"SourceIPs\"]],\n", + " column=\"SourceIPs\",\n", + " join=\"left\"\n", + ").query(\"Severity != 'information'\")\n", + "\n", + "summary_report.add_summary_data(\n", + " data=ti_user_ip,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Threat intel reports for user sign-in IP address(es)\"\n", + ")\n", + "\n", + "df_caption(ti_user_ip, caption=\"Threat intel reports for risky sign-in IPs\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 7. Look for unusual Azure Audit entries\n", + "\n", + "Look for operations in Azure audit for selected accounts\n", + "where account used operations type in the current time slot that\n", + "it had not used in the baseline period (default prior 30 days)" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Azure audit activity types not seen in baseline period.
 IdentityUserPrincipalNameOperationNameLoggedByServiceInitiatedByAdditionalDetailsTargetResources
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Azure Audit\n", + "# Find any operation types for current period that weren't seen for\n", + "# that user in previous baseline period\n", + "azure_audit_query = \"\"\"\n", + "let start = datetime(\"{start}\");\n", + "let end = datetime(\"{end}\");\n", + "let baseline_start = start - ({baseline_period} * 1d);\n", + "let bl_threshold = {threshold};\n", + "let operation_history = AuditLogs\n", + "| where TimeGenerated between(baseline_start .. start)\n", + "| where Identity !in (\"Azure AD Cloud Sync\", \"Managed Service Identity\", \"Microsoft.Azure.SyncFabric\")\n", + "| where bag_has_key(InitiatedBy, \"user\")\n", + "| extend UserPrincipalName = tostring(InitiatedBy[\"user\"][\"userPrincipalName\"])\n", + "| where UserPrincipalName in~ ({users})\n", + "| summarize EventCount=count() by UserPrincipalName, OperationName\n", + "| where EventCount > bl_threshold;\n", + "AuditLogs\n", + "| where TimeGenerated between(end .. start)\n", + "| where Identity !in (\"Azure AD Cloud Sync\", \"Managed Service Identity\", \"Microsoft.Azure.SyncFabric\")\n", + "| where bag_has_key(InitiatedBy, \"user\")\n", + "| extend UserPrincipalName = tostring(InitiatedBy[\"user\"][\"userPrincipalName\"]), IPAddress = InitiatedBy[\"user\"][\"ipAddress\"]\n", + "| where UserPrincipalName in~ ({users})\n", + "| join kind=leftanti (operation_history) on UserPrincipalName, OperationName\n", + "| project Identity, UserPrincipalName, OperationName, LoggedByService, InitiatedBy, AdditionalDetails, TargetResources\n", + "\"\"\"\n", + "\n", + "end = datetime.now(tz=timezone.utc)\n", + "start = end-timedelta(1)\n", + "from datetime import datetime, timezone, timedelta\n", + "fmt_query = azure_audit_query.format(\n", + " start=start,\n", + " end=end,\n", + " baseline_period=baseline_period,\n", + " threshold=0,\n", + " users=get_user_param(risk_users_df),\n", + ")\n", + "az_audit_df = qry_prov.exec_query(fmt_query)\n", + "summary_report.add_summary_data(\n", + " data=az_audit_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Unusual Azure Audit log entries for user\"\n", + ")\n", + "df_caption(az_audit_df, caption=\"Azure audit activity types not seen in baseline period.\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 8. Look for unusual Office 365 activity\n", + "\n", + "Office operations occurring in the measured period that had\n", + "not occurred or rarely occurred in the baseline period." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "o365_baseline_activity_query = \"\"\"\n", + "let num_stddev = {std_dev_scale};\n", + "let bl_period = datetime_add(\"day\", -{baseline_period}, datetime({start}));\n", + "OfficeActivity\n", + "| where TimeGenerated between (bl_period .. datetime({start}))\n", + "| where UserId in~ ({users})\n", + "// count operations by user and op type per day\n", + "| summarize OpCount = count() by UserId, OfficeWorkload, Operation, bin(TimeGenerated, 1d)\n", + "// calculate mean and average values for the user/op combos\n", + "| summarize OpStdev = stdev(OpCount), OpMean = avg(OpCount) by UserId, OfficeWorkload, Operation\n", + "// Calculate a baseline score Mean + N StdDevs * StdDev (default to 1 if 0 variance)\n", + "| extend OpBase = OpMean + (num_stddev * iif(OpStdev > 0, OpStdev, 1.0))\n", + "| extend RecType=\"baseline\"\n", + "\"\"\"\n", + "\n", + "o365_current_activity_query = \"\"\"\n", + "OfficeActivity\n", + "| where TimeGenerated between (datetime({start}) .. datetime({end}))\n", + "| where UserId in~ ({users})\n", + "| summarize OpCount = count() by UserId, OfficeWorkload, Operation\n", + "| extend RecType=\"current\"\n", + "\"\"\"\n", + "\n", + "# set number of std deviations from mean to use as indicating\n", + "# anomalous activity\n", + "_STD_THRESHOLD = 2\n", + "\n", + "end = datetime.now(tz=timezone.utc)\n", + "start = end - timedelta(1)\n", + "office_baseline_df = qry_prov.exec_query(\n", + " o365_baseline_activity_query.format(\n", + " users=get_user_param(risk_users_df),\n", + " std_dev_scale=_STD_THRESHOLD,\n", + " start=start,\n", + " baseline_period=baseline_period,\n", + " )\n", + ")\n", + "office_current_df = qry_prov.exec_query(\n", + " o365_current_activity_query.format(\n", + " users=get_user_param(risk_users_df),\n", + " start=start,\n", + " end=end\n", + " )\n", + ")\n", + "\n", + "# Pull out any current activity that exceeds the baseline threshold (mean + N*stddev)\n", + "office_activity_df = (\n", + " office_current_df\n", + " .merge(office_baseline_df, on=[\"UserId\", \"OfficeWorkload\", \"Operation\"], how=\"left\")\n", + " .fillna({\"OpBase\": 0})\n", + " .query(\"OpCount > OpBase\")\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Office baseline operations.
 UserIdOfficeWorkloadOperationOpStdevOpMeanOpBaseRecType
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Office current operations.
 UserIdOfficeWorkloadOperationOpCountRecType
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Office anomalous operations.
 OpCountRecType_xUserIdOfficeWorkloadOperationOpStdevOpMeanOpBaseRecType_y
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "df_caption(office_baseline_df, \"Office baseline operations.\")\n", + "df_caption(office_current_df, \"Office current operations.\")\n", + "df_caption(office_activity_df, \"Office anomalous operations.\")\n", + "summary_report.add_summary_data(\n", + " data=office_activity_df,\n", + " user_column=\"UserId\",\n", + " section=\"Unusual Office activity for user\"\n", + ")\n", + "summary_report.add_summary_data(\n", + " data=office_current_df,\n", + " user_column=\"UserId\",\n", + " section=\"Summarized current Office activity for user\"\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 9. Look for unusual Azure activity\n", + "\n", + "Azure activity operations occurring in the measured period that had\n", + "not occurred in the baseline period.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
Azure activity operations not seen in baseline period.
 UserPrincipalNameOperationNameValueIPAddressResourceGroupSubscriptionIdTenantIdEventCountActivityStatusValueStartTimeEndTime
\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Azure Activity\n", + "azure_activity_query = \"\"\"\n", + "let start = datetime(\"{start}\");\n", + "let end = datetime(\"{end}\");\n", + "let baseline_start = start - ({period} * 1d);\n", + "let bl_threshold = {threshold};\n", + "let operation_history = AzureActivity\n", + "| where TimeGenerated between(baseline_start .. start)\n", + "| where Caller in~ ({users})\n", + "| project UserPrincipalName=Caller, OperationNameValue\n", + "| summarize EventCount=count() by UserPrincipalName, OperationNameValue\n", + "| where EventCount > bl_threshold;\n", + "AzureActivity\n", + "| where TimeGenerated between(start .. end)\n", + "| where Caller in~ ({users})\n", + "| project-rename UserPrincipalName=Caller\n", + "| join kind=leftanti (operation_history) on UserPrincipalName, OperationNameValue\n", + "| project TimeGenerated, UserPrincipalName, OperationNameValue, IPAddress=CallerIpAddress,\n", + " EventDataId, ActivityStatusValue, ResourceGroup, SubscriptionId, TenantId\n", + "\"\"\"\n", + "\n", + "fmt_query = azure_activity_query.format(\n", + " end=datetime.now(tz=timezone.utc),\n", + " start=end-timedelta(1),\n", + " period=28,\n", + " threshold=0,\n", + " users=get_user_param(risk_users_df),\n", + ")\n", + "azure_activity_df = qry_prov.exec_query(fmt_query)\n", + "\n", + "aa_summary_cols = [\n", + " \"UserPrincipalName\",\n", + " \"OperationNameValue\",\n", + " \"IPAddress\",\n", + " \"ResourceGroup\",\n", + " \"SubscriptionId\",\n", + " \"TenantId\",\n", + "]\n", + "\n", + "azure_activity_summary_df = azure_activity_df.groupby(aa_summary_cols).agg(\n", + " EventCount=pd.NamedAgg(\"TimeGenerated\", \"count\"),\n", + " ActivityStatusValue=pd.NamedAgg(\"ActivityStatusValue\", \"unique\"),\n", + " StartTime=pd.NamedAgg(\"TimeGenerated\", \"min\"),\n", + " EndTime=pd.NamedAgg(\"TimeGenerated\", \"max\"),\n", + ").reset_index().sort_values(\"StartTime\", ascending=True)\n", + "\n", + "summary_report.add_summary_data(\n", + " data=azure_activity_summary_df,\n", + " user_column=\"UserPrincipalName\",\n", + " section=\"Unusual Azure activity for user\"\n", + ")\n", + "df_caption(azure_activity_summary_df, \"Azure activity operations not seen in baseline period.\")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# 10. Summarize and upload data\n", + "\n", + "Create dynamic summaries for each user and upload to sentinel\n", + "\n", + "> Note: we could offer the option to group by report type instead\n", + "> of user. That would result in a Dynamic Summary entry for each\n", + "> report type (with consistent schema) but with data from (potentially)\n", + "> multiple users." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Uploading AccountEvaluation - aauvinen@microsoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - adstadel@microsoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - aguruswamy@contosohotels.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - asekstee@microsoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - aweinkopf@microsoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - elsherif@microsoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - jank@seccxpninja.onmicrosoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - malarkin@microsoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - ragomeri@microsoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - rickkotlarz@microsoft.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - suzanac@contosohotels.com\n", + "Dynamic summary created/updated.\n", + "Uploading AccountEvaluation - takuyaot@microsoft.com\n", + "Dynamic summary created/updated.\n" + ] + } + ], + "source": [ + "# Iterate through summary reports and create a summary for each user\n", + "try:\n", + " from notebookutils import mssparkutils\n", + " synapse_workspace = mssparkutils.env.getWorkspaceName\n", + "except ImportError:\n", + " synapse_workspace = \"none\"\n", + "\n", + "def create_dynamic_summaries(summary_report):\n", + " \"\"\"Create a dynamic summary for each account.\"\"\"\n", + " dynamic_summaries = []\n", + " for user, reports in summary_report.summary_reports.items():\n", + " # Create a summary for each user\n", + " user_ds = DynamicSummary(\n", + " summary_name=f\"AccountEvaluation - {user}\",\n", + " summary_description=\"Summary generated from AccountSignInEvaluation notebook.\",\n", + " source_info={\n", + " \"source\": \"notebooks\",\n", + " \"notebook\": \"AccountSignInEvaluation.ipynb\",\n", + " \"synapse_workspace\": synapse_workspace\n", + " }\n", + " )\n", + "\n", + " for report_type, summary_item in reports.items():\n", + " ds_item_params = {\n", + " \"event_time_utc\": end,\n", + " \"search_key\": user,\n", + " \"observable_type\": \"report_type\",\n", + " \"observable_value\": report_type\n", + " }\n", + " user_ds.add_summary_items(\n", + " data=summary_item.data,\n", + " **ds_item_params\n", + " )\n", + " dynamic_summaries.append(user_ds)\n", + " return dynamic_summaries\n", + "\n", + "\n", + "def upload_dynamic_summaries(sentinel, dynamic_summaries):\n", + " \"\"\"Upload summaries to Sentinel.\"\"\"\n", + " for dyn_summary in dynamic_summaries:\n", + " # Create or update the report\n", + " print(\"Uploading\", dyn_summary.summary_name)\n", + " sentinel.create_dynamic_summary(dyn_summary)\n", + "\n", + "\n", + "dynamic_summaries = create_dynamic_summaries(summary_report)\n", + "upload_dynamic_summaries(sentinel, dynamic_summaries)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Appendix - Pickling and restoring data\n", + "\n", + "The following cells allow the summary data to be picked\n", + "and stored as a file. The final cell will\n", + "store the dynamic summary (base64 encoded) in a notebook\n", + "cell, so it can be restored later.\n", + "\n", + "These are all commented-out. Uncomment to use any of these." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# # Save dynamic summaries to a pickle file\n", + "# import pickle\n", + "# obj = pickle.dumps(dynamic_summaries)\n", + "\n", + "# with open(\"acct_nb_summaries.pkl\", \"wb\") as pickle_file:\n", + "# pickle_file.write(obj)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "['AccountEvaluation - tamuto@seccxpninja.onmicrosoft.com']\n" + ] + } + ], + "source": [ + "# # Restore dynamic summaries from a pickle file\n", + "# # note - you need to have the DynamicSummary class imported\n", + "# import pickle\n", + "# from msticpy.context.azure.sentinel_dynamic_summary import DynamicSummary\n", + "# # to successfully restore the summary report\n", + "# with open(\"acct_nb_summaries.pkl\", \"rb\") as pickle_file:\n", + "# summary_obj = pickle_file.read()\n", + "# dynamic_summaries_copy = pickle.loads(summary_obj)\n", + "\n", + "# print([ds.summary_name for ds in dynamic_summaries_copy])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# # Save dynamic summaries to a base64-encoded cell\n", + "\n", + "# from base64 import b64encode\n", + "# from IPython.core.getipython import get_ipython\n", + "\n", + "# cell_code = \"\"\"#########################################\n", + "# # Run this cell to restore cached data to\n", + "# # the object \"{var_name}\"\n", + "# #########################################\n", + "\n", + "# from base64 import b64decode\n", + "# import pickle\n", + "\n", + "# ## Store dynamic summaries as base64 byte string\n", + "# summary_data = {encoded_bytes}\n", + "\n", + "# # decode and unpickle the summaries\n", + "# {var_name} = pickle.loads(b64decode(summary_data))\n", + "# {var_name}\n", + "# \"\"\"\n", + "\n", + "# def persist_to_cell(dynamic_summaries, var_name=\"acct_summaries\"):\n", + "# encoded_bytes = b64encode(pickle.dumps(dynamic_summaries))\n", + "# cell_text = cell_code.format(\n", + "# encoded_bytes=encoded_bytes,\n", + "# var_name=var_name,\n", + "# )\n", + "# shell = get_ipython()\n", + "# # create a new cell using `cell_code` as the code contents.\n", + "# shell.set_next_input(cell_text)\n", + "\n", + "# persist_to_cell(dynamic_summaries)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "msticpy", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.7" + }, + "orig_nbformat": 4, + "vscode": { + "interpreter": { + "hash": "0f1a8e166ce5c1ec1911a36e4fdbd34b2f623e2a3442791008b8ac429a1d6070" + } + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/scenario-notebooks/Automated-Notebooks/Authoring automated notebooks.md b/scenario-notebooks/Automated-Notebooks/Authoring automated notebooks.md index 3561c83a..d7706b1f 100644 --- a/scenario-notebooks/Automated-Notebooks/Authoring automated notebooks.md +++ b/scenario-notebooks/Automated-Notebooks/Authoring automated notebooks.md @@ -1,7 +1,7 @@ # Authoring automated Sentinel notebooks -Sentinel notebooks automation is build on the top of Azure Synapse Analytics pipeline platform. +Sentinel notebooks automation is build on the top of Azure Synapse Analytics pipeline platform. In this article, we will discuss a few topics: 1. How to retrieve secrets saved in Azure Key Vault @@ -15,8 +15,8 @@ Here is the generic information about [Azure Synapse notebooks](https://docs.mic --- -## Retrieval of Azure Key Vault secrets -An instance of Azure Key Vault is created during notebook configuration time. Project related secrets can be saved there. From notebooks, it is easy to read the secrets in Azure Key Vault by using Synapse linked service, which is created during notebook configuration time to link Synapse workspace to Azure Key Vault. +## Retrieval of Azure Key Vault secrets +An instance of Azure Key Vault is created during notebook configuration time. Project related secrets can be saved there. From notebooks, it is easy to read the secrets in Azure Key Vault by using Synapse linked service, which is created during notebook configuration time to link Synapse workspace to Azure Key Vault. ``` secret = mssparkutils.credentials.getSecret(akv_name, secret_name, akv_link_name) @@ -32,8 +32,8 @@ client_secret = mssparkutils.credentials.getSecret(akv_name, client_secret_name, Secrets saved in Azure Key Vault are fetched through Synapse linked service. ``` credential = ClientSecretCredential( - tenant_id=tenant_id, - client_id=client_id, + tenant_id=tenant_id, + client_id=client_id, client_secret=client_secret) cred = AzureIdentityCredentialAdapter(credential) ``` @@ -75,7 +75,7 @@ Using mssparkutils.notebook.exit will not fail the pipeline, but provide an outp ``` mssparkutils.notebook.exit("Auth failed") ``` -As a notebook developer, you need to decide which way is the right way to handle exceptions based on your scenarios. +As a notebook developer, you need to decide which way is the right way to handle exceptions based on your scenarios. ## Permission check in notebooks @@ -87,22 +87,29 @@ Azure service principal is used in Sentinel automation notebooks to access vario | Subscription | W/R in sub | RO in sub | | Resource group | W/R in RG | RO in RG | -Since each notebook template may access different data sources and REST APIs with different actions (w/r), it is possible that notebooks will fail during execution due to insufficient permissions. +Since each notebook template may access different data sources and REST APIs with different actions (w/r), it is possible that notebooks will fail during execution due to insufficient permissions. To avoid the situation, the service principal should be given peoper permissions to execute target notebooks. At the same time, notebook authors should try to catch the exception and render meaningful error message. Usually, client object initizliation will not throw exception, but when the client object is used to access a resource object, permission exception will be thrown. ## How to persist key findings in Sentinel through REST API -Sentinel Dynamic Summaries REST API is the recommended way to persist notebook execution results to Azure Log Analytics, where the notebook data can be joined with other data for further analysis. And regular Sentinel users can query the data as long as they have proper permissions. [The cred scan notebook on Azure Log Analytics](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/scenario-notebooks/Automated-Notebooks/AutomationGallery-CredentialScanOnAzureLogAnalytics.ipynb) and [The cred scan notebook on Azure blob storage](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/scenario-notebooks/Automated-Notebooks/AutomationGallery-CredentialScanOnAzureBlobStorage.ipynb) provide good examples to send the results to the Dynamic Summaries table in an Azure Log Analytics workspace. +Sentinel Dynamic Summaries REST API is the recommended way to persist notebook execution results to Azure Log Analytics, where the notebook data can be joined with other data for further analysis. And regular Sentinel users can query the data as long as they have proper permissions. + +MSTICPy now includes support for creating and uploading Dynamic Summaries. +See the [Dynamic Summary documenation](https://msticpy.readthedocs.io/en/latest/data_acquisition/SentinelDynamicSummaries.html). See the [AccountSigningEvaluation notebook](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/scenario-notebooks/Automated-Notebooks/AccountSignInEvaluation.ipynb) + +The Credscan notebooks contain examples of how you can use the +Dynamic Summaries API using the Python requests package. +[The cred scan notebook on Azure Log Analytics](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/scenario-notebooks/Automated-Notebooks/AutomationGallery-CredentialScanOnAzureLogAnalytics.ipynb) and [The cred scan notebook on Azure blob storage](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/scenario-notebooks/Automated-Notebooks/AutomationGallery-CredentialScanOnAzureBlobStorage.ipynb) provide good examples to send the results to the Dynamic Summaries table in an Azure Log Analytics workspace. During notebook automation provisioning step, an ADLS storage instance is created for Azure Synapse workspace. So it is possible to upload the result as a file to blob storage, through the build-in MSSparkUtils module via ADLS linked service. Very few individual users have access to the storage, but it can be used for sequential notebooks in later time. ``` mount_name = "testmount" -mssparkutils.fs.mount( - "abfss://sentinelfiles@synapse4sentinel.dfs.core.windows.net", +mssparkutils.fs.mount( + "abfss://sentinelfiles@synapse4sentinel.dfs.core.windows.net", "/" + mount_name, - {"linkedService":"synapse4sentinel-WorkspaceDefaultStorage"} -) + {"linkedService":"synapse4sentinel-WorkspaceDefaultStorage"} +) job_id = mssparkutils.env.getJobId()