Skip to content

Commit ccbfbed

Browse files
kshitiz-progKshitiz sharmaalexandair
authored
Data-35022 : On-Demand Scans Configured for Sensitive Information Discovery (#760)
* Feature-35022 * Feature-35022 : function name fix * md file fix * md file fix * Set of code change * 35022-fix * Finetuned * Finetuned code * removal of unwanted file * Code refinement * Casing fix * SITs logic fix * 35022 - Git Comment resolved * Remove unnecessary spaces --------- Co-authored-by: Kshitiz sharma <kshiti.p@perennialsys.com> Co-authored-by: alexandair <alexandair@live.com>
1 parent 3588aa5 commit ccbfbed

2 files changed

Lines changed: 346 additions & 0 deletions

File tree

‎src/powershell/tests/Test-Assessment.35022.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
Organizations with large volumes of historical content in SharePoint, OneDrive, and Exchange that predates auto-labeling policy implementation lack visibility into the extent of unclassified sensitive data across their tenants. Auto-labeling policies only classify new and modified content going forward; existing files and emails remain unclassified and invisible to data loss prevention policies that depend on label detection. Without on-demand scans, organizations cannot perform a baseline assessment of sensitive information already present in their environments, making it impossible to quantify compliance risk, plan remediation, or validate that DLP controls are effectively protecting all sensitive data. On-demand scans allow organizations to manually trigger sensitive information type detection across specified SharePoint sites, OneDrive accounts, and Exchange mailboxes, identifying where sensitive data exists and enabling targeted classification through retroactive labeling. Configuring at least one on-demand scan enables organizations to discover and classify historical sensitive data, providing a comprehensive view of their information protection posture beyond the forward-looking coverage of auto-labeling policies and creating a complete baseline for compliance and risk management.
2+
3+
**Remediation action**
4+
5+
To configure on-demand scans for sensitive information discovery and classification, follow these steps:
6+
1. **Plan your scan strategy** by identifying locations with historical sensitive data (finance, HR, legal departments) that predate auto-labeling policies.
7+
2. **Access the scan creation interface** in the Microsoft Purview Portal: Information Protection > Classifiers > On-demand classification OR Data Loss Prevention > Classifiers > On-demand classification.
8+
3. **Select target locations** (specific SharePoint sites, OneDrive accounts, and/or Exchange mailboxes) and **choose sensitive information types to detect** (credit card numbers, SSNs, healthcare identifiers, trade secrets).
9+
4. **Configure scan settings** including confidence thresholds (lower = more matches but higher false positives; higher = fewer false positives but may miss data) and file type filters. For trainable classifiers, ensure high-quality training data.
10+
5. **Schedule or run the scan** immediately for baseline scans or set recurring schedules (daily/weekly/monthly). Note: Large scans can take days or weeks depending on data volume and may impact resource utilization.
11+
6. **Monitor progress and analyze results** by tracking completion in the Microsoft Purview Portal. Upon completion, identify sensitive data locations, review prevalence by type, and determine remediation actions.
12+
13+
- [On-demand classification in Microsoft Purview](https://learn.microsoft.com/en-us/purview/on-demand-classification)
14+
- [Sensitive information types entity reference](https://learn.microsoft.com/en-us/purview/sensitive-information-type-entity-definitions)
15+
16+
<!--- Results --->
17+
%TestResult%

‎src/powershell/tests/Test-Assessment.35022.ps1‎

Lines changed: 329 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,329 @@
1+
<#
2+
.SYNOPSIS
3+
On-Demand Scans Configured for Sensitive Information Discovery
4+
5+
.DESCRIPTION
6+
Checks if on-demand scans are configured for sensitive information discovery in
7+
SharePoint, OneDrive, and Exchange. Implements dynamic SIT GUID -> friendly name
8+
resolution and generates a markdown result suitable for inclusion in test reports.
9+
10+
Reference: https://learn.microsoft.com/en-us/purview/on-demand-classification
11+
12+
.NOTES
13+
Test ID: 35022
14+
Pillar: Data
15+
Risk Level: Medium
16+
User Impact: Low
17+
Implementation Cost: Medium
18+
#>
19+
20+
function Test-Assessment-35022 {
21+
[ZtTest(
22+
Category = 'Information Protection',
23+
ImplementationCost = 'Medium',
24+
MinimumLicense = 'Microsoft 365 E5',
25+
Pillar = 'Data',
26+
RiskLevel = 'Medium',
27+
SfiPillar = 'Protect tenants and production systems',
28+
TenantType = 'Workforce',
29+
TestId = 35022,
30+
Title = 'On-Demand scans configured for sensitive information discovery',
31+
UserImpact = 'Low'
32+
)]
33+
[CmdletBinding()]
34+
param()
35+
36+
#region Data Collection
37+
Write-PSFMessage '🟦 Start' -Tag Test -Level VeryVerbose
38+
39+
$activity = 'Checking On-Demand Scans Configured for Sensitive Information Discovery'
40+
Write-ZtProgress -Activity $activity -Status 'Getting SIT Catalog'
41+
42+
$sitGuidMap = @{}
43+
$scansList = $null
44+
$errorMsg = $null
45+
46+
try {
47+
# Build dynamic SIT catalog from tenant
48+
$sitCatalog = Get-DlpSensitiveInformationType -ErrorAction Stop
49+
foreach ($sit in $sitCatalog) {
50+
try {
51+
$id = $null
52+
$id = $sit.Identity
53+
$name = $sit.Name
54+
$sitGuidMap[$id] = $name
55+
}
56+
catch {
57+
# Ignore individual SIT failures, continue
58+
}
59+
}
60+
}
61+
catch {
62+
Write-PSFMessage "Warning: Failed to build SIT catalog from tenant: $($_.Exception.Message)" -Level Warning
63+
}
64+
65+
# Fallback common SIT mapping
66+
$fallbackMap = @{
67+
'50842eb7-edc8-4019-85dd-5a5c1f2bb085' = 'Credit Card Number'
68+
'a44669fe-0d48-453d-a9b1-2cc83f2cba77' = 'U.S. Social Security Number (SSN)'
69+
'ed36cf51-9d63-40f3-a9a6-5a865c418d21' = 'U.S. Bank Account Number'
70+
'48ee9090-3f74-4238-89c9-6c0a93767a8f' = 'SWIFT Code'
71+
'50f56e32-3a6f-459f-82e9-e2b27b96b430' = 'Drivers License Number (U.S.)'
72+
'65ce4b3d-79b3-46c0-ba9d-8226d98130c8' = 'IBAN (International Banking Account Number)'
73+
'3b35900d-fd2d-446b-b3ad-b4723419e2d5' = 'ABA Routing Number'
74+
'f3dbc5dd-e2d4-4487-b43c-ebd87f349aa4' = 'Canada Social Insurance Number'
75+
'f87b75b6-570d-465d-a91a-f0d9b9e0b000' = 'U.K. National Insurance Number (NINO)'
76+
'b3a2fd72-cc1b-40fc-b0dc-6c5ca0e00f6f' = 'International Medical Record Number (MRN)'
77+
}
78+
79+
Write-ZtProgress -Activity $activity -Status 'Getting On-Demand Scans'
80+
81+
try {
82+
$scansList = Get-SensitiveInformationScan -ErrorAction Stop
83+
}
84+
catch {
85+
$errorMsg = $_
86+
Write-PSFMessage "Error querying on-demand scans: $_" -Level Error
87+
}
88+
#endregion Data Collection
89+
90+
#region Assessment Logic
91+
$scanCount = 0
92+
$passed = $false
93+
$tableData = @()
94+
$statusCounts = @{}
95+
$hasSharePoint = 0
96+
$hasOneDrive = 0
97+
$hasExchange = 0
98+
$customStatus = $null
99+
$mostRecentScan = $null
100+
101+
if ($errorMsg) {
102+
$passed = $false
103+
}
104+
else {
105+
$scanCount = @($scansList).Count
106+
$passed = $scanCount -ge 1
107+
if ($scanCount -gt 0) {
108+
foreach ($scan in $scansList) {
109+
# Use scan object directly - already contains full details from Get-SensitiveInformationScan
110+
# Normalize fields
111+
$name = $scan.Name
112+
$status = $scan.SensitiveInformationScanStatus
113+
114+
# Workload may be string or array
115+
$workload = ''
116+
if ($scan.Workload -is [System.Collections.IEnumerable] -and -not ($scan.Workload -is [string])) {
117+
$workload = ($scan.Workload -join ', ')
118+
}
119+
else {
120+
$workload = $scan.Workload
121+
}
122+
123+
# Parse ItemStatistics.SIT
124+
$sitDetails = @()
125+
# Put into list to simulate cmdlet output
126+
try {
127+
if ($scan.ItemStatistics -and $scan.ItemStatistics.SIT) {
128+
$sits = $scan.ItemStatistics.SIT
129+
130+
# Determine SIT keys depending on object type
131+
if ($sits -is [System.Collections.IDictionary]) {
132+
$sitKeys = $sits.Keys
133+
}
134+
elseif ($sits -is [PSCustomObject]) {
135+
$sitKeys = $sits.PSObject.Properties | ForEach-Object { $_.Name }
136+
}
137+
else {
138+
$sitKeys = @()
139+
}
140+
141+
foreach ($guid in $sitKeys) {
142+
$guidString = $guid.ToString().Trim()
143+
144+
# Obtain count for this GUID
145+
$count = 0
146+
if ($sits -is [System.Collections.IDictionary]) {
147+
$count = $sits[$guid]
148+
}
149+
else {
150+
try {
151+
$count = $sits.$guid
152+
}
153+
catch {
154+
$count = 0
155+
}
156+
}
157+
158+
# 🔹 SPEC RULE: Ignore SITs with zero matches
159+
if (-not $count -or $count -le 0) {
160+
continue
161+
}
162+
163+
# Resolve SIT GUID to friendly name
164+
$friendlyName = $null
165+
if ($sitGuidMap.ContainsKey($guidString)) {
166+
$friendlyName = $sitGuidMap[$guidString]
167+
}
168+
elseif ($fallbackMap.ContainsKey($guidString)) {
169+
$friendlyName = $fallbackMap[$guidString]
170+
}
171+
else {
172+
try {
173+
$sitObj = Get-DlpSensitiveInformationType -Identity $guidString -ErrorAction SilentlyContinue
174+
if ($sitObj) {
175+
if ($sitObj.PSObject.Properties['Name']) {
176+
$friendlyName = $sitObj.Name
177+
}
178+
elseif ($sitObj.PSObject.Properties['DisplayName']) {
179+
$friendlyName = $sitObj.DisplayName
180+
}
181+
else {
182+
$friendlyName = $sitObj.ToString()
183+
}
184+
}
185+
}
186+
catch {}
187+
}
188+
189+
if (-not $friendlyName) {
190+
$friendlyName = "Unknown SIT - $guidString"
191+
}
192+
193+
$sitDetails += "$friendlyName`: $count matches"
194+
}
195+
}
196+
}
197+
catch {
198+
199+
}
200+
201+
$sitString = if ($sitDetails.Count -gt 0) {
202+
$sitDetails -join "; "
203+
}
204+
else {
205+
'None'
206+
}
207+
208+
$createdUtc = ''
209+
if ($scan.WhenCreatedUTC) {
210+
$createdUtc = $scan.WhenCreatedUTC
211+
}
212+
$lastScanStart = ''
213+
if ($scan.LastScanStartTime) {
214+
$lastScanStart = $scan.LastScanStartTime
215+
}
216+
217+
# Build output row
218+
$row = [PSCustomObject]@{
219+
Name = $name
220+
Status = $status
221+
Workload = $workload
222+
'SIT Detected' = $sitString
223+
'Created (UTC)' = $createdUtc
224+
'Last Scan Start' = $lastScanStart
225+
}
226+
$tableData += $row
227+
228+
# Status counts
229+
if ($status -ne '') {
230+
if ($statusCounts.ContainsKey($status)) {
231+
$statusCounts[$status]++
232+
}
233+
else {
234+
$statusCounts[$status] = 1
235+
}
236+
}
237+
}
238+
}
239+
240+
# Workload coverage counts
241+
$hasSharePoint = (@($scansList) | Where-Object { $_.Workload -and (($_.Workload -contains 'SharePoint') -or (($_.Workload -join ',') -match 'SharePoint')) }).Count
242+
$hasOneDrive = (@($scansList) | Where-Object { $_.Workload -and (($_.Workload -contains 'OneDrive') -or (($_.Workload -join ',') -match 'OneDrive')) }).Count
243+
$hasExchange = (@($scansList) | Where-Object { $_.Workload -and (($_.Workload -contains 'Exchange') -or (($_.Workload -join ',') -match 'Exchange')) }).Count
244+
245+
# Most recent scan start
246+
$mostRecentScan = @($scansList) | Where-Object { $_.LastScanStartTime } | Sort-Object LastScanStartTime -Descending | Select-Object -First 1 | ForEach-Object { $_.LastScanStartTime }
247+
}
248+
249+
250+
#endregion Assessment Logic
251+
252+
#region Report Generation
253+
$testResultMarkdown = ""
254+
255+
if ($errorMsg) {
256+
$testResultMarkdown = "Unable to determine on-demand scan configuration due to permissions issues or query failure.`n`n"
257+
$customStatus = 'Investigate'
258+
}
259+
else {
260+
if ($passed) {
261+
$testResultMarkdown = "✅ At least one on-demand scan is configured in the organization, enabling discovery and classification of historical sensitive information.`n`n"
262+
}
263+
else {
264+
$testResultMarkdown = "❌ No on-demand scans are configured in the organization; historical sensitive data cannot be discovered.`n`n"
265+
}
266+
267+
$testResultMarkdown += "### On-Demand scan configuration summary`n`n"
268+
269+
if ($scanCount -gt 0 -and $tableData) {
270+
$testResultMarkdown += "**Scan details:**`n`n"
271+
$testResultMarkdown += "| Name | Sensitive information scan status | Workload | Sensitive information types detected | When created UTC | Last scan start time|`n"
272+
$testResultMarkdown += "|------|--------|----------|--------------|---------------|-----------------|`n"
273+
274+
foreach ($row in $tableData) {
275+
$nameEsc = $row.Name
276+
$statusEsc = $row.Status
277+
$workEsc = $row.Workload
278+
$sitEsc = $row.'SIT Detected'
279+
$created = if ($row.'Created (UTC)') {
280+
Get-FormattedDate -DateString $row.'Created (UTC)'
281+
}
282+
else {
283+
''
284+
}
285+
$last = if ($row.'Last Scan Start') {
286+
Get-FormattedDate -DateString $row.'Last Scan Start'
287+
}
288+
else {
289+
''
290+
}
291+
292+
$testResultMarkdown += "| $nameEsc | $statusEsc | $workEsc | $sitEsc | $created | $last |`n"
293+
}
294+
295+
$testResultMarkdown += "`n**Summary:**`n`n"
296+
$testResultMarkdown += "* **Total on-demand scans configured:** $scanCount`n"
297+
$testResultMarkdown += "* **Scans by status:**`n"
298+
foreach ($status in ($statusCounts.Keys | Sort-Object)) {
299+
$testResultMarkdown += " * $status`: $($statusCounts[$status])`n"
300+
}
301+
$testResultMarkdown += "* **Locations scanned:**`n"
302+
$testResultMarkdown += " * SharePoint: $(if ($hasSharePoint -gt 0) { 'Yes' } else { 'No' })`n"
303+
$testResultMarkdown += " * OneDrive: $(if ($hasOneDrive -gt 0) { 'Yes' } else { 'No' })`n"
304+
$testResultMarkdown += " * Exchange: $(if ($hasExchange -gt 0) { 'Yes' } else { 'No' })`n"
305+
$testResultMarkdown += "* **Most recent scan completion:** $(if ($mostRecentScan) { $mostRecentScan } else { 'No completed scans' })`n"
306+
}
307+
else {
308+
$testResultMarkdown += "* **Total on-demand scans configured:** 0`n"
309+
$testResultMarkdown += "* **Status:** No scans are configured`n"
310+
}
311+
312+
$testResultMarkdown += "`n[Microsoft Purview Portal > Information Protection > Classifiers > On-demand classification](https://purview.microsoft.com/informationprotection/dataclassification/colddatascans)`n"
313+
$testResultMarkdown += "or"
314+
$testResultMarkdown += "`n[Microsoft Purview Portal > Data Loss Prevention > Classifiers > On-demand classification](https://purview.microsoft.com/datalossprevention/dataclassification/colddatascans)`n"
315+
316+
}
317+
#endregion Report Generation
318+
319+
$params = @{
320+
TestId = '35022'
321+
Title = 'On-Demand scans configured for sensitive information discovery'
322+
Status = $passed
323+
Result = $testResultMarkdown
324+
}
325+
if ($null -ne $customStatus) {
326+
$params.CustomStatus = $customStatus
327+
}
328+
Add-ZtTestResultDetail @params
329+
}

0 commit comments

Comments
 (0)