Lookout v3.0.1: Parser fixes for Streaming/Polling API, workbook updates, threat hunting notebooks

[fgravato]

Summary

Updates Lookout solution v3.0.1 with parser fixes to support the actual streaming API data structure and adds threat hunting notebooks.

Changes Made

Parser v3.1.0

• ✅ Added coalesce() support for Streaming API (device.info.email, device.hardware.*, device.software.*, device.status.*)
• ✅ Added support for Polling API (same nested structure)
• ✅ Added support for REST API (device.email_address, device.manufacturer, etc.)
• ✅ Handles all 4 event types: THREAT, DEVICE, SMISHING_ALERT, AUDIT

Workbooks

• ✅ Updated all 5 workbooks to use correct streaming API field paths
• ✅ Fixed queries to use tostring() for safe field extraction

Notebooks (NEW)

• ✅ Lookout-ThreatHunting-MobileMalware.ipynb - Mobile malware investigation
• ✅ Lookout-ThreatHunting-Smishing.ipynb - SMS phishing campaign analysis
• ✅ Lookout-ThreatHunting-DeviceCompliance.ipynb - Device compliance hunting
• ✅ Lookout-ThreatHunting-AuditInsider.ipynb - Audit & insider threat analysis

Cleanup

• ✅ Removed development/documentation files
• ✅ Removed test scripts and validation files
• ✅ Fixed gated URLs (replaced enterprise.support.lookout.com with public URLs)

Technical Details

The Lookout MRA v2 Streaming API (/mra/stream/v2/events) returns data with a different nested structure than documented:

Field	REST API Docs	Streaming API (Actual)
Email	device.email_address	device.info.email
Manufacturer	device.manufacturer	device.hardware.manufacturer
OS Version	device.os_version	device.software.os_version

The parser now uses coalesce() to handle both:

DeviceEmailAddress = coalesce(tostring(threat.device.email), tostring(device.info.email), tostring(device.email_address))

Testing

• ✅ All analytic rules use LookoutAPI connector ID
• ✅ Parser handles multiple API structures
• ✅ Version 3.0.1 consistent across metadata files
• ✅ No hardcoded locations
• ✅ No gated URLs

Files Changed

• 29 files (Lookout solution only)
• Net: -3,134 lines (cleanup of dev files)

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[fgravato]

Fix Applied

Fixed Microsoft Sentinel branding validation error:

• Replaced 'Azure Sentinel' with 'Microsoft Sentinel' in 4 files:
  • Workbooks/LookoutEvents.json
  • Workbooks/LookoutEventsV2.json
  • Package/mainTemplate.json
  • Package/createUiDefinition.json

CI should pass now.

[v-shukore]

Hi @fgravato, could you repackage this solution using the V3 tool and generate an updated package with current version? Currently, only the maintemplate and createui files were updated; the package also needs to be updated. Thanks!

[fgravato]

Repackaged using V3 Tool

As requested by @v-shukore, I've repackaged the solution using the V3 tool (createSolutionV3.ps1).

Changes:

• ✅ Regenerated 3.0.1.zip with all solution components
• ✅ Updated mainTemplate.json
• ✅ Updated createUiDefinition.json
• ✅ All ARM-TTK validations passed

Ready for review!

[fgravato]

Hi @v-shuklasumit, could you please review this PR when you have a chance?

The latest commit fixes the Microsoft Sentinel branding validation failure - updated 'Azure Sentinel Solution' to 'Microsoft Sentinel Solution' in the workbook descriptions.

Thank you!

[v-shukore]

Hi @fgravato, we checked the following URL: https://www.lookout.com/hc/en-us/articles/115002741773-Mobile-Risk-API-Guide, and it is returning a 404 error.

Could you please share the updated workbook, parser screenshots, and screenshots of the newly added notebooks running?
Thanks!

[fgravato]

Added Notebook Screenshots

✅ Added preview screenshots for all 4 threat hunting notebooks (dark/light themes):

• Lookout-ThreatHunting-MobileMalware
• Lookout-ThreatHunting-Smishing
• Lookout-ThreatHunting-DeviceCompliance
• Lookout-ThreatHunting-AuditInsider

✅ Updated notebook outputs with sample data for documentation

Screenshots located in Notebooks/Images/Preview/

[fgravato]

Regarding the 404 URL

The documentation link https://www.lookout.com/hc/en-us/articles/115002741773-Mobile-Risk-API-Guide is returning 404 because Lookout has moved their API documentation behind their customer support portal at https://esupport.lookout.com/s/login/

This requires an active Lookout subscription to access. Unfortunately, there is no public URL available for the Mobile Risk API documentation.

[v-shukore]

Hi @fgravato,
Ok. can you share the running URL screenshot of it.

[fgravato]

## URL Screenshot

Here's the screenshot showing the Mobile Risk API V2 Guide is accessible at the Lookout Enterprise Customer Community portal:

URL: https://esupport.lookout.com/s/article/Mobile-Risk-API-V2-Guide

Note: This documentation requires a Lookout customer account to access. The page contains the API guide with a downloadable PDF version.

@v-shukore Please let me know if you need anything else!

[v-shukore]

Hi @fgravato, could you update the solution by replacing the old URL with this one wherever it appears, and then repackage the solution - https://esupport.lookout.com/s/article/Mobile-Risk-API-V2-Guide. Thanks!

[fgravato]

Updated API Documentation URL

✅ Replaced old URL https://www.lookout.com/hc/en-us/articles/115002741773-Mobile-Risk-API-Guide with https://esupport.lookout.com/s/article/Mobile-Risk-API-V2-Guide in:

• Data Connectors/Lookout_API_FunctionApp.json
• Data Connectors/LookoutStreamingConnector_ccp/LookoutStreaming_DataConnectorDefinition.json

✅ Repackaged solution using V3 tool
✅ All ARM-TTK validations passed

[v-shukore]

Hi @fgravato, could you clarify the reason for adding the package-lock.json file? Thanks

[fgravato]

@v-shukore The package-lock.json was unintentionally modified when running npm install locally (lockfile version was upgraded from v1 to v3). I've reverted it to match upstream/master.