Question Analysis
Using Windows process event logs, investigate PowerShell activity around lsass.exe memory capture. What was the name of the script file invoked to perform the dump?
6502c8f0-b775-4dbd-9193-1298f56b6781
Question ID
6/16
Models Correct
1
Valid Answer
38%
Success Rate
Reference Answer
Human-validated correct answer for this question
Out-Minidump.ps1
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | Out-Minidump.ps1 | DeviceProcessEvents
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where ProcessCommandLine contains "lsass"
| where ProcessCommandLine contains "dump" or ProcessCommandLine contains "minidump" or ProcessCommandLine contains "credential" or ProcessCommandLine contains "memory"
| project Timestamp, DeviceName, AccountName, ProcessCommandLine | Correct | 1 | 19.84s | 0.0068 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0115 |
| gpt-4-turbo-2024-04-09 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.2371 |
| gpt-4.1 | Out-Minidump.ps1 | DeviceProcessEvents
| where ActionType == "ProcessCreated"
| where tolower(FileName) in ("powershell.exe", "pwsh.exe")
| where tolower(ProcessCommandLine) has_any ("lsass", "procdump", "minidump", "out-minidump", "sekurlsa")
| extend ScriptFile = extract(@"([\w\-\\.:]+\.ps1)", 1, ProcessCommandLine)
| where isnotempty(ScriptFile)
| project TimeGenerated, DeviceName, AccountName, InitiatingProcessCommandLine, ProcessCommandLine, ScriptFile
| order by TimeGenerated desc | Correct | 3 | 15.20s | 0.0280 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0364 |
| gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0080 |
| gpt-4.1-nano | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0015 |
| gpt-4o | Out-Minidump.ps1 | DeviceProcessEvents
| where InitiatingProcessFileName == "powershell.exe"
| where ProcessCommandLine has "lsass"
| extend ScriptFileName = FileName | Correct | 1 | 5.43s | 0.0233 |
| grok-3-beta | Out-Minidump.ps1 | DeviceProcessEvents
| where (InitiatingProcessFileName =~ 'powershell.exe' or FileName =~ 'powershell.exe')
| where (ProcessCommandLine contains 'lsass' or InitiatingProcessCommandLine contains 'lsass')
| project
ScriptFileName = extract('([^ \"]*\\.ps1)', 1, ProcessCommandLine),
ProcessFileName = FileName,
ProcessCommandLine,
InitiatingProcessFileName,
InitiatingProcessCommandLine,
AccountName,
AccountDomain,
DeviceName,
DeviceId,
Timestamp
| order by Timestamp desc | Correct | 1 | 12.18s | 0.0242 |
| grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0079 |
| o1-high | Out-Minidump.ps1 | DeviceProcessEvents
| where (FileName == "powershell.exe" or InitiatingProcessFileName == "powershell.exe")
and (ProcessCommandLine has "lsass" or InitiatingProcessCommandLine has "lsass")
| extend InvokedScript = extract(@"([^\s]+\.ps1)",1,ProcessCommandLine)
| project TimeGenerated, DeviceName, AccountName, InvokedScript, ProcessCommandLine, InitiatingProcessCommandLine | Correct | 1 | 33.34s | 0.2002 |
| o1-low | Out-Minidump.ps1 | DeviceProcessEvents
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where ProcessCommandLine has "lsass"
| extend ScriptFile = extract(@"(\S+\.ps1)", 1, ProcessCommandLine)
| where isnotempty(ScriptFile)
| project TimeGenerated, DeviceName, AccountName, ScriptFile, ProcessCommandLine
| distinct ScriptFile | Correct | 3 | 48.28s | 0.4114 |
| o3-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0402 |
| o3-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0425 |
| o4-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0417 |
| o4-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0345 |