Question Analysis
On a Windows endpoint, review any events showing content being appended to a user’s PowerShell profile that introduce new process launches. What exact command line was added?
090e5aa5-32b6-473b-a49b-21e843a56896
Question ID
1/16
Models Correct
1
Valid Answer
6%
Success Rate
Reference Answer
Human-validated correct answer for this question
Start-Process calc.exe
Model Performance Comparison
See how different AI models performed on this question
Model | Answer | KQL Query | Status | Attempts | Time | Cost |
---|---|---|---|---|---|---|
gemini-2.5-flash-preview-04-17 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0364 |
gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0112 |
gpt-4-turbo-2024-04-09 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.2444 |
gpt-4.1 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0608 |
gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0545 |
gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0081 |
gpt-4.1-nano | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0013 |
gpt-4o | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0507 |
grok-3-beta | Start-Process calc.exe | let PowerShellProfilePaths = dynamic(['Microsoft.PowerShell_profile.ps1', 'profile.ps1']);
DeviceFileEvents
| where FolderPath has 'Users' and FileName in (PowerShellProfilePaths)
| where ActionType in ('FileModified', 'FileCreated', 'FileWritten')
| project Timestamp, DeviceName, AccountName = InitiatingProcessAccountName, FileName, FolderPath, ActionType, CommandLine = InitiatingProcessCommandLine
| union (
DeviceProcessEvents
| where ProcessCommandLine has 'powershell' or ProcessCommandLine has 'pwsh'
| where ProcessCommandLine has 'profile.ps1' or ProcessCommandLine has 'Microsoft.PowerShell_profile.ps1' or ProcessCommandLine has 'start' or ProcessCommandLine has 'cmd'
| project Timestamp, DeviceName, AccountName, FileName = InitiatingProcessFileName, FolderPath = InitiatingProcessFolderPath, ActionType = 'ProcessLaunch', CommandLine = ProcessCommandLine
)
| union (
DeviceRegistryEvents
| where RegistryKey has 'PowerShell' or RegistryValueData has 'profile.ps1'
| project Timestamp, DeviceName, AccountName = InitiatingProcessAccountName, FileName = '', FolderPath = RegistryKey, ActionType = 'RegistryModification', CommandLine = InitiatingProcessCommandLine
)
| order by Timestamp desc | Correct | 2 | 22.38s | 0.0625 |
grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0071 |
o1-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 1.0149 |
o1-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 1.0557 |
o3-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0426 |
o3-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0443 |
o4-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0481 |
o4-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0544 |