Question Analysis
On a Windows host, find any scheduled task that was registered using PowerShell native cmdlets instead of schtasks.exe. What was the name given to the new task?
af9fd58f-c4ac-4bf2-a9ba-224b71ff25fd
Question ID
3/16
Models Correct
1
Valid Answer
19%
Success Rate
Reference Answer
Human-validated correct answer for this question
AtomicTask
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | "AtomicTask" | DeviceProcessEvents
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where ProcessCommandLine contains "Register-ScheduledTask"
| extend TaskNameRaw = extract(@'-TaskName\s+(.*?)(?:\s+-|$)', 1, ProcessCommandLine)
| where isnotempty(TaskNameRaw)
| extend TaskName = trim_start('"', trim_start('\'', trim_end('"', trim_end('\'', TaskNameRaw)))) // Trim potential quotes
| extend TaskName = trim_start(' ', trim_end(' ', TaskName)) // Trim potential spaces
| project Timestamp, DeviceName, InitiatingProcessCommandLine, TaskName
| distinct Timestamp, DeviceName, TaskName, InitiatingProcessCommandLine | Correct | 5 | 104.99s | 0.0749 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0109 |
| gpt-4-turbo-2024-04-09 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.2423 |
| gpt-4.1 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0362 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0337 |
| gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0072 |
| gpt-4.1-nano | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0013 |
| gpt-4o | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0533 |
| grok-3-beta | AtomicTask | DeviceProcessEvents
| where InitiatingProcessFileName =~ 'powershell.exe'
| where ProcessCommandLine has_any ('New-ScheduledTask', 'Register-ScheduledTask')
| where ProcessCommandLine !has 'schtasks.exe'
| project Timestamp, DeviceName, InitiatingProcessFileName, ProcessCommandLine, TaskName = extract('(?i)TaskName\\s*=\\s*["]?([^"\\s]+)["]?', 1, ProcessCommandLine)
| order by Timestamp desc | Correct | 3 | 19.69s | 0.0572 |
| grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0075 |
| o1-high | AtomicTask | DeviceProcessEvents
| where InitiatingProcessFileName in~ ("powershell.exe", "pwsh.exe", "powershell_ise.exe")
| where ProcessCommandLine has_any ("Register-ScheduledTask", "Set-ScheduledTask", "New-ScheduledTask")
| extend TaskNameQuoted = extract('-TaskName\\s+"([^"]+)"', 1, ProcessCommandLine)
| extend TaskNameUnquoted = extract('-TaskName\\s+(\\S+)', 1, ProcessCommandLine)
| extend ScheduledTaskName = coalesce(TaskNameQuoted, TaskNameUnquoted)
| project TimeGenerated, DeviceName, InitiatingProcessAccountName, ScheduledTaskName, ProcessCommandLine
| order by TimeGenerated desc | Correct | 4 | 150.43s | 0.8065 |
| o1-low | None | DeviceProcessEvents
| where FileName in~ ("powershell.exe", "pwsh.exe")
| where ProcessCommandLine has_any ("Register-ScheduledTask", "New-ScheduledTask")
| extend TaskName = extract("-TaskName\\s+\\\"([^\\\"]+)\\\"", 1, ProcessCommandLine)
| project TimeGenerated, DeviceName, AccountName, TaskName
| order by TimeGenerated desc | Incorrect | 5 | 120.09s | 0.8895 |
| o3-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0429 |
| o3-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0409 |
| o4-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0480 |
| o4-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0579 |