Question Analysis
A Windows host logs show PowerShell fetching and executing a remote script to gain SeDebugPrivilege token duplication. Which Empire module was invoked?
34f0a430-9d04-4d98-bcb5-1989f14719f0
Question ID
3/16
Models Correct
1
Valid Answer
19%
Success Rate
Reference Answer
Human-validated correct answer for this question
Get-System
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | Get-System | DeviceProcessEvents
| where FileName in~ ('powershell.exe', 'pwsh.exe')
| where ActionType == 'ProcessCreated'
| where ProcessCommandLine has_any ('iex', 'Invoke-Expression') and ProcessCommandLine has_any ('DownloadString', 'Invoke-WebRequest', 'http://', 'https://')
| project Timestamp, DeviceName, AccountName, InitiatingProcessCommandLine, ProcessCommandLine, ReportId | Correct | 1 | 20.51s | 0.0098 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0115 |
| gpt-4-turbo-2024-04-09 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.2320 |
| gpt-4.1 | Get-System | DeviceProcessEvents
| where FileName =~ "powershell.exe"
| where ProcessCommandLine has_any ("SeDebugPrivilege", "token", "Invoke-TokenManipulation", "Invoke-Token")
| where ProcessCommandLine has_any ("http://", "https://") // Indicates remote script fetch
| project TimeGenerated, DeviceName, InitiatingProcessAccountName, ProcessCommandLine, FileName, FolderPath
| top 20 by TimeGenerated desc | Correct | 1 | 5.32s | 0.0118 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0420 |
| gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0079 |
| gpt-4.1-nano | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0017 |
| gpt-4o | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0556 |
| grok-3-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.1036 |
| grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0078 |
| o1-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.8012 |
| o1-low | Get-System | DeviceProcessEvents
| where FileName in~ ("powershell.exe", "powershell_ise.exe")
// Filter for PowerShell processes.
| where isnotempty(ProcessCommandLine)
// Ensure we have a command line to investigate.
| where ProcessCommandLine has_any ("IEX", "Invoke-Expression", "SeDebugPrivilege", "Invoke-TokenManipulation", "empire")
// Look for script loading, token manipulation, or Empire references.
| parse kind=regex ProcessCommandLine with * "powershell/" ModuleName " " *
// Attempt to parse out the Empire module name from the command.
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine, ModuleName
| order by TimeGenerated desc | Correct | 1 | 38.59s | 0.2849 |
| o3-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0395 |
| o3-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0432 |
| o4-mini-high | WinPwn | DeviceProcessEvents
| where FileName =~ "powershell.exe"
| where ProcessCommandLine contains "DownloadString" and ProcessCommandLine contains "IEX"
| extend EmpireModuleWithExt = extract(@"\/([^\/]+\.ps1)", 1, ProcessCommandLine)
| extend EmpireModule = replace(@"\.ps1$", "", EmpireModuleWithExt)
| project TimeGenerated, DeviceName, InitiatingProcessAccountName, ProcessCommandLine, EmpireModule | Incorrect | 2 | 63.39s | 0.0218 |
| o4-mini-low | Invoke-Seatbelt | DeviceProcessEvents
| where ProcessCommandLine matches regex "(?i)powershell.*DownloadString.*Invoke-[A-Za-z-]+"
// Extract the Empire module name (Invoke-…)
| extend EmpireModule = extract(@"(Invoke-[A-Za-z-]+)", 1, ProcessCommandLine)
| project TimeGenerated, DeviceName, ProcessCommandLine, EmpireModule
| summarize Count = count() by EmpireModule, DeviceName
| order by Count desc | Incorrect | 1 | 26.97s | 0.0067 |