Question Analysis
Windows process creation logs show a PowerShell-driven file transfer to an FTP service. Which executable was leveraged to perform this exfiltration?
b854eb97-bf9b-45ab-a1b5-b94e4880c56b
Question ID
12/16
Models Correct
1
Valid Answer
75%
Success Rate
Reference Answer
Human-validated correct answer for this question
rclone.exe
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | rclone.exe | DeviceProcessEvents
| where InitiatingProcessFileName =~ "powershell.exe"
| where ProcessCommandLine has "ftp"
| distinct FileName | Correct | 1 | 7.19s | 0.0026 |
| gpt-35-turbo | rclone.exe | DeviceProcessEvents
| where ProcessCommandLine contains 'powershell' and ProcessCommandLine contains 'ftp'
| project DeviceName, ProcessId, ProcessCommandLine | Correct | 1 | 2.31s | 0.0024 |
| gpt-4-turbo-2024-04-09 | rclone.exe | DeviceProcessEvents
| where InitiatingProcessFileName == 'powershell.exe' and ProcessCommandLine has 'ftp'
| project DeviceName, InitiatingProcessFileName, FileName, ProcessCommandLine, TimeGenerated | Correct | 1 | 9.39s | 0.0541 |
| gpt-4.1 | rclone.exe | DeviceProcessEvents
| where InitiatingProcessFileName =~ "powershell.exe" or InitiatingProcessCommandLine has "powershell"
| where ProcessCommandLine has_any ("ftp", "put", "ftp:")
| where FileName !~ "powershell.exe"
| project TimeGenerated, DeviceName, AccountName, ParentProcess=InitiatingProcessFileName, ParentCommand=InitiatingProcessCommandLine, ExfilExecutable=FileName, ExfilCommand=ProcessCommandLine
| summarize ExfilCount=count() by ExfilExecutable, ExfilCommand
| sort by ExfilCount desc | Correct | 2 | 11.56s | 0.0223 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0445 |
| gpt-4.1-mini | rclone.exe | DeviceProcessEvents
| where InitiatingProcessFileName == "powershell.exe"
| where ProcessCommandLine has_any ("ftp", "-ftp", "ftp://", "ftp.")
| project TimeGenerated, DeviceName, AccountName, InitiatingProcessFileName, ProcessFileName = FileName, ProcessCommandLine
| distinct ProcessFileName, ProcessCommandLine | Correct | 1 | 6.22s | 0.0023 |
| gpt-4.1-nano | rclone.exe | DeviceProcessEvents
| where ProcessCommandLine contains 'powershell' and ProcessCommandLine contains 'ftp'
| project InitiatingProcessFileName, ProcessCommandLine, InitiatingProcessCreationTime, DeviceName | Correct | 1 | 1.84s | 0.0005 |
| gpt-4o | rclone.exe | let PowerShell_FTP_Events = DeviceProcessEvents
| where InitiatingProcessFileName =~ 'powershell.exe' and InitiatingProcessCommandLine contains 'ftp'
| project Timestamp, InitiatingProcessCommandLine, InitiatingProcessFileName, DeviceName, AccountName;
PowerShell_FTP_Events | Correct | 3 | 23.44s | 0.0350 |
| grok-3-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0923 |
| grok-3-mini-beta | rclone.exe | DeviceProcessEvents
| where InitiatingProcessFileName =~ 'powershell.exe'
| where ProcessCommandLine has "ftp" or ProcessCommandLine has "21"
| distinct FileName | Correct | 2 | 26.93s | 0.0029 |
| o1-high | rclone.exe | DeviceProcessEvents
| where ActionType == "ProcessCreated"
| where InitiatingProcessFileName =~ "powershell.exe"
| where ProcessCommandLine has_any ("ftp", "ftp://", "FtpWebRequest")
| project TimeGenerated, DeviceName, FileName, FolderPath, ProcessCommandLine
| order by TimeGenerated desc | Correct | 1 | 25.88s | 0.1952 |
| o1-low | rclone.exe | DeviceProcessEvents
| where ActionType == "ProcessCreated"
| where InitiatingProcessFileName =~ "powershell.exe"
| where ProcessCommandLine has "ftp"
| project TimeGenerated, DeviceName, InitiatingProcessFileName, ProcessCommandLine, FileName, FolderPath
| order by TimeGenerated desc | Correct | 1 | 20.68s | 0.1803 |
| o3-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0515 |
| o3-mini-low | rclone.exe | union withsource=TableName DeviceProcessEvents, DeviceEvents
| extend ParentProc = tolower(InitiatingProcessFileName), ChildProc = tolower(FileName)
| where ParentProc == "powershell.exe"
| where (tostring(ProcessCommandLine) contains "ftp" or tostring(InitiatingProcessCommandLine) contains "ftp")
| where ChildProc != "powershell.exe"
| project TimeGenerated, DeviceName, ProcessId, FileName, InitiatingProcessFileName, ProcessCommandLine, InitiatingProcessCommandLine
| sort by TimeGenerated asc | Correct | 5 | 149.76s | 0.0520 |
| o4-mini-high | rclone.exe | DeviceProcessEvents
| where InitiatingProcessFileName in ("powershell.exe", "pwsh.exe")
| where ProcessCommandLine has "ftp"
| project TimeGenerated, DeviceName, InitiatingProcessCommandLine, ExfiltrationProcess = FileName, ProcessCommandLine
| summarize Count = count() by ExfiltrationProcess, ProcessCommandLine
| order by Count desc | Correct | 3 | 81.96s | 0.0248 |
| o4-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0413 |