Question Analysis
A Windows host shows a process launching with install-driver switches, likely signaling malicious driver deployment. What is the name of the tool that was executed?
5cb0b071-8a5a-412f-839d-116beb2ed9f7
Question ID
3/16
Models Correct
1
Valid Answer
19%
Success Rate
Reference Answer
Human-validated correct answer for this question
pnputil.exe
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | pnputil.exe | DeviceProcessEvents
| where ActionType == "ProcessCreated"
| where (FileName =~ "pnputil.exe" and (ProcessCommandLine has "add-driver" or ProcessCommandLine has "install"))
or (FileName =~ "devcon.exe" and ProcessCommandLine has "install")
or (ProcessCommandLine has "install" and ProcessCommandLine has "driver")
| project Timestamp, DeviceName, FileName, ProcessCommandLine | Correct | 3 | 26.55s | 0.0112 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0106 |
| gpt-4-turbo-2024-04-09 | install_helper | DeviceProcessEvents
| where ProcessCommandLine contains '/install' or ProcessCommandLine contains '/driver'
| where FileName has_any ('setup', 'install', 'update')
| project FileName, ProcessCommandLine
| distinct FileName | Incorrect | 1 | 8.49s | 0.0497 |
| gpt-4.1 | pnputil.exe | DeviceProcessEvents
| where isnotempty(ProcessCommandLine) or isnotempty(InitiatingProcessCommandLine)
| where ProcessCommandLine has_any ('install', 'driver', 'install-driver', 'installdriver') or InitiatingProcessCommandLine has_any ('install', 'driver', 'install-driver', 'installdriver') or FileName has_any ('install', 'driver')
| summarize count() by FileName
| sort by count_ desc | Correct | 4 | 14.30s | 0.0354 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0301 |
| gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0056 |
| gpt-4.1-nano | pnputil.exe | DeviceProcessEvents
| where ProcessCommandLine contains 'install-driver' or ProcessCommandLine contains 'install drivers' or ProcessCommandLine contains 'driver'
| project InitiatingProcessFileName, ProcessCommandLine, InitiatingProcessCreationTime
| order by InitiatingProcessCreationTime desc | Correct | 4 | 5.33s | 0.0010 |
| gpt-4o | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0499 |
| grok-3-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0835 |
| grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0068 |
| o1-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.7220 |
| o1-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.7187 |
| o3-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0407 |
| o3-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0376 |
| o4-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0427 |
| o4-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0371 |