Question Analysis
A Linux host’s Syslog shows a shell-based network discovery script ran multiple commands. One of them listed current TCP connections. Which utility was invoked?
c141bbdb-7fca-4254-9fd6-f47e79447e17
Question ID
7/16
Models Correct
1
Valid Answer
44%
Success Rate
Reference Answer
Human-validated correct answer for this question
netstat
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | netstat | DeviceProcessEvents
| where ProcessCommandLine has_any ("netstat", "ss", "lsof") or FileName has_any ("netstat", "ss", "lsof")
| project Timestamp, DeviceName, FileName, ProcessCommandLine
| sort by Timestamp desc | Correct | 2 | 16.10s | 0.0079 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0108 |
| gpt-4-turbo-2024-04-09 | netstat | DeviceProcessEvents
| where InitiatingProcessCommandLine has_any ('netstat', 'ss -t', 'ss -a', 'ss -r')
| project DeviceName, InitiatingProcessCommandLine, ProcessCreationTime | Correct | 1 | 10.12s | 0.1152 |
| gpt-4.1 | netstat | DeviceProcessEvents
| where FileName in~ ('netstat', 'ss', 'lsof')
or ProcessCommandLine contains "netstat"
or ProcessCommandLine contains "ss "
or ProcessCommandLine contains "lsof -i"
| summarize count() by FileName, ProcessCommandLine, DeviceName, AccountName, TimeGenerated
| sort by count_ desc, TimeGenerated desc | Correct | 1 | 7.21s | 0.0325 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0172 |
| gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0084 |
| gpt-4.1-nano | ss | DeviceProcessEvents | where ProcessCommandLine contains 'netstat' or ProcessCommandLine contains 'ss' | project ProcessCommandLine, InitiatingProcessCreationTime, DeviceName | order by InitiatingProcessCreationTime desc | take 50 | Incorrect | 4 | 23.63s | 0.0016 |
| gpt-4o | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0524 |
| grok-3-beta | netstat | DeviceProcessEvents
| where ProcessCommandLine contains 'netstat' or ProcessCommandLine contains 'ss' or ProcessCommandLine contains 'lsof'
| where FileName in ('netstat', 'ss', 'lsof')
| project DeviceName, FileName, ProcessCommandLine, ProcessCreationTime, InitiatingProcessFileName, InitiatingProcessCommandLine
| order by ProcessCreationTime desc | Correct | 1 | 10.47s | 0.0259 |
| grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0074 |
| o1-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.7670 |
| o1-low | netstat | DeviceProcessEvents
| where DeviceId in (DeviceInfo
| where OSPlatform == "Linux"
| distinct DeviceId)
| where ActionType == "ProcessCreated"
| where ProcessCommandLine has_any ("netstat", "ss ", "lsof -i")
| project TimeGenerated, DeviceName, FileName, ProcessCommandLine, InitiatingProcessCommandLine | Correct | 1 | 34.25s | 0.3251 |
| o3-mini-high | netstat | DeviceProcessEvents
| where tolower(InitiatingProcessCommandLine) contains "netstat" or tolower(InitiatingProcessCommandLine) contains " ss "
| project TimeGenerated, DeviceName, InitiatingProcessFileName, InitiatingProcessCommandLine
| distinct TimeGenerated, DeviceName, InitiatingProcessFileName, InitiatingProcessCommandLine | Correct | 3 | 35.37s | 0.0368 |
| o3-mini-low | netstat | DeviceProcessEvents
| where InitiatingProcessCommandLine contains "netstat"
| project TimeGenerated, DeviceName, InitiatingProcessFileName, InitiatingProcessCommandLine
| sort by TimeGenerated desc | Correct | 2 | 44.17s | 0.0252 |
| o4-mini-high | sh | DeviceInfo
| where OSPlatform == "Linux"
| project DeviceId, DeviceName
| join kind=inner (
DeviceProcessEvents
// look for commands referencing TCP connections
| where ProcessCommandLine contains "tcp"
// common utilities for listing TCP connections
| where ProcessCommandLine contains "netstat" or ProcessCommandLine contains "ss" or ProcessCommandLine contains "lsof"
) on DeviceId
| extend Utility = tostring(split(ProcessCommandLine, ' ')[0])
| summarize Invocations = count() by DeviceName, Utility
| project DeviceName, Utility, Invocations
| sort by Invocations desc | Incorrect | 1 | 45.88s | 0.0114 |
| o4-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0385 |