Question Analysis
An analyst notices a PowerShell process on a Windows host that appears to be checking SMB connectivity. Which PowerShell cmdlet was executed to perform this outbound port 445 test?
d9c32b3b-7916-45ad-aca5-6c902da80319
Question ID
6/16
Models Correct
1
Valid Answer
38%
Success Rate
Reference Answer
Human-validated correct answer for this question
Test-NetConnection
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | Test-NetConnection | DeviceNetworkEvents
| where RemotePort == 445
| where InitiatingProcessFileName =~ "powershell.exe"
| distinct InitiatingProcessCommandLine | Correct | 1 | 12.16s | 0.0057 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0114 |
| gpt-4-turbo-2024-04-09 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.3084 |
| gpt-4.1 | New-Object | DeviceProcessEvents
| where (FileName in~ ('powershell.exe', 'pwsh.exe'))
| where ProcessCommandLine has_any ('Test-NetConnection', 'System.Net.Sockets.TcpClient', 'Test-Connection', 'Invoke-Expression', 'New-Object') and ProcessCommandLine has '445'
| project TimeGenerated, DeviceName, AccountName, FileName, ProcessCommandLine
| order by TimeGenerated desc | Incorrect | 2 | 11.38s | 0.0248 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0394 |
| gpt-4.1-mini | Test-NetConnection | DeviceProcessEvents
| where FileName in ("powershell.exe", "pwsh.exe")
| join kind=inner (
DeviceNetworkEvents
| where RemotePort == 445
| project InitiatingProcessId, RemotePort
) on $left.ProcessId == $right.InitiatingProcessId
| project TimeGenerated, DeviceName, InitiatingProcessAccountName, ProcessCommandLine
| order by TimeGenerated desc
| take 50 | Correct | 4 | 16.37s | 0.0182 |
| gpt-4.1-nano | Test-NetConnection -CommonTCPPort SMB | DeviceProcessEvents
| where ProcessCommandLine contains "PowerShell" or FileName endswith ".ps1"
| where ProcessCommandLine contains "Test-NetConnection" or ProcessCommandLine contains "Invoke-Expression" or ProcessCommandLine contains "net" or ProcessCommandLine contains "smb"
| project TimeGenerated, DeviceName, ProcessCommandLine, ProcessId, InitiatingProcessFileName, InitiatingProcessParentFileName | Correct | 2 | 3.23s | 0.0013 |
| gpt-4o | Invoke-AtomicTest | DeviceProcessEvents
| where InitiatingProcessFileName =~ "powershell.exe" and ProcessCommandLine has "445"
| project TimeGenerated, DeviceName, AccountName, InitiatingProcessCommandLine, InitiatingProcessVersionInfoFileDescription | Incorrect | 3 | 9.38s | 0.0319 |
| grok-3-beta | ConnectAsync | DeviceNetworkEvents
| where InitiatingProcessFileName =~ 'powershell.exe'
| where RemotePort == 445
| where ActionType == 'ConnectionSuccess' or ActionType == 'OutboundConnection'
| project Timestamp, DeviceName, InitiatingProcessFileName, InitiatingProcessCommandLine, RemoteIP, RemotePort
| order by Timestamp desc
| limit 100 | Incorrect | 1 | 12.62s | 0.0235 |
| grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0074 |
| o1-high | Test-NetConnection | DeviceProcessEvents
| where ActionType == "ProcessCreated"
| where FileName in~ ("powershell.exe","pwsh.exe","powershell_ise.exe")
// Look for references to a port test in command line
| where ProcessCommandLine has_any ("Test-NetConnection", "Test-Connection", "-TcpPort", "445")
| project TimeGenerated, DeviceName, AccountName, InitiatingProcessCommandLine, ProcessCommandLine
| order by TimeGenerated desc | Correct | 3 | 89.57s | 0.5518 |
| o1-low | Test-NetConnection | // This query checks both DeviceProcessEvents and DeviceNetworkEvents to find
// PowerShell commands that used the 'Test-NetConnection' (or 'tnc') cmdlet
// specifically mentioning port 445.
union DeviceProcessEvents, DeviceNetworkEvents
| where TimeGenerated >= ago(7d) // Adjust time range as needed
| where (FileName in~ ("powershell.exe", "pwsh.exe") or InitiatingProcessFileName in~ ("powershell.exe", "pwsh.exe"))
| where tostring(ProcessCommandLine) has_any ("Test-NetConnection", "tnc") or tostring(InitiatingProcessCommandLine) has_any ("Test-NetConnection", "tnc")
| where tostring(ProcessCommandLine) has "445" or tostring(InitiatingProcessCommandLine) has "445" or RemotePort == 445
| project TimeGenerated, DeviceName, Type,
FileName, ProcessCommandLine, InitiatingProcessFileName, InitiatingProcessCommandLine,
RemoteIP, RemotePort
| order by TimeGenerated desc | Correct | 2 | 52.68s | 0.3693 |
| o3-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0339 |
| o3-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0396 |
| o4-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0421 |
| o4-mini-low | Test-NetConnection | DeviceNetworkEvents
| where InitiatingProcessFileName == "powershell.exe" // only PowerShell processes
| where RemotePort == 445 // SMB port
| project Timestamp, DeviceName, InitiatingProcessId, InitiatingProcessCommandLine
| distinct InitiatingProcessCommandLine | Correct | 1 | 48.94s | 0.0142 |