Question Analysis
On Windows devices, identify the full command used when the built-in credential manager utility was invoked to list stored credentials. What was the exact command?
56506854-89d6-46a3-9804-b7fde90791f9
Question ID
15/16
Models Correct
1
Valid Answer
94%
Success Rate
Reference Answer
Human-validated correct answer for this question
cmdkey /list
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | cmdkey.exe /list | DeviceProcessEvents
| where ActionType == "ProcessCreated"
| where FileName == "cmdkey.exe"
| where ProcessCommandLine has "/list"
| project Timestamp, DeviceName, AccountName, ProcessCommandLine | Correct | 1 | 14.20s | 0.0054 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0108 |
| gpt-4-turbo-2024-04-09 | cmdkey.exe /list | DeviceProcessEvents
| where FileName in ('cmd.exe', 'powershell.exe')
| where ProcessCommandLine has 'cmdkey'
| project DeviceName, InitiatingProcessCommandLine, ProcessCommandLine, TimeGenerated | Correct | 1 | 12.23s | 0.0532 |
| gpt-4.1 | cmdkey.exe /list | DeviceProcessEvents
| where FileName =~ "cmdkey.exe"
| where ProcessCommandLine contains "/list"
| project TimeGenerated, DeviceName, AccountName, FileName, ProcessCommandLine
| order by TimeGenerated desc | Correct | 1 | 5.45s | 0.0111 |
| gpt-4.1-finetuned | cmdkey.exe /list | DeviceProcessEvents
| where ProcessCommandLine has "cmdkey.exe" and ProcessCommandLine has "list"
| summarize ProcessCommandLineSet = makeset(ProcessCommandLine) by DeviceId, DeviceName
| extend HostName = tostring(split(DeviceName, '.') [0]), DnsDomain = tostring(strcat_array(array_slice(split(DeviceName, '.'), 1, -1), '.'))
| project-away DeviceName, DeviceId
| extend HostName = tolower(HostName)
| extend DnsDomain = tolower(DnsDomain)
| Correct | 1 | 7.25s | 0.0036 |
| gpt-4.1-mini | cmdkey.exe /list | DeviceProcessEvents
| where InitiatingProcessCommandLine contains "cmdkey" and InitiatingProcessCommandLine contains "/list"
| project TimeGenerated, DeviceName, InitiatingProcessCommandLine
| order by TimeGenerated desc | Correct | 2 | 21.17s | 0.0031 |
| gpt-4.1-nano | cmdkey.exe /list | DeviceProcessEvents
| where ProcessCommandLine contains 'cmd.exe' or ProcessCommandLine contains 'powershell.exe' or ProcessCommandLine contains 'rundll32.exe'
| where ProcessCommandLine contains 'CredentialManager' or ProcessCommandLine contains '/list' or ProcessCommandLine contains 'credman' or ProcessCommandLine contains 'cmdkey'
| project Timestamp, DeviceName, ProcessCommandLine
| order by Timestamp desc
| take 10 | Correct | 5 | 27.42s | 0.0017 |
| gpt-4o | cmdkey.exe /list | DeviceProcessEvents
| where ProcessCommandLine contains "cmdkey.exe" and ProcessCommandLine has "list"
| project DeviceId, DeviceName, ProcessCommandLine | Correct | 3 | 28.58s | 0.0343 |
| grok-3-beta | cmdkey.exe /list | DeviceProcessEvents
| where FileName == 'cmdkey.exe' and ProcessCommandLine contains '/list'
| project DeviceName, ProcessCommandLine, ProcessCreationTime, InitiatingProcessFileName, InitiatingProcessCommandLine
| order by ProcessCreationTime desc | Correct | 1 | 9.70s | 0.0215 |
| grok-3-mini-beta | cmdkey.exe /list | DeviceProcessEvents
| where FileName == 'cmdkey.exe'
| where ProcessCommandLine contains '/list'
| project TimeGenerated, DeviceName, ProcessCommandLine, AccountName | Correct | 1 | 11.63s | 0.0015 |
| o1-high | cmdkey.exe /list | DeviceProcessEvents
| where FileName =~ "cmdkey.exe"
| where ProcessCommandLine has "/list"
| project TimeGenerated, DeviceId, DeviceName, AccountDomain, AccountName, ProcessCommandLine
| order by TimeGenerated desc | Correct | 1 | 41.01s | 0.2296 |
| o1-low | cmdkey.exe /list | DeviceProcessEvents
| where FileName =~ "cmdkey.exe" or FileName =~ "vaultcmd.exe"
| where ProcessCommandLine contains "list"
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine, FolderPath
| order by TimeGenerated desc | Correct | 1 | 89.57s | 0.4932 |
| o3-mini-high | cmdkey.exe /list | union
(DeviceProcessEvents
| where FileName == "cmdkey.exe" and ProcessCommandLine has "/list"
| project TimeGenerated, DeviceName, FullCommand = ProcessCommandLine),
(DeviceProcessEvents
| where FileName == "rundll32.exe" and ProcessCommandLine has "keymgr.dll,KRShowKeyMgr"
| project TimeGenerated, DeviceName, FullCommand = ProcessCommandLine)
| sort by TimeGenerated desc | Correct | 1 | 16.29s | 0.0113 |
| o3-mini-low | cmdkey.exe /list | DeviceProcessEvents
| where tolower(FileName) == "cmdkey.exe"
| where ProcessCommandLine contains "/list"
| project TimeGenerated, DeviceName, ProcessCommandLine
| order by TimeGenerated desc | Correct | 1 | 34.17s | 0.0109 |
| o4-mini-high | cmdkey.exe /list | DeviceProcessEvents
| where ProcessCommandLine has_cs "cmdkey" and ProcessCommandLine has_cs "/list"
| project TimeGenerated, DeviceName, InitiatingProcessAccountName, ProcessCommandLine
| distinct TimeGenerated, DeviceName, InitiatingProcessAccountName, ProcessCommandLine
| order by TimeGenerated desc | Correct | 1 | 40.44s | 0.0150 |
| o4-mini-low | cmdkey.exe /list | DeviceProcessEvents
| where FileName =~ "cmdkey.exe"
| where ProcessCommandLine has "/list"
| project TimeGenerated, DeviceName, DeviceId, InitiatingProcessAccountName, ProcessCommandLine
| order by TimeGenerated desc | Correct | 3 | 116.97s | 0.0298 |