Question Analysis
On a Windows device, examine registry event logs for modifications under the System policies path. Determine which registry value name was altered to disable the shutdown button at login.
6e0d1131-2d7e-4905-8ca5-d6172f05d03d
Question ID
6/16
Models Correct
1
Valid Answer
38%
Success Rate
Reference Answer
Human-validated correct answer for this question
shutdownwithoutlogon
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | shutdownwithoutlogon | DeviceRegistryEvents
| where ActionType == "RegistryValueSet"
| where RegistryKey contains @"Policies\System"
| project Timestamp, DeviceName, RegistryKey, RegistryValueName, RegistryValueData, ActionType | Correct | 3 | 23.70s | 0.0100 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0114 |
| gpt-4-turbo-2024-04-09 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.2317 |
| gpt-4.1 | shutdownwithoutlogon | DeviceRegistryEvents
| where tolower(RegistryKey) has @"policies\system"
| where ActionType in ("SetValue", "ModifyValue", "ValueModified", "RegistryValueSet")
| where tolower(RegistryValueName) == "shutdownwithoutlogon"
| project TimeGenerated, DeviceName, InitiatingProcessAccountName, RegistryKey, RegistryValueName, RegistryValueData, ActionType | Correct | 2 | 8.73s | 0.0215 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0428 |
| gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0077 |
| gpt-4.1-nano | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0017 |
| gpt-4o | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0477 |
| grok-3-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0889 |
| grok-3-mini-beta | shutdownwithoutlogon | DeviceRegistryEvents
| where RegistryKey contains 'HKEY_LOCAL_MACHINE\\SOFTWARE\\Policies\\System' or RegistryKey contains 'HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Policies\\System'
| where ActionType has_any ('Set', 'RegistryValueSet')
| where RegistryValueName in~ ('NoClose', 'ShutdownWithoutLogon', 'DisableShutdown')
| project Timestamp, DeviceName, RegistryKey, RegistryValueName, RegistryValueData, PreviousRegistryValueData
| order by Timestamp desc | Correct | 3 | 32.63s | 0.0044 |
| o1-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.7958 |
| o1-low | shutdownwithoutlogon | DeviceRegistryEvents
| where Timestamp >= ago(30d)
| where RegistryKey has "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Policies\\System"
| where ActionType in ("RegistryValueSet", "RegistryValueModified", "SetValue", "ModifyValue")
| where RegistryValueName in~ ("DisableShutdownButton", "ShutdownWithoutLogon", "HideShutdownButton")
| project TimeGenerated,
DeviceName,
ActionType,
RegistryKey,
RegistryValueName,
PreviousRegistryValueData,
RegistryValueData,
InitiatingProcessAccountName,
InitiatingProcessCommandLine
| order by TimeGenerated desc | Correct | 2 | 46.08s | 0.3536 |
| o3-mini-high | shutdownwithoutlogon | DeviceRegistryEvents
| where RegistryKey has "Policies\\System"
| where RegistryValueData == "0"
| summarize EventCount = count() by RegistryValueName, DeviceName
| order by EventCount desc | Correct | 1 | 11.92s | 0.0123 |
| o3-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0417 |
| o4-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0348 |
| o4-mini-low | shutdownwithoutlogon | DeviceRegistryEvents
| where ActionType == "RegistryValueSet"
| where RegistryKey startswith "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Policies\\System"
| summarize arg_max(TimeGenerated, *) by RegistryValueName
| project TimeGenerated, DeviceName, RegistryValueName, PreviousRegistryValueData, RegistryValueData
| sort by TimeGenerated desc | Correct | 4 | 125.10s | 0.0380 |