Question Analysis
Review registry event logs on the Windows host for PowerShell-driven writes to system policy and file system keys. Which registry value names were created during this BlackByte preparation simulation?
0b79c06f-c788-44a2-8630-d69051f1123d
Question ID
2/16
Models Correct
3
Valid Answers
13%
Success Rate
Reference Answers
Human-validated correct answers for this question
LocalAccountTokenFilterPolicyEnableLinkedConnectionsLongPathsEnabled
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | '', 'Blob', 'Debugger', 'PromptOnSecureDesktop', 'UseLogonCredential', 'ConsentPromptBehaviorAdmin', 'DisableAntiSpyware', 'DelegateExecute', 'AutoRun', 'Userinit', 'Shell', 'ReportingMode', 'EnableLinkedConnections', 'ServiceDll', 'Common Startup', 'Startup', 'GlobalFlag', 'BootExecute', 'Security Packages', 'LegalNoticeText', 'StubPath', 'ProcessCreationIncludeCmdLine_Enabled', 'DisableAntiVirus', 'MonitorProcess', 'LegalNoticeCaption', 'ASRRules', 'AmsiEnable', 'Shell-backup', 'Userinit-backup', 'atomictest', 'socks5_powershell', 'NextRun' | DeviceRegistryEvents
| where ActionType == "RegistryValueSet"
| where InitiatingProcessFileName =~ "powershell.exe"
| distinct RegistryValueName | Correct | 1 | 9.01s | 0.0036 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0110 |
| gpt-4-turbo-2024-04-09 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.2253 |
| gpt-4.1 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0433 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0829 |
| gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0064 |
| gpt-4.1-nano | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0014 |
| gpt-4o | GlobalFlag, ReportingMode, MonitorProcess, AmsiEnable | DeviceRegistryEvents
| where InitiatingProcessFileName contains "powershell" and (ActionType contains "Create" or ActionType contains "Add")
| project TimeGenerated, DeviceName, RegistryKey, RegistryValueName, RegistryValueData, InitiatingProcessCommandLine, InitiatingProcessFileName | Incorrect | 1 | 5.37s | 0.0241 |
| grok-3-beta | Shell | DeviceRegistryEvents
| where TimeGenerated between (startofday(ago(7d)) .. endofday(now()))
| where InitiatingProcessFileName has_cs 'powershell'
| where ActionType in ('RegistryValueSet', 'RegistryValueWritten')
| summarize by RegistryValueName, DeviceName, RegistryKey, InitiatingProcessFileName
| project RegistryValueName, DeviceName, RegistryKey, InitiatingProcessFileName | Incorrect | 3 | 37.47s | 0.0617 |
| grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0070 |
| o1-high | None | DeviceRegistryEvents
| where ActionType contains "create" // focuses on new registry value creations
| where InitiatingProcessCommandLine has "powershell" // identifies PowerShell-driven changes
| where RegistryKey has_any ("Policy", "FileSystem") // filters only relevant policy/file system keys
| summarize CreatedRegistryValueNames = make_set(RegistryValueName) | Incorrect | 1 | 23.52s | 0.1943 |
| o1-low | PromptOnSecureDesktop_ConsentPromptBehaviorAdmin_DisableAntiSpyware_EnableLinkedConnections_LegalNoticeText_ProcessCreationIncludeCmdLine_Enabled_DisableAntiVirus_LegalNoticeCaption_ASRRules_atomictest | DeviceRegistryEvents
| where InitiatingProcessFileName endswith "powershell.exe"
| where RegistryKey has_any ("Policies", "FileSystem")
| where ActionType in ("RegistryValueSet", "RegistryValueCreated", "SetValue")
| distinct RegistryValueName | Correct | 1 | 45.09s | 0.2942 |
| o3-mini-high | atomictest | DeviceRegistryEvents
| where ActionType in ("RegistryValueCreated", "RegistryValueSet")
| where (InitiatingProcessCommandLine has_cs "powershell" or InitiatingProcessFileName has_cs "powershell.exe" or InitiatingProcessCommandLine has_cs "pwsh" or InitiatingProcessFileName has_cs "pwsh.exe")
| where tolower(RegistryKey) has "policies" or tolower(RegistryKey) has "filesystem"
| distinct RegistryValueName | Incorrect | 4 | 103.35s | 0.0391 |
| o3-mini-low | atomictest | DeviceRegistryEvents
| where InitiatingProcessFileName has "powershell.exe"
| where RegistryKey has_any ("Policies", "FileSystem")
| distinct RegistryValueName | Incorrect | 1 | 26.99s | 0.0135 |
| o4-mini-high | ASRRules, DisableAntiVirus, DisableAntiSpyware | DeviceRegistryEvents
| where ActionType in ("RegistryValueSet", "RegistryValueAdded")
| where tolower(InitiatingProcessFileName) has "powershell"
| where RegistryKey has "\\SYSTEM\\CurrentControlSet\\Control\\FileSystem" or RegistryKey has "\\SOFTWARE\\Policies"
| where isnotempty(RegistryValueName)
| summarize CreatedValueNames = make_set(RegistryValueName) by RegistryKey
| project RegistryKey, CreatedValueNames | Incorrect | 2 | 50.65s | 0.0167 |
| o4-mini-low | ASRRules, DisableAntiSpyware, DisableAntiVirus | DeviceRegistryEvents
| where InitiatingProcessFileName =~ "powershell.exe"
| where ActionType == "RegistryValueSet"
| where RegistryKey startswith "HKEY_LOCAL_MACHINE\\SOFTWARE\\Policies"
or RegistryKey startswith "HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Policies"
or RegistryKey startswith "HKEY_LOCAL_MACHINE\\SYSTEM\\CurrentControlSet\\Control\\FileSystem"
| distinct RegistryValueName
| sort by RegistryValueName asc | Incorrect | 2 | 57.16s | 0.0223 |