Question Analysis

Investigate Windows registry modification events to find the name of the registry value that was changed under the WindowsFirewall policy path when someone turned the firewall off.

a4651931-ebbb-4cde-9363-ddf3d66214cb
Question ID
4/23
Models Correct
1
Valid Answer
17%
Success Rate
Reference Answer

Human-validated correct answer for this question

EnableFirewall
Model Performance Comparison

See how different AI models performed on this question

ModelAnswerKQL QueryStatusAttemptsTimeCost
gemini-2.5-flash-preview-04-17
EnableFirewall
DeviceRegistryEvents | where ActionType == "RegistryValueSet" | where RegistryKey contains "WindowsFirewall" | project Timestamp, DeviceName, InitiatingProcessCommandLine, RegistryKey, RegistryValueName, PreviousRegistryValueData, RegistryValueData, ActionType
Correct
1
13.20s
0.0063
gpt-35-turbo
No answer provided
No query generated
Incorrect
5
0.00s
0.0110
gpt-4-turbo-2024-04-09
No answer provided
No query generated
Incorrect
5
0.00s
0.2353
gpt-4.1
No answer provided
No query generated
Incorrect
5
0.00s
0.0412
gpt-4.1-finetuned
No answer provided
No query generated
Incorrect
5
0.00s
0.0434
gpt-4.1-mini
No answer provided
No query generated
Incorrect
5
0.00s
0.0077
gpt-4.1-nano
No answer provided
No query generated
Incorrect
5
0.00s
0.0015
gpt-4o
No answer provided
No query generated
Incorrect
5
0.00s
0.0530
gpt-5-high
No answer provided
No query generated
Incorrect
5
0.00s
0.2364
gpt-5-mini-high
No answer provided
No query generated
Incorrect
5
0.00s
0.0402
gpt-5-mini-low
No answer provided
No query generated
Incorrect
5
0.00s
0.0107
gpt-5-mini-medium
No answer provided
No query generated
Incorrect
5
0.00s
0.0141
gpt-5-nano-high
No answer provided
No query generated
Incorrect
5
0.00s
0.0077
gpt-5-nano-medium
No answer provided
No query generated
Incorrect
5
0.00s
0.0064
grok-3-beta
No answer provided
No query generated
Incorrect
5
0.00s
0.0957
grok-3-mini-beta
EnableFirewall
DeviceRegistryEvents | where ActionType == 'RegistryValueSet' | where RegistryKey has 'WindowsFirewall' | project RegistryValueName, RegistryKey, RegistryValueData, Timestamp | summarize arg_max(Timestamp, *) by RegistryValueName, RegistryKey // This helps get the latest change per unique value name and key
Correct
1
15.00s
0.0016
o1-high
EnableFirewall
DeviceRegistryEvents | where ActionType in ("RegistryValueSet", "RegistryValueModified") | where RegistryKey has_any ("WindowsFirewall", "FirewallPolicy") | project TimeGenerated, DeviceName, InitiatingProcessAccountName, ActionType, RegistryKey, RegistryValueName, PreviousRegistryValueData, RegistryValueData | order by TimeGenerated desc
Correct
5
128.98s
0.9753
o1-low
No answer provided
No query generated
Incorrect
5
0.00s
1.1727
o3-high
EnableFirewall
// Investigate registry modifications that disabled Windows Firewall and return the value name that was flipped off // Time range is supplied by the hunting portal / API wrapper DeviceRegistryEvents // 1. Only registry–value changes (set / modify / delete) | where ActionType in ("RegistryValueSet", // Defender event name "RegValueModified", // legacy "RegistryKeyValueModified", // Win-10/11 "RegistryValueDeleted") // value removed ‑ also disables FW // 2. Limit to the Windows Firewall policy locations (GPO and local) | where RegistryKey has "\\WindowsFirewall" // GPO based – HKLM\SOFTWARE\Policies\Microsoft\WindowsFirewall\* or RegistryKey has "\\FirewallPolicy" // Local policy – HKLM\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\* // 3. We are interested in the moment it was turned OFF – EnableFirewall == 0 or value removed | extend NewValue = tostring(RegistryValueData), OldValue = tostring(PreviousRegistryValueData) | where RegistryValueName =~ "EnableFirewall" // the switch for every profile (Domain / Private / Public) and (NewValue == "0" or isnull(NewValue) or NewValue == "") // 4. (Optional) make sure it was previously on | where isempty(OldValue) or OldValue !in ("0", "False", "Off") // 5. Return a single event per device/profile with the data we care about | summarize LastTimeDisabled = arg_max(TimeGenerated, *) by DeviceId, RegistryKey | project LastTimeDisabled, // when it was switched off DeviceName, RegistryPath = RegistryKey, DisabledBy = strcat(InitiatingProcessAccountDomain, "\\", InitiatingProcessAccountName), Process = InitiatingProcessFileName, ProcessCmd = InitiatingProcessCommandLine, NewValue, OldValue, RegistryValueName // 6. If you only need the value-name that was flipped, uncomment the next line // | summarize by RegistryValueName
Correct
2
67.22s
0.0381
o3-mini-high
No answer provided
No query generated
Incorrect
5
0.00s
0.0399
o3-mini-low
No answer provided
No query generated
Incorrect
5
0.00s
0.0427
o4-mini-high
No answer provided
No query generated
Incorrect
5
0.00s
0.0491
o4-mini-low
No answer provided
No query generated
Incorrect
5
0.00s
0.0360