Question Analysis
A Windows host shows a suspicious registry change under the LSA hive. Review recent registry events to locate any new entries under Authentication Packages and determine the name of the DLL the attacker added.
be2590e8-4ac3-47ac-b4b5-945820f2fbe9
Question ID
3/16
Models Correct
1
Valid Answer
19%
Success Rate
Reference Answer
Human-validated correct answer for this question
package.dll
Model Performance Comparison
See how different AI models performed on this question
| Model | Answer | KQL Query | Status | Attempts | Time | Cost |
|---|---|---|---|---|---|---|
| gemini-2.5-flash-preview-04-17 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0353 |
| gpt-35-turbo | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0108 |
| gpt-4-turbo-2024-04-09 | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.2224 |
| gpt-4.1 | package.dll | DeviceRegistryEvents
| where RegistryKey has "Control\\Lsa"
| where RegistryValueName contains_cs "Authenticat"
| where ActionType in ("RegistryValueSet", "RegistryKeyValueSet", "RegistryKeyWritten", "SetValue", "RegValueSet", "Modified", "Value Set")
| project TimeGenerated, DeviceName, InitiatingProcessAccountName, InitiatingProcessFileName, RegistryKey, RegistryValueName, RegistryValueData
| order by TimeGenerated desc | Correct | 3 | 11.44s | 0.0322 |
| gpt-4.1-finetuned | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0490 |
| gpt-4.1-mini | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0073 |
| gpt-4.1-nano | msv1_0\package.dll | DeviceRegistryEvents
| where RegistryKey has_any ('HKEY_LOCAL_MACHINE\\System\\CurrentControlSet\\Control\\Lsa', 'HKEY_LOCAL_MACHINE\\System\\ControlSet001\\Control\\Lsa')
| where RegistryValueName == 'Authentication Packages'
| project TimeGenerated, RegistryKey, RegistryValueName, RegistryValueData
| order by TimeGenerated desc
| where RegistryValueData contains '.dll'
| summarize arg_max(TimeGenerated, *) by RegistryValueData | Correct | 3 | 4.83s | 0.0009 |
| gpt-4o | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0531 |
| grok-3-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0858 |
| grok-3-mini-beta | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0070 |
| o1-high | package.dll | DeviceRegistryEvents
| where RegistryKey has "\\Control\\Lsa"
| where RegistryValueName =~ "Authentication Packages"
| where ActionType in ("RegistryValueSet", "RegistryValueCreated", "RegistryValueModified")
| project TimeGenerated, DeviceName, ActionType, RegistryKey, RegistryValueName, RegistryValueData, PreviousRegistryValueData, InitiatingProcessAccountName, InitiatingProcessCommandLine
| order by TimeGenerated desc | Correct | 1 | 27.39s | 0.2260 |
| o1-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.7242 |
| o3-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0348 |
| o3-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0397 |
| o4-mini-high | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0420 |
| o4-mini-low | No answer provided | No query generated | Incorrect | 5 | 0.00s | 0.0594 |