gpt-5-mini-low vs grok-3-mini-beta KQL Benchmark
grok-3-mini-beta wins by 12.3%
Compared on 187 shared test questions
Overall Accuracy
gpt-5-mini-low
46.0%
86 / 187 correct
grok-3-mini-beta
58.3%
109 / 187 correct
Average Cost per Query
gpt-5-mini-low: $0.0146
grok-3-mini-beta: $0.0040
gpt-5-mini-low costs 268.2% more
Average Execution Time
gpt-5-mini-low: 54.72s
grok-3-mini-beta: 22.38s
gpt-5-mini-low takes 144.5% longer
Question-by-Question Analysis
Question-by-Question Comparison
Detailed comparison showing where each model succeeded or failed
Showing 1 to 25 of 187 questions
Page 1 of 8
T1016.001 | An analyst notices a PowerShell process on a Windows host that appears to be checking SMB connectivity. Which PowerShell cmdlet was executed to perform this outbound port 445 test? | ✓ | ✗ | gpt-5-mini-low Wins |
T1039 | On a Windows system, someone ran PowerShell to copy a file from a remote machine’s C$ share to the local TEMP folder. Using process event logs, what full PowerShell command was executed to perform this action? | ✓ | ✗ | gpt-5-mini-low Wins |
T1003.001 | Using Windows process event logs, investigate PowerShell activity around lsass.exe memory capture. What was the name of the script file invoked to perform the dump? | ✓ | ✗ | gpt-5-mini-low Wins |
T1036.003 | In a Linux environment, you observe a process labeled like the cron daemon but running from an unexpected path. Investigate creation events to uncover the actual filename used by this fake cron process. | ✓ | ✗ | gpt-5-mini-low Wins |
T1036.004 | A threat actor on a Windows system crafted and registered a service named almost identically to the standard time service, but redirecting execution to a custom script. Review the logging data to determine which native command-line tool was used to perform this action. What utility was invoked? | ✓ | ✗ | gpt-5-mini-low Wins |
T1057 | While reviewing Windows process events, you spot a PowerShell process executing a WMI enumeration cmdlet. What WMI class name did the attacker query? | ✓ | ✗ | gpt-5-mini-low Wins |
T1070.006 | On a Windows host, suspicious PowerShell activity adjusted the system clock and recorded a value. What numeric value was used to slip the system date? | ✓ | ✗ | gpt-5-mini-low Wins |
T1070.004 | A Linux host executed a native utility to overwrite and then remove a temporary file in one step. Identify the name of the file that was securely deleted by this action. | ✓ | ✗ | gpt-5-mini-low Wins |
T1070.004 | On a Linux device, a file was silently removed from the /tmp/victim-files directory. Search through file event or syslog records to identify the exact file name that was deleted. | ✓ | ✗ | gpt-5-mini-low Wins |
T1120 | Review Windows process and PowerShell activity for commands that enumerate PnP entities through WMI. Which PowerShell cmdlet was invoked to perform this hardware inventory? | ✓ | ✗ | gpt-5-mini-low Wins |
T1120 | Review Windows process execution logs to find any native utility that was used to enumerate connected drives. Which utility was invoked? | ✓ | ✗ | gpt-5-mini-low Wins |
T1124 | In Windows process event logs, you notice both the net time and w32tm commands being executed to display the system time and timezone. Which executor name from the test configuration was responsible for launching these utilities? | ✓ | ✗ | gpt-5-mini-low Wins |
T1124 | On a Linux host, an activity was recorded where the local clock and timezone were queried. Review the available process execution logs to uncover what full command was run to fetch the system time and timezone. | ✓ | ✗ | gpt-5-mini-low Wins |
T1059.004 | On a Linux system, review process execution records for any shell process that set an environment variable containing executable code and then piped it into another shell instance. Determine which environment variable name was used to store the script content. | ✓ | ✗ | gpt-5-mini-low Wins |
T1547.002 | A Windows host shows a suspicious registry change under the LSA hive. Review recent registry events to locate any new entries under Authentication Packages and determine the name of the DLL the attacker added. | ✓ | ✗ | gpt-5-mini-low Wins |
T1548.001 | A Linux host’s Syslog contains records of an elevated shell executing a command that granted group execute rights and enabled the SetGID bit on a file. Investigate the logs and report the name of the file whose group ID bit was modified. | ✓ | ✗ | gpt-5-mini-low Wins |
T1547 | A Windows host shows a process launching with install-driver switches, likely signaling malicious driver deployment. What is the name of the tool that was executed? | ✓ | ✗ | gpt-5-mini-low Wins |
T1021.006 | On Windows hosts, look through recent PowerShell execution records to find any elevated session where remote management was turned on. What exact command was run to enable PSRemoting? | ✓ | ✗ | gpt-5-mini-low Wins |
T1560.001 | A Linux host may have undergone automated data collection and compression right before sensitive information is exfiltrated. Using process execution logs, determine which archive file name was created when the tar utility was run with gzip compression. | ✓ | ✗ | gpt-5-mini-low Wins |
T1548.001 | Investigate Linux process or syslog records to find any invocation of the 'find' utility used to scan /usr/bin for files with the setuid bit. What was the full command executed? | ✓ | ✗ | gpt-5-mini-low Wins |
T1560 | Windows system logs show PowerShell zipping up the contents of a user’s profile folder. Investigate process and file events to determine the exact name of the ZIP archive created. | ✓ | ✗ | gpt-5-mini-low Wins |
T1555 | A security investigator suspects that someone attempted to dump stored web credentials on a Windows system using an in-built command-line tool. Review process creation logs to determine which executable was called to list the Web Credentials vault. | ✓ | ✗ | gpt-5-mini-low Wins |
T1003 | On a Windows host, an attacker leveraged COMSVCS.DLL via rundll32.exe to dump the memory of svchost.exe into a file under the Temp directory. Review the file creation logs to determine the exact name of the dump file that was generated. | ✗ | ✓ | grok-3-mini-beta Wins |
T1053.006 | Examine the logs from the Linux system for events related to the systemd timer activation. Identify any records indicating that a new timer unit was started and enabled, and determine which timer name was used. | ✗ | ✓ | grok-3-mini-beta Wins |
T1057 | On a Windows device, review the process execution logs to find instances where a built-in listing tool was piped into a string filter. Identify the process name that the attacker was searching for. | ✗ | ✓ | grok-3-mini-beta Wins |
Page 1 of 8
Explore individual model performance and detailed analysis