gpt-5-mini-low vs o1-low KQL Benchmark
o1-low wins by 17.1%
Compared on 187 shared test questions
Overall Accuracy
gpt-5-mini-low
46.0%
86 / 187 correct
o1-low
63.1%
118 / 187 correct
Average Cost per Query
gpt-5-mini-low: $0.0146
o1-low: $0.4994
o1-low costs 3320.9% more
Average Execution Time
gpt-5-mini-low: 54.72s
o1-low: 50.90s
gpt-5-mini-low takes 7.5% longer
Question-by-Question Analysis
Question-by-Question Comparison
Detailed comparison showing where each model succeeded or failed
Showing 1 to 25 of 187 questions
Page 1 of 8
T1039 | On a Windows system, someone ran PowerShell to copy a file from a remote machine’s C$ share to the local TEMP folder. Using process event logs, what full PowerShell command was executed to perform this action? | ✓ | ✗ | gpt-5-mini-low Wins |
T1036.003 | In a Linux environment, you observe a process labeled like the cron daemon but running from an unexpected path. Investigate creation events to uncover the actual filename used by this fake cron process. | ✓ | ✗ | gpt-5-mini-low Wins |
T1057 | On a Windows host, investigate process events to find when Task Manager was launched via cmd with an unusual flag. What was the full command executed? | ✓ | ✗ | gpt-5-mini-low Wins |
T1057 | While reviewing Windows process events, you spot a PowerShell process executing a WMI enumeration cmdlet. What WMI class name did the attacker query? | ✓ | ✗ | gpt-5-mini-low Wins |
T1070.003 | On a Windows endpoint, review process execution logs to see if any PowerShell sessions were wiped clean. Which command was executed to clear the PowerShell history? | ✓ | ✗ | gpt-5-mini-low Wins |
T1082 | On Windows systems, identify when the built-in Shadow Copy utility is used to enumerate existing snapshots. What was the full command executed? | ✓ | ✗ | gpt-5-mini-low Wins |
T1120 | Review Windows process and PowerShell activity for commands that enumerate PnP entities through WMI. Which PowerShell cmdlet was invoked to perform this hardware inventory? | ✓ | ✗ | gpt-5-mini-low Wins |
T1120 | Review Windows process execution logs to find any native utility that was used to enumerate connected drives. Which utility was invoked? | ✓ | ✗ | gpt-5-mini-low Wins |
T1124 | In Windows process event logs, you notice both the net time and w32tm commands being executed to display the system time and timezone. Which executor name from the test configuration was responsible for launching these utilities? | ✓ | ✗ | gpt-5-mini-low Wins |
T1059.004 | On a Linux system, review process execution records for any shell process that set an environment variable containing executable code and then piped it into another shell instance. Determine which environment variable name was used to store the script content. | ✓ | ✗ | gpt-5-mini-low Wins |
T1547.002 | A Windows host shows a suspicious registry change under the LSA hive. Review recent registry events to locate any new entries under Authentication Packages and determine the name of the DLL the attacker added. | ✓ | ✗ | gpt-5-mini-low Wins |
T1548.001 | A Linux host’s Syslog contains records of an elevated shell executing a command that granted group execute rights and enabled the SetGID bit on a file. Investigate the logs and report the name of the file whose group ID bit was modified. | ✓ | ✗ | gpt-5-mini-low Wins |
T1547 | A Windows host shows a process launching with install-driver switches, likely signaling malicious driver deployment. What is the name of the tool that was executed? | ✓ | ✗ | gpt-5-mini-low Wins |
T1546.004 | A suspicious file modification on a Linux device targeted the ~/.bash_profile file, apparently adding a new line. What was the full command string that was appended? | ✓ | ✗ | gpt-5-mini-low Wins |
T1021.006 | On Windows hosts, look through recent PowerShell execution records to find any elevated session where remote management was turned on. What exact command was run to enable PSRemoting? | ✓ | ✗ | gpt-5-mini-low Wins |
T1560.001 | A Linux host may have undergone automated data collection and compression right before sensitive information is exfiltrated. Using process execution logs, determine which archive file name was created when the tar utility was run with gzip compression. | ✓ | ✗ | gpt-5-mini-low Wins |
T1622 | On the Windows device, a security check was run to detect debugger processes via PowerShell. Which tool (process) carried out this check? | ✓ | ✗ | gpt-5-mini-low Wins |
T1555 | A security investigator suspects that someone attempted to dump stored web credentials on a Windows system using an in-built command-line tool. Review process creation logs to determine which executable was called to list the Web Credentials vault. | ✓ | ✗ | gpt-5-mini-low Wins |
T1016.001 | On a Linux host, a ping command was executed to test internet connectivity. Determine which IP address was used as the ping target. | ✗ | ✓ | o1-low Wins |
T1016 | A Linux host’s Syslog shows a shell-based network discovery script ran multiple commands. One of them listed current TCP connections. Which utility was invoked? | ✗ | ✓ | o1-low Wins |
T1003 | On a Windows host, an attacker leveraged COMSVCS.DLL via rundll32.exe to dump the memory of svchost.exe into a file under the Temp directory. Review the file creation logs to determine the exact name of the dump file that was generated. | ✗ | ✓ | o1-low Wins |
T1036.004 | Analyze Windows process events for any schtasks.exe commands that created a new task invoking PowerShell. What is the name of the .ps1 script specified to run? | ✗ | ✓ | o1-low Wins |
T1053.006 | Examine the logs from the Linux system for events related to the systemd timer activation. Identify any records indicating that a new timer unit was started and enabled, and determine which timer name was used. | ✗ | ✓ | o1-low Wins |
T1018 | Review Linux process execution records for any commands that list TCP metric cache entries and filter out loopback interfaces. Which utility was used? | ✗ | ✓ | o1-low Wins |
T1059.004 | An analyst suspects that a restricted shell escape was executed using a common Perl package manager on Linux. Review the process execution records to determine which tool was invoked to spawn the shell. | ✗ | ✓ | o1-low Wins |
Page 1 of 8
Explore individual model performance and detailed analysis