gpt-4o vs gpt-5-high KQL Benchmark
gpt-5-high wins by 25.5%
Compared on 188 shared test questions
Overall Accuracy
gpt-4o
37.8%
71 / 188 correct
gpt-5-high
63.3%
119 / 188 correct
Average Cost per Query
gpt-4o: $0.0433
gpt-5-high: $0.1529
gpt-5-high costs 252.9% more
Average Execution Time
gpt-4o: 14.30s
gpt-5-high: 192.47s
gpt-5-high takes 1245.8% longer
Question-by-Question Analysis
Question-by-Question Comparison
Detailed comparison showing where each model succeeded or failed
Showing 1 to 25 of 188 questions
Page 1 of 8
T1021.006 | On Windows hosts, look through recent PowerShell execution records to find any elevated session where remote management was turned on. What exact command was run to enable PSRemoting? | ✓ | ✗ | gpt-4o Wins |
T1059.004 | Which full interactive shell command, as recorded in the Linux process logs, repeatedly echoed a distinctive marker message to the terminal? | ✓ | ✗ | gpt-4o Wins |
T1059.004 | An attacker on a Linux host may try to enumerate installed shells by reading the system file that lists valid shells. Using process or syslog data, determine which command was executed to perform this enumeration. | ✓ | ✗ | gpt-4o Wins |
T1059.004 | On a Linux system, review process execution records for any shell process that set an environment variable containing executable code and then piped it into another shell instance. Determine which environment variable name was used to store the script content. | ✓ | ✗ | gpt-4o Wins |
T1176 | A Windows host shows chrome.exe starting with a --load-extension parameter. What folder name was specified in that flag? | ✓ | ✗ | gpt-4o Wins |
T1201 | Windows systems may be probed for their password policy settings using a native command-line tool. Determine which command was executed to list the local password policy on the target hosts. | ✓ | ✗ | gpt-4o Wins |
T1505.005 | A suspicious registry change was made on a Windows system modifying the Terminal Services DLL path. Investigate registry events to find out which DLL file name was set as the ServiceDll value under TermService. What was the file name? | ✓ | ✗ | gpt-4o Wins |
T1546.004 | On Linux systems, an attacker may gain persistence by appending instructions to the global shell profile. Investigate process or file modification events to find evidence of text being added to /etc/profile, and identify the exact command invocation that carried out this change. | ✓ | ✗ | gpt-4o Wins |
T1547 | A Windows host shows evidence of a driver being installed using a built-in utility. Investigate process creation events to find the INF filename that was specified in the add-driver invocation. | ✓ | ✗ | gpt-4o Wins |
T1552.003 | A Linux user’s bash history was searched for patterns like ‘pass’ and ‘ssh’, and the matching lines were redirected into a new file. Determine the name of that file. | ✓ | ✗ | gpt-4o Wins |
T1555 | An endpoint shows a PowerShell process that downloaded and executed a remote script aimed at extracting credentials from the Windows Credential Manager. Review the process creation logs and identify the function name that was invoked to dump the web credentials. | ✓ | ✗ | gpt-4o Wins |
T1562.012 | On a Linux host, auditing has been turned off. Review process execution or syslog data to determine which command was executed to disable the audit subsystem. | ✓ | ✗ | gpt-4o Wins |
T1614.001 | In a Windows environment, locate any occurrences where an elevated DISM utility was run to enumerate the system’s international (locale) settings. What was the exact command line used? | ✓ | ✗ | gpt-4o Wins |
T1006 | Identify the PowerShell cmdlet used on Windows to format and display the raw volume bytes after an attacker read the boot sector via a DOS device path. | ✗ | ✓ | gpt-5-high Wins |
T1003.008 | In a Linux environment, an elevated process was used to execute a command that read /etc/shadow and redirected its output to a file. Identify what file name was employed to store these results. | ✗ | ✓ | gpt-5-high Wins |
T1016.001 | An analyst notices a PowerShell process on a Windows host that appears to be checking SMB connectivity. Which PowerShell cmdlet was executed to perform this outbound port 445 test? | ✗ | ✓ | gpt-5-high Wins |
T1007 | An analyst suspects a user or script ran a service enumeration command on a Linux system. Review process events to find the service-listing invocation and specify the full command that was executed. | ✗ | ✓ | gpt-5-high Wins |
T1016.002 | On a Windows host, someone appears to have run a built-in network shell utility to list saved wireless network profiles and their passwords in clear text. Review the process creation logs to discover the exact command that was executed. | ✗ | ✓ | gpt-5-high Wins |
T1016 | A Linux host’s Syslog shows a shell-based network discovery script ran multiple commands. One of them listed current TCP connections. Which utility was invoked? | ✗ | ✓ | gpt-5-high Wins |
T1027 | A Windows host shows a process launch with an extremely obfuscated command line that dynamically builds and invokes code at runtime. Which process name was used to execute this payload? | ✗ | ✓ | gpt-5-high Wins |
T1027 | On a Windows endpoint, look for evidence of a base64-encoded PowerShell payload execution. Which executable launched the encoded command? | ✗ | ✓ | gpt-5-high Wins |
T1027 | On a Linux system, identify the script that was generated by decoding a base64 data file and then executed. What was the filename of that script? | ✗ | ✓ | gpt-5-high Wins |
T1036.003 | In a Linux environment, you observe a process labeled like the cron daemon but running from an unexpected path. Investigate creation events to uncover the actual filename used by this fake cron process. | ✗ | ✓ | gpt-5-high Wins |
T1036.003 | A process is running under a familiar Windows host name but originates from a user's AppData folder rather than the System32 directory. Identify the filename used to masquerade the PowerShell binary on this Windows device. | ✗ | ✓ | gpt-5-high Wins |
T1049 | In a Windows log analytics workspace, search for PowerShell processes that were used to enumerate network connections. Determine which PowerShell cmdlet was executed to list active TCP connections. | ✗ | ✓ | gpt-5-high Wins |
Page 1 of 8
Explore individual model performance and detailed analysis