gpt-4-turbo-2024-04-09 vs gpt-4o KQL Benchmark

gpt-4-turbo-2024-04-09 wins by 1.6%

Compared on 188 shared test questions

Overall Accuracy

gpt-4-turbo-2024-04-09

39.4%

74 / 188 correct

gpt-4o

37.8%

71 / 188 correct

Average Cost per Query

gpt-4-turbo-2024-04-09: $0.1737
gpt-4o: $0.0433
gpt-4-turbo-2024-04-09 costs 300.9% more

Average Execution Time

gpt-4-turbo-2024-04-09: 16.84s
gpt-4o: 14.30s
gpt-4-turbo-2024-04-09 takes 17.7% longer
Question-by-Question Analysis

Question-by-Question Comparison

Detailed comparison showing where each model succeeded or failed

Showing 1 to 25 of 188 questions
Page 1 of 8
T1016.002
On a Windows host, someone appears to have run a built-in network shell utility to list saved wireless network profiles and their passwords in clear text. Review the process creation logs to discover the exact command that was executed.
gpt-4-turbo-2024-04-09 Wins
T1016
A Linux host’s Syslog shows a shell-based network discovery script ran multiple commands. One of them listed current TCP connections. Which utility was invoked?
gpt-4-turbo-2024-04-09 Wins
T1027
On a Windows endpoint, look for evidence of a base64-encoded PowerShell payload execution. Which executable launched the encoded command?
gpt-4-turbo-2024-04-09 Wins
T1036.003
In a Linux environment, you observe a process labeled like the cron daemon but running from an unexpected path. Investigate creation events to uncover the actual filename used by this fake cron process.
gpt-4-turbo-2024-04-09 Wins
T1036.004
A threat actor on a Windows system crafted and registered a service named almost identically to the standard time service, but redirecting execution to a custom script. Review the logging data to determine which native command-line tool was used to perform this action. What utility was invoked?
gpt-4-turbo-2024-04-09 Wins
T1049
In a Windows log analytics workspace, search for PowerShell processes that were used to enumerate network connections. Determine which PowerShell cmdlet was executed to list active TCP connections.
gpt-4-turbo-2024-04-09 Wins
T1057
On a Windows device, PowerShell was used to collect a snapshot of running processes. Identify the exact cmdlet that was executed.
gpt-4-turbo-2024-04-09 Wins
T1057
On a Windows host, investigate process events to find when Task Manager was launched via cmd with an unusual flag. What was the full command executed?
gpt-4-turbo-2024-04-09 Wins
T1059.004
An analyst suspects that a restricted shell escape was executed using a common Perl package manager on Linux. Review the process execution records to determine which tool was invoked to spawn the shell.
gpt-4-turbo-2024-04-09 Wins
T1059.007
On a Windows endpoint, wscript.exe was used to run a JScript. Identify the exact script path passed to wscript.
gpt-4-turbo-2024-04-09 Wins
T1070.003
On a Windows endpoint, commands are no longer being logged to PowerShell history, suggesting PSReadLine settings were altered. Using process execution logs, determine the exact command that was run to set the history save style to 'SaveNothing'.
gpt-4-turbo-2024-04-09 Wins
T1070.004
A Linux host executed a native utility to overwrite and then remove a temporary file in one step. Identify the name of the file that was securely deleted by this action.
gpt-4-turbo-2024-04-09 Wins
T1070.005
On a Windows system, an attacker used the command prompt to remove one or more default administrative shares. Which share names were deleted?
gpt-4-turbo-2024-04-09 Wins
T1070.003
On a Linux system, you suspect someone erased their command history by linking the history file to /dev/null. Investigate process events and determine which utility was executed to achieve this.
gpt-4-turbo-2024-04-09 Wins
T1078.003
Review the Linux process creation records to find which user account management utility was used to reactivate the previously locked and expired account.
gpt-4-turbo-2024-04-09 Wins
T1112
On Windows systems, disabling RDP via the registry generates registry write events. Investigate registry event logs for modifications under the Terminal Server configuration path. What is the name of the registry value that was changed to disable Remote Desktop Protocol?
gpt-4-turbo-2024-04-09 Wins
T1112
On a Windows endpoint, review the registry write events to spot when the WDigest key is altered to permit plaintext credential storage. What registry value name was changed?
gpt-4-turbo-2024-04-09 Wins
T1124
An analyst reviewing Windows process logs wants to spot instances where a native time tool was repurposed to introduce a delay. Which full W32tm invocation, including the stripchart and period flags, appears in the logs?
gpt-4-turbo-2024-04-09 Wins
T1124
On a Linux host, an activity was recorded where the local clock and timezone were queried. Review the available process execution logs to uncover what full command was run to fetch the system time and timezone.
gpt-4-turbo-2024-04-09 Wins
T1201
You are reviewing Linux syslog records on a CentOS/RHEL 7.x server. You notice entries for shell commands that access system configuration files under /etc/security. Determine exactly which configuration file was being inspected by the command.
gpt-4-turbo-2024-04-09 Wins
T1218.010
An attacker has attempted to sideload code by invoking regsvr32.exe in a Windows host against a file that does not use the standard .dll extension. Investigate the process event logs to determine the name of the file that was registered.
gpt-4-turbo-2024-04-09 Wins
T1217
On Linux, review the process execution logs to uncover when Chromium’s bookmark JSON files were being located and the results persisted. Focus on shell commands that search under .config/chromium and write output to a file. What was the filename used to save the findings?
gpt-4-turbo-2024-04-09 Wins
T1542.001
Investigate Windows file creation logs to uncover any new executable added directly to the System32 directory, which may indicate a UEFI persistence implant. What was the name of the file created?
gpt-4-turbo-2024-04-09 Wins
T1531
Within Windows process event logs, identify instances where the built-in net.exe utility is used to change a user account password. What was the new password argument passed in?
gpt-4-turbo-2024-04-09 Wins
T1548.002
On a Windows endpoint, someone may have disabled the secure desktop for elevation prompts by modifying a registry setting. Review the registry event logs to identify which registry value name was changed to 0.
gpt-4-turbo-2024-04-09 Wins
Page 1 of 8

Explore individual model performance and detailed analysis