gpt-4-turbo-2024-04-09 vs gpt-4.1 KQL Benchmark
gpt-4.1 wins by 22.3%
Compared on 188 shared test questions
Overall Accuracy
gpt-4-turbo-2024-04-09
39.4%
74 / 188 correct
gpt-4.1
61.7%
116 / 188 correct
Average Cost per Query
gpt-4-turbo-2024-04-09: $0.1737
gpt-4.1: $0.0285
gpt-4-turbo-2024-04-09 costs 509.6% more
Average Execution Time
gpt-4-turbo-2024-04-09: 16.84s
gpt-4.1: 9.95s
gpt-4-turbo-2024-04-09 takes 69.3% longer
Question-by-Question Analysis
Question-by-Question Comparison
Detailed comparison showing where each model succeeded or failed
Showing 1 to 25 of 188 questions
Page 1 of 8
T1036.003 | In a Linux environment, you observe a process labeled like the cron daemon but running from an unexpected path. Investigate creation events to uncover the actual filename used by this fake cron process. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1059.004 | An analyst suspects that a restricted shell escape was executed using a common Perl package manager on Linux. Review the process execution records to determine which tool was invoked to spawn the shell. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1059.004 | On a Linux system, review process execution records for any shell process that set an environment variable containing executable code and then piped it into another shell instance. Determine which environment variable name was used to store the script content. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1070.004 | A Linux host executed a native utility to overwrite and then remove a temporary file in one step. Identify the name of the file that was securely deleted by this action. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1059.004 | On a Linux system, find any process creation record where awk is used with a BEGIN rule to launch a shell. What was the exact command invoked? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1070.005 | On a Windows system, an attacker used the command prompt to remove one or more default administrative shares. Which share names were deleted? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1112 | A Windows host logs a change to the Terminal Server registry key disabling single-session per user. Which command-line utility executed this registry modification? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1112 | On Windows systems, disabling RDP via the registry generates registry write events. Investigate registry event logs for modifications under the Terminal Server configuration path. What is the name of the registry value that was changed to disable Remote Desktop Protocol? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1112 | Investigate Windows registry events to identify any newly set ProxyServer entry under the user Internet Settings hive. What proxy server address was configured? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1201 | You are reviewing Linux syslog records on a CentOS/RHEL 7.x server. You notice entries for shell commands that access system configuration files under /etc/security. Determine exactly which configuration file was being inspected by the command. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1531 | Within Windows process event logs, identify instances where the built-in net.exe utility is used to change a user account password. What was the new password argument passed in? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1548.002 | On a Windows endpoint, someone may have disabled the secure desktop for elevation prompts by modifying a registry setting. Review the registry event logs to identify which registry value name was changed to 0. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1562.003 | During a Linux engagement, an operator exported HISTSIZE=0 to turn off shell history and then ran another utility. Using the DeviceProcessEvents table, identify the exact command executed immediately after the history was disabled. What was the command? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1614.001 | Windows process creation logs show a cmd.exe launch that retrieved the system’s locale code page. Which exact command was executed to discover the system language? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1007 | An analyst suspects a user or script ran a service enumeration command on a Linux system. Review process events to find the service-listing invocation and specify the full command that was executed. | ✗ | ✓ | gpt-4.1 Wins |
T1006 | Identify the PowerShell cmdlet used on Windows to format and display the raw volume bytes after an attacker read the boot sector via a DOS device path. | ✗ | ✓ | gpt-4.1 Wins |
T1003.001 | Using Windows process event logs, investigate PowerShell activity around lsass.exe memory capture. What was the name of the script file invoked to perform the dump? | ✗ | ✓ | gpt-4.1 Wins |
T1018 | Review Linux process execution records for any commands that list TCP metric cache entries and filter out loopback interfaces. Which utility was used? | ✗ | ✓ | gpt-4.1 Wins |
T1027 | A Windows host shows a process launch with an extremely obfuscated command line that dynamically builds and invokes code at runtime. Which process name was used to execute this payload? | ✗ | ✓ | gpt-4.1 Wins |
T1027 | On a Linux system, identify the script that was generated by decoding a base64 data file and then executed. What was the filename of that script? | ✗ | ✓ | gpt-4.1 Wins |
T1036.003 | A process is running under a familiar Windows host name but originates from a user's AppData folder rather than the System32 directory. Identify the filename used to masquerade the PowerShell binary on this Windows device. | ✗ | ✓ | gpt-4.1 Wins |
T1053.003 | Linux hosts may log events when new files are added to /var/spool/cron/crontabs. Query those logs for a creation or write action in that directory and determine the file name that was added. | ✗ | ✓ | gpt-4.1 Wins |
T1053.005 | On Windows, review recent registry changes to detect when the MSC file association was hijacked by a reg add operation. What executable file was configured as the default command under HKCU\Software\Classes\mscfile\shell\open\command? | ✗ | ✓ | gpt-4.1 Wins |
T1053.005 | Investigate Windows process events for PowerShell activity that leverages WMI to register a scheduled task via XML import. What was the name of the XML file supplied to the RegisterByXml method? | ✗ | ✓ | gpt-4.1 Wins |
T1053.006 | Examine the logs from the Linux system for events related to the systemd timer activation. Identify any records indicating that a new timer unit was started and enabled, and determine which timer name was used. | ✗ | ✓ | gpt-4.1 Wins |
Page 1 of 8
Explore individual model performance and detailed analysis