gpt-4.1-finetuned vs gpt-5-high KQL Benchmark
gpt-5-high wins by 37.2%
Compared on 188 shared test questions
Overall Accuracy
gpt-4.1-finetuned
26.1%
49 / 188 correct
gpt-5-high
63.3%
119 / 188 correct
Average Cost per Query
gpt-4.1-finetuned: $0.0414
gpt-5-high: $0.1529
gpt-5-high costs 269.5% more
Average Execution Time
gpt-4.1-finetuned: 33.24s
gpt-5-high: 192.47s
gpt-5-high takes 479.1% longer
Question-by-Question Analysis
Question-by-Question Comparison
Detailed comparison showing where each model succeeded or failed
Showing 1 to 25 of 188 questions
Page 1 of 8
T1021.006 | On Windows hosts, look through recent PowerShell execution records to find any elevated session where remote management was turned on. What exact command was run to enable PSRemoting? | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1059.004 | An attacker on a Linux host may try to enumerate installed shells by reading the system file that lists valid shells. Using process or syslog data, determine which command was executed to perform this enumeration. | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1070.003 | On a Windows endpoint, commands are no longer being logged to PowerShell history, suggesting PSReadLine settings were altered. Using process execution logs, determine the exact command that was run to set the history save style to 'SaveNothing'. | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1124 | In Windows process event logs, you notice both the net time and w32tm commands being executed to display the system time and timezone. Which executor name from the test configuration was responsible for launching these utilities? | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1217 | An attacker leveraged a PowerShell command on a Windows host to enumerate browser bookmark files across all user profiles. Examine the process execution logs to determine the exact filename that was being searched for. | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1201 | On a Linux system, logs show that the password expiration settings file was accessed. Identify which command was executed to list its contents. | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1505.005 | A suspicious registry change was made on a Windows system modifying the Terminal Services DLL path. Investigate registry events to find out which DLL file name was set as the ServiceDll value under TermService. What was the file name? | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1531 | Within Windows process event logs, identify instances where the built-in net.exe utility is used to change a user account password. What was the new password argument passed in? | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1547 | A Windows host shows evidence of a driver being installed using a built-in utility. Investigate process creation events to find the INF filename that was specified in the add-driver invocation. | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1562.012 | On a Linux host, auditing has been turned off. Review process execution or syslog data to determine which command was executed to disable the audit subsystem. | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1614.001 | In a Windows environment, locate any occurrences where an elevated DISM utility was run to enumerate the system’s international (locale) settings. What was the exact command line used? | ✓ | ✗ | gpt-4.1-finetuned Wins |
T1003.001 | Using Windows process event logs, investigate PowerShell activity around lsass.exe memory capture. What was the name of the script file invoked to perform the dump? | ✗ | ✓ | gpt-5-high Wins |
T1003.008 | In a Linux environment, an elevated process was used to execute a command that read /etc/shadow and redirected its output to a file. Identify what file name was employed to store these results. | ✗ | ✓ | gpt-5-high Wins |
T1006 | Identify the PowerShell cmdlet used on Windows to format and display the raw volume bytes after an attacker read the boot sector via a DOS device path. | ✗ | ✓ | gpt-5-high Wins |
T1007 | An analyst suspects a user or script ran a service enumeration command on a Linux system. Review process events to find the service-listing invocation and specify the full command that was executed. | ✗ | ✓ | gpt-5-high Wins |
T1016.001 | An analyst notices a PowerShell process on a Windows host that appears to be checking SMB connectivity. Which PowerShell cmdlet was executed to perform this outbound port 445 test? | ✗ | ✓ | gpt-5-high Wins |
T1016 | A Linux host’s Syslog shows a shell-based network discovery script ran multiple commands. One of them listed current TCP connections. Which utility was invoked? | ✗ | ✓ | gpt-5-high Wins |
T1018 | On a Windows endpoint, review process creation logs to uncover when a built-in utility was used to reveal ARP entries. What exact command was used to list the ARP cache? | ✗ | ✓ | gpt-5-high Wins |
T1018 | A Windows host executed an ICMP-based network reconnaissance using a looping instruction in cmd.exe. Identify the exact command line that was used to perform the ping sweep. | ✗ | ✓ | gpt-5-high Wins |
T1027 | A Windows host shows a process launch with an extremely obfuscated command line that dynamically builds and invokes code at runtime. Which process name was used to execute this payload? | ✗ | ✓ | gpt-5-high Wins |
T1036.003 | A process is running under a familiar Windows host name but originates from a user's AppData folder rather than the System32 directory. Identify the filename used to masquerade the PowerShell binary on this Windows device. | ✗ | ✓ | gpt-5-high Wins |
T1039 | On a Windows system, someone ran PowerShell to copy a file from a remote machine’s C$ share to the local TEMP folder. Using process event logs, what full PowerShell command was executed to perform this action? | ✗ | ✓ | gpt-5-high Wins |
T1036.004 | Analyze Windows process events for any schtasks.exe commands that created a new task invoking PowerShell. What is the name of the .ps1 script specified to run? | ✗ | ✓ | gpt-5-high Wins |
T1036.003 | In a Linux environment, you observe a process labeled like the cron daemon but running from an unexpected path. Investigate creation events to uncover the actual filename used by this fake cron process. | ✗ | ✓ | gpt-5-high Wins |
T1048.003 | A Linux host briefly hosted an HTTP service under /tmp. Examine process creation logs to determine the exact python3 command that was used to start the server on port 9090. | ✗ | ✓ | gpt-5-high Wins |
Page 1 of 8
Explore individual model performance and detailed analysis