gpt-4-turbo-2024-04-09 vs gpt-4.1-mini KQL Benchmark
gpt-4.1-mini wins by 2.1%
Compared on 188 shared test questions
Overall Accuracy
gpt-4-turbo-2024-04-09
39.4%
74 / 188 correct
gpt-4.1-mini
41.5%
78 / 188 correct
Average Cost per Query
gpt-4-turbo-2024-04-09: $0.1737
gpt-4.1-mini: $0.0057
gpt-4-turbo-2024-04-09 costs 2937.1% more
Average Execution Time
gpt-4-turbo-2024-04-09: 16.84s
gpt-4.1-mini: 14.13s
gpt-4-turbo-2024-04-09 takes 19.1% longer
Question-by-Question Analysis
Question-by-Question Comparison
Detailed comparison showing where each model succeeded or failed
Showing 1 to 25 of 188 questions
Page 1 of 8
T1016.001 | On a Linux host, a ping command was executed to test internet connectivity. Determine which IP address was used as the ping target. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1016 | A Linux host’s Syslog shows a shell-based network discovery script ran multiple commands. One of them listed current TCP connections. Which utility was invoked? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1049 | A user launched a Windows command prompt and executed a built-in utility to enumerate all active network connections. Using process creation logs, identify the exact tool that produced the list of current connections. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1057 | On a Windows host, investigate process events to find when Task Manager was launched via cmd with an unusual flag. What was the full command executed? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1059.007 | On a Windows endpoint, wscript.exe was used to run a JScript. Identify the exact script path passed to wscript. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1059.004 | On a Linux system, review process execution records for any shell process that set an environment variable containing executable code and then piped it into another shell instance. Determine which environment variable name was used to store the script content. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1070.004 | A Linux host executed a native utility to overwrite and then remove a temporary file in one step. Identify the name of the file that was securely deleted by this action. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1070.003 | On a Linux endpoint, you suspect malicious clearing of the bash history by redirecting from the null device. Explore process or file events to uncover the exact shell command that performed this action. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1059.004 | On a Linux system, find any process creation record where awk is used with a BEGIN rule to launch a shell. What was the exact command invoked? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1070.005 | On a Windows system, an attacker used the command prompt to remove one or more default administrative shares. Which share names were deleted? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1070.004 | Suspiciously, the recycle bin appears empty system-wide. Determine which command was executed on Windows to clear the system's recycle bin directory, including any switches and environment variables. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1082 | While investigating process creation logs on a Linux device, you observe a privileged hardware interrogation step used to reveal virtualization details. Which utility was invoked? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1082 | On Windows systems, identify when the built-in Shadow Copy utility is used to enumerate existing snapshots. What was the full command executed? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1082 | Review Windows process logs to find which built-in command was executed to reveal the system’s hostname. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1112 | On Windows systems, disabling RDP via the registry generates registry write events. Investigate registry event logs for modifications under the Terminal Server configuration path. What is the name of the registry value that was changed to disable Remote Desktop Protocol? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1112 | Investigate Windows registry events to identify any newly set ProxyServer entry under the user Internet Settings hive. What proxy server address was configured? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1124 | An analyst reviewing Windows process logs wants to spot instances where a native time tool was repurposed to introduce a delay. Which full W32tm invocation, including the stripchart and period flags, appears in the logs? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1218.004 | Review Windows process creation events for evidence of a .NET assembly being installed. Which executable was launched with an "/action=install" argument? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1218.010 | An attacker has attempted to sideload code by invoking regsvr32.exe in a Windows host against a file that does not use the standard .dll extension. Investigate the process event logs to determine the name of the file that was registered. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1531 | Within Windows process event logs, identify instances where the built-in net.exe utility is used to change a user account password. What was the new password argument passed in? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1548.001 | Investigate Linux process or syslog records to find any invocation of the 'find' utility used to scan /usr/bin for files with the setuid bit. What was the full command executed? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1555.003 | On a Windows system, PowerShell was used to gather multiple browser credential files into a temp folder and then archive them. What was the name of the resulting ZIP file? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1562.003 | During a Linux engagement, an operator exported HISTSIZE=0 to turn off shell history and then ran another utility. Using the DeviceProcessEvents table, identify the exact command executed immediately after the history was disabled. What was the command? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1562.012 | On a Linux host, auditing has been turned off. Review process execution or syslog data to determine which command was executed to disable the audit subsystem. | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
T1571 | On a Windows system, identify any PowerShell Test-NetConnection executions against an uncommon port. Which port number was checked? | ✓ | ✗ | gpt-4-turbo-2024-04-09 Wins |
Page 1 of 8
Explore individual model performance and detailed analysis