gpt-4-turbo-2024-04-09 vs gpt-4.1-mini KQL Benchmark

gpt-4.1-mini wins by 2.1%

Compared on 188 shared test questions

Overall Accuracy

gpt-4-turbo-2024-04-09

39.4%

74 / 188 correct

gpt-4.1-mini

41.5%

78 / 188 correct

Average Cost per Query

gpt-4-turbo-2024-04-09: $0.1737
gpt-4.1-mini: $0.0057
gpt-4-turbo-2024-04-09 costs 2937.1% more

Average Execution Time

gpt-4-turbo-2024-04-09: 16.84s
gpt-4.1-mini: 14.13s
gpt-4-turbo-2024-04-09 takes 19.1% longer
Question-by-Question Analysis

Question-by-Question Comparison

Detailed comparison showing where each model succeeded or failed

Showing 1 to 25 of 188 questions
Page 1 of 8
T1016.001
On a Linux host, a ping command was executed to test internet connectivity. Determine which IP address was used as the ping target.
gpt-4-turbo-2024-04-09 Wins
T1016
A Linux host’s Syslog shows a shell-based network discovery script ran multiple commands. One of them listed current TCP connections. Which utility was invoked?
gpt-4-turbo-2024-04-09 Wins
T1049
A user launched a Windows command prompt and executed a built-in utility to enumerate all active network connections. Using process creation logs, identify the exact tool that produced the list of current connections.
gpt-4-turbo-2024-04-09 Wins
T1057
On a Windows host, investigate process events to find when Task Manager was launched via cmd with an unusual flag. What was the full command executed?
gpt-4-turbo-2024-04-09 Wins
T1059.007
On a Windows endpoint, wscript.exe was used to run a JScript. Identify the exact script path passed to wscript.
gpt-4-turbo-2024-04-09 Wins
T1059.004
On a Linux system, review process execution records for any shell process that set an environment variable containing executable code and then piped it into another shell instance. Determine which environment variable name was used to store the script content.
gpt-4-turbo-2024-04-09 Wins
T1070.004
A Linux host executed a native utility to overwrite and then remove a temporary file in one step. Identify the name of the file that was securely deleted by this action.
gpt-4-turbo-2024-04-09 Wins
T1070.003
On a Linux endpoint, you suspect malicious clearing of the bash history by redirecting from the null device. Explore process or file events to uncover the exact shell command that performed this action.
gpt-4-turbo-2024-04-09 Wins
T1059.004
On a Linux system, find any process creation record where awk is used with a BEGIN rule to launch a shell. What was the exact command invoked?
gpt-4-turbo-2024-04-09 Wins
T1070.005
On a Windows system, an attacker used the command prompt to remove one or more default administrative shares. Which share names were deleted?
gpt-4-turbo-2024-04-09 Wins
T1070.004
Suspiciously, the recycle bin appears empty system-wide. Determine which command was executed on Windows to clear the system's recycle bin directory, including any switches and environment variables.
gpt-4-turbo-2024-04-09 Wins
T1082
While investigating process creation logs on a Linux device, you observe a privileged hardware interrogation step used to reveal virtualization details. Which utility was invoked?
gpt-4-turbo-2024-04-09 Wins
T1082
On Windows systems, identify when the built-in Shadow Copy utility is used to enumerate existing snapshots. What was the full command executed?
gpt-4-turbo-2024-04-09 Wins
T1082
Review Windows process logs to find which built-in command was executed to reveal the system’s hostname.
gpt-4-turbo-2024-04-09 Wins
T1112
On Windows systems, disabling RDP via the registry generates registry write events. Investigate registry event logs for modifications under the Terminal Server configuration path. What is the name of the registry value that was changed to disable Remote Desktop Protocol?
gpt-4-turbo-2024-04-09 Wins
T1112
Investigate Windows registry events to identify any newly set ProxyServer entry under the user Internet Settings hive. What proxy server address was configured?
gpt-4-turbo-2024-04-09 Wins
T1124
An analyst reviewing Windows process logs wants to spot instances where a native time tool was repurposed to introduce a delay. Which full W32tm invocation, including the stripchart and period flags, appears in the logs?
gpt-4-turbo-2024-04-09 Wins
T1218.004
Review Windows process creation events for evidence of a .NET assembly being installed. Which executable was launched with an "/action=install" argument?
gpt-4-turbo-2024-04-09 Wins
T1218.010
An attacker has attempted to sideload code by invoking regsvr32.exe in a Windows host against a file that does not use the standard .dll extension. Investigate the process event logs to determine the name of the file that was registered.
gpt-4-turbo-2024-04-09 Wins
T1531
Within Windows process event logs, identify instances where the built-in net.exe utility is used to change a user account password. What was the new password argument passed in?
gpt-4-turbo-2024-04-09 Wins
T1548.001
Investigate Linux process or syslog records to find any invocation of the 'find' utility used to scan /usr/bin for files with the setuid bit. What was the full command executed?
gpt-4-turbo-2024-04-09 Wins
T1555.003
On a Windows system, PowerShell was used to gather multiple browser credential files into a temp folder and then archive them. What was the name of the resulting ZIP file?
gpt-4-turbo-2024-04-09 Wins
T1562.003
During a Linux engagement, an operator exported HISTSIZE=0 to turn off shell history and then ran another utility. Using the DeviceProcessEvents table, identify the exact command executed immediately after the history was disabled. What was the command?
gpt-4-turbo-2024-04-09 Wins
T1562.012
On a Linux host, auditing has been turned off. Review process execution or syslog data to determine which command was executed to disable the audit subsystem.
gpt-4-turbo-2024-04-09 Wins
T1571
On a Windows system, identify any PowerShell Test-NetConnection executions against an uncommon port. Which port number was checked?
gpt-4-turbo-2024-04-09 Wins
Page 1 of 8

Explore individual model performance and detailed analysis