gpt-4.1-nano vs gpt-5-mini-low KQL Benchmark
gpt-5-mini-low wins by 21.4%
Compared on 187 shared test questions
Overall Accuracy
gpt-4.1-nano
24.6%
46 / 187 correct
gpt-5-mini-low
46.0%
86 / 187 correct
Average Cost per Query
gpt-4.1-nano: $0.0014
gpt-5-mini-low: $0.0146
gpt-5-mini-low costs 935.5% more
Average Execution Time
gpt-4.1-nano: 10.78s
gpt-5-mini-low: 54.72s
gpt-5-mini-low takes 407.7% longer
Question-by-Question Analysis
Question-by-Question Comparison
Detailed comparison showing where each model succeeded or failed
Showing 1 to 25 of 187 questions
Page 1 of 8
T1082 | A Windows system shows a cmd.exe process spawn that appears to have been used for environment discovery. Review the process creation records to identify the exact command the adversary ran to enumerate environment variables. | ✓ | ✗ | gpt-4.1-nano Wins |
T1082 | Review Windows process logs to find which built-in command was executed to reveal the system’s hostname. | ✓ | ✗ | gpt-4.1-nano Wins |
T1082 | A user‐space process on a Linux device invoked a shell to capture and display the system’s environment variables and path. Which exact command was used to perform this discovery? | ✓ | ✗ | gpt-4.1-nano Wins |
T1112 | Evidence shows that the Windows Defender startup entry was tampered with via an elevated command prompt. Investigate registry events related to the Run key to discover which executable replaced the default SecurityHealth value. What is the name of the new program? | ✓ | ✗ | gpt-4.1-nano Wins |
T1222.002 | On a Linux host, process execution logs show a chmod invocation with a recursive flag. Which file or folder was targeted by this recursive permission change? | ✓ | ✗ | gpt-4.1-nano Wins |
T1542.001 | Investigate Windows file creation logs to uncover any new executable added directly to the System32 directory, which may indicate a UEFI persistence implant. What was the name of the file created? | ✓ | ✗ | gpt-4.1-nano Wins |
T1547 | A Windows host shows evidence of a driver being installed using a built-in utility. Investigate process creation events to find the INF filename that was specified in the add-driver invocation. | ✓ | ✗ | gpt-4.1-nano Wins |
T1552.001 | A Linux system shows a 'find' command used to search within .aws directories. Which specific AWS credential filename was the attacker attempting to locate? | ✓ | ✗ | gpt-4.1-nano Wins |
T1218.011 | You notice rundll32.exe being used with desk.cpl,InstallScreenSaver on a Windows endpoint. Investigate your process creation logs to find which .scr file was loaded by this unusual invocation. | ✓ | ✗ | gpt-4.1-nano Wins |
T1007 | An analyst suspects a user or script ran a service enumeration command on a Linux system. Review process events to find the service-listing invocation and specify the full command that was executed. | ✗ | ✓ | gpt-5-mini-low Wins |
T1003.001 | Using Windows process event logs, investigate PowerShell activity around lsass.exe memory capture. What was the name of the script file invoked to perform the dump? | ✗ | ✓ | gpt-5-mini-low Wins |
T1016.002 | On a Windows host, someone appears to have run a built-in network shell utility to list saved wireless network profiles and their passwords in clear text. Review the process creation logs to discover the exact command that was executed. | ✗ | ✓ | gpt-5-mini-low Wins |
T1021.006 | On Windows hosts, look through recent PowerShell execution records to find any elevated session where remote management was turned on. What exact command was run to enable PSRemoting? | ✗ | ✓ | gpt-5-mini-low Wins |
T1027 | On a Windows endpoint, look for evidence of a base64-encoded PowerShell payload execution. Which executable launched the encoded command? | ✗ | ✓ | gpt-5-mini-low Wins |
T1036.003 | A process is running under a familiar Windows host name but originates from a user's AppData folder rather than the System32 directory. Identify the filename used to masquerade the PowerShell binary on this Windows device. | ✗ | ✓ | gpt-5-mini-low Wins |
T1036.004 | A threat actor on a Windows system crafted and registered a service named almost identically to the standard time service, but redirecting execution to a custom script. Review the logging data to determine which native command-line tool was used to perform this action. What utility was invoked? | ✗ | ✓ | gpt-5-mini-low Wins |
T1036.003 | In a Linux environment, you observe a process labeled like the cron daemon but running from an unexpected path. Investigate creation events to uncover the actual filename used by this fake cron process. | ✗ | ✓ | gpt-5-mini-low Wins |
T1053.003 | Linux hosts may log events when new files are added to /var/spool/cron/crontabs. Query those logs for a creation or write action in that directory and determine the file name that was added. | ✗ | ✓ | gpt-5-mini-low Wins |
T1053.005 | You suspect malicious persistence via scheduled tasks on a Windows endpoint. Review the process execution logs to identify the built-in utility used to register tasks at logon or startup. What is the name of this utility? | ✗ | ✓ | gpt-5-mini-low Wins |
T1057 | A malicious actor may attempt to list running processes on a Windows machine using a WMI-based command. Review the process creation events to find out which utility was invoked to perform this enumeration. | ✗ | ✓ | gpt-5-mini-low Wins |
T1057 | On a Windows host, investigate process events to find when Task Manager was launched via cmd with an unusual flag. What was the full command executed? | ✗ | ✓ | gpt-5-mini-low Wins |
T1059.004 | On a Linux system, find any process creation record where awk is used with a BEGIN rule to launch a shell. What was the exact command invoked? | ✗ | ✓ | gpt-5-mini-low Wins |
T1069.001 | Review recent Windows process event logs for PowerShell activity that suggests local group enumeration through WMI. What exact command was executed? | ✗ | ✓ | gpt-5-mini-low Wins |
T1059.004 | On a Linux host, identify the process invocation that altered a user’s login shell. What was the full command used? | ✗ | ✓ | gpt-5-mini-low Wins |
T1070.003 | On a Windows endpoint, review process execution logs to see if any PowerShell sessions were wiped clean. Which command was executed to clear the PowerShell history? | ✗ | ✓ | gpt-5-mini-low Wins |
Page 1 of 8
Explore individual model performance and detailed analysis