Freedom251 › SWE-bench

Instead of SWE-bench

SWE-bench alternatives

Apps that do what SWE-bench does, from the llm evaluation tools list, the ones you can start free with first.

1. 1 A

   AgentBench

   Free plan APILinuxSelf-hosted
   7.3 Free plan Visit site
2. 2

   Galileo

   Free plan APISelf-hostedWeb
   7.3 Free plan · paid from $100/mo (annual) Visit site
3. 3 A

   ARES

   Free plan APILinuxSelf-hosted
   7.2 Open source Visit site
4. 4

   HELM

   Free plan APISelf-hostedWeb
   7.2 Open source Visit site
5. 5 E

   EvalPlus

   Free plan APILinuxSelf-hosted
   7.1 Open source Visit site
6. 6 L

   LM Evaluation Harness

   Free plan APILinuxSelf-hosted
   7.1 Open source Visit site
7. 7

   Inspect AI

   Free plan APIExtension
   7.0 Open source Visit site
8. 8

   Maxim AI

   Free plan APISelf-hostedWeb
   7.0 Free plan · paid from $29/user/mo · 14-day trial Visit site
9. 9 O

   OpenCompass

   Free plan APILinuxSelf-hosted
   7.0 Free plan Visit site
10. 10 P

    Parea AI

    Free plan APISelf-hostedWeb
    7.0 Free plan Visit site
11. 11 P

    PyRIT

    Free plan APILinuxSelf-hosted+1
    7.0 Open source Visit site
12. 12 H

    HarmBench

    Free plan APILinuxSelf-hosted
    6.9 Free plan Visit site
13. 13

    Ragas

    Free plan APILinuxSelf-hosted
    6.6 Free plan Visit site
14. 14 R

    RAGChecker

    Free plan Self-hosted
    6.5 Open source Visit site
15. 15 O

    OpenAI Evals

    Free plan Web
    5.6 Free plan Visit site
16. 16

    DecodingTrust

    Paid APISelf-hosted
    5.4 Open source Visit site
17. 17 L

    LiveBench

    Paid LinuxMacSelf-hosted+2
    5.3 Open source Visit site
18. 18 W

    WebArena

    Paid Self-hosted
    4.6 Open source Visit site
19. 19 K

    kube-bench

    Paid Linux
    4.2 Open source Visit site
20. 20 L

    Lighteval

    Paid
    4.0 Open source Visit site

← Back to SWE-bench The whole llm evaluation tools list