File tree Expand file tree Collapse file tree 1 file changed +19
-2
lines changed Expand file tree Collapse file tree 1 file changed +19
-2
lines changed Original file line number Diff line number Diff line change 22
33<center >
44
5- Under construction
5+ # SciCode Leaderboard
6+
7+ | Model | Main Problem Resolve Rate |
8+ | ------------------------| ---------------------------|
9+ | OpenAI o1-preview | 7.7% |
10+ | Claude3.5-Sonnet | 4.6% |
11+ | Deepseek-Coder-v2 | 3.1% |
12+ | GPT-4o | 1.5% |
13+ | GPT-4-Turbo | 1.5% |
14+ | OpenAI o1-mini | 1.5% |
15+ | Gemini 1.5 Pro | 1.5% |
16+ | Claude3-Opus | 1.5% |
17+ | Claude3-Sonnet | 1.5% |
18+ | Qwen2-72B-Instruct | 1.5% |
19+ | Llama-3.1-405B-Instruct| 0% |
20+ | Llama-3.1-70B-Instruct | 0% |
21+ | Mixtral-8x22B-Instruct | 0% |
22+ | Llama-3-70B-Chat | 0% |
623
724<!-- Once you've added the results to the submission repository,
825 bring back the table here -->
@@ -11,4 +28,4 @@ Under construction
1128</center >
1229
1330!!! tip "How to submit"
14- Want to submit your own model? Head over to the [ documentation] ( docs/index.md ) .
31+ Want to submit your own model? Head over to the [ documentation] ( docs/index.md ) .
You can’t perform that action at this time.
0 commit comments