Skip to content

Commit 3e8b3a4

Browse files
committed
Add example project to benchmark download speeds
1 parent 805b037 commit 3e8b3a4

File tree

5 files changed

+447
-1
lines changed

5 files changed

+447
-1
lines changed

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
.DS_Store
2-
/.build
2+
.build
33
/Packages
44
xcuserdata/
55
DerivedData/

Example/Package.resolved

Lines changed: 33 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Example/Package.swift

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
// swift-tools-version: 6.0
2+
import PackageDescription
3+
4+
let package = Package(
5+
name: "download-speed-test",
6+
platforms: [
7+
.macOS(.v14),
8+
.iOS(.v16),
9+
],
10+
products: [
11+
.executable(
12+
name: "download-speed-test",
13+
targets: ["DownloadSpeedTest"]
14+
)
15+
],
16+
dependencies: [
17+
.package(path: "../"),
18+
.package(url: "https://github.com/apple/swift-argument-parser", from: "1.3.0"),
19+
],
20+
targets: [
21+
.executableTarget(
22+
name: "DownloadSpeedTest",
23+
dependencies: [
24+
.product(name: "HuggingFace", package: "swift-huggingface"),
25+
.product(name: "ArgumentParser", package: "swift-argument-parser"),
26+
]
27+
)
28+
]
29+
)

Example/README.md

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
# Download Speed Test Example
2+
3+
This example demonstrates how to use the HuggingFace Swift package to download files from a repository and measure download performance. It's designed to compare download speeds with and without Xet support.
4+
5+
## Usage
6+
7+
### Running the Test
8+
9+
From the `Example` directory:
10+
11+
```bash
12+
swift run download-speed-test
13+
```
14+
15+
Use `--help` to see all arguments:
16+
17+
```bash
18+
swift run download-speed-test --help
19+
```
20+
21+
### Command Line Options
22+
23+
- `--repo <owner/name>` or `-r <owner/name>`: Repository to benchmark (default: Qwen/Qwen3-0.6B)
24+
- `--file <path>` or `-f <path>`: Download a specific file (e.g., `model.safetensors`)
25+
- `--min-size-mb <MB>`: Minimum file size in MB to test (default: 10, filters out small files)
26+
- `--xet` / `--no-xet`: Enable or disable Xet acceleration
27+
28+
### Testing with Xet Enabled/Disabled
29+
30+
To compare performance with and without Xet:
31+
32+
**Basic comparison (auto-selects large files):**
33+
```bash
34+
# With Xet
35+
swift run download-speed-test
36+
37+
# Without Xet (LFS)
38+
swift run download-speed-test --no-xet
39+
```
40+
41+
**Test a specific large file:**
42+
```bash
43+
# Download specific model file
44+
swift run download-speed-test --file model.safetensors
45+
46+
# Compare Xet vs LFS for the same file
47+
swift run download-speed-test --file model.safetensors --no-xet
48+
```
49+
50+
**Test different repository:**
51+
```bash
52+
swift run download-speed-test --repo meta-llama/Llama-3.2-1B
53+
```
54+
55+
**Adjust minimum file size filter:**
56+
```bash
57+
# Only test files >= 100 MB (better for Xet benchmarking)
58+
swift run download-speed-test --min-size-mb 100
59+
60+
# Include smaller files (>= 1 MB)
61+
swift run download-speed-test --min-size-mb 1
62+
```
63+
64+
**Notes:**
65+
- Xet excels at large files (10+ MB), so the benchmark filters out small files by default
66+
- Small files (configs, JSONs) add overhead that doesn't showcase Xet's strengths
67+
- Use `--file` to benchmark a specific large model file for accurate comparison
68+
69+
### Performance Features
70+
71+
Xet is optimized for high-performance downloads by default:
72+
73+
- **256 concurrent range GET requests** per file (automatically set)
74+
- **High-performance mode enabled** for maximum throughput
75+
- **XetClient reuse** across downloads for HTTP/TLS connection pooling
76+
- **JWT token caching** per repository/revision to avoid redundant API calls
77+
78+
**No configuration required!** Xet should match or exceed LFS speeds out of the box.
79+
80+
If you need to adjust settings:
81+
82+
- **XET_NUM_CONCURRENT_RANGE_GETS**: Override default per-file concurrency
83+
```bash
84+
XET_NUM_CONCURRENT_RANGE_GETS=128 swift run download-speed-test # Lower for slow networks
85+
```
86+
87+
- **XET_HIGH_PERFORMANCE**: Disable high-performance mode
88+
```bash
89+
XET_HIGH_PERFORMANCE=0 swift run download-speed-test # Conservative mode
90+
```
91+
92+
## What It Does
93+
94+
The test:
95+
1. Connects to the Hugging Face Hub
96+
2. Lists files in the specified repository (default: `Qwen/Qwen3-0.6B`)
97+
3. Selects large files (default: >= 10 MB) that showcase Xet's performance:
98+
- Model files (`.safetensors`, `.bin`, `.gguf`, `.pt`, `.pth`)
99+
- Prioritizes the largest files for meaningful benchmarking
100+
4. Downloads each file and measures:
101+
- Download time
102+
- File size
103+
- Download speed (MB/s)
104+
5. Provides a summary with total time, size, and average speed
105+
106+
**Why filter small files?**
107+
Xet is optimized for large files through:
108+
- Content-addressable storage with chunking
109+
- Parallel chunk downloads
110+
- Deduplication across files
111+
112+
Small files (<10 MB) don't benefit from these optimizations and add per-file overhead that skews results.
113+
114+
## Output Example
115+
116+
```
117+
🚀 Hugging Face Download Speed Test
118+
Repository: Qwen/Qwen3-0.6B
119+
============================================================
120+
121+
✅ Xet support: ENABLED
122+
123+
📋 Listing files in repository...
124+
📦 Selected 3 files for testing:
125+
• model.safetensors (1.2 GB)
126+
• model-00001-of-00002.safetensors (987 MB)
127+
• model-00002-of-00002.safetensors (256 MB)
128+
129+
⬇️ Starting download tests...
130+
131+
✅ [1/3] model.safetensors
132+
Time: 12.34s
133+
Size: 1.2 GB
134+
Speed: 99.2 MB/s
135+
136+
✅ [2/3] model-00001-of-00002.safetensors
137+
Time: 9.87s
138+
Size: 987 MB
139+
Speed: 100.1 MB/s
140+
141+
✅ [3/3] model-00002-of-00002.safetensors
142+
Time: 2.56s
143+
Size: 256 MB
144+
Speed: 100.0 MB/s
145+
146+
============================================================
147+
📊 Summary
148+
============================================================
149+
Total files: 3
150+
Total time: 24.77s
151+
Total size: 2.4 GB
152+
Average speed: 99.8 MB/s
153+
154+
💡 Tip: toggle Xet via --xet / --no-xet to compare backends.
155+
```
156+
157+
## Notes
158+
159+
- The test uses a temporary directory that is automatically cleaned up
160+
- Files are downloaded sequentially to get accurate timing
161+
- The test automatically selects a mix of small and large files
162+
- Progress is shown for each file download
163+

0 commit comments

Comments
 (0)