Benchmarks
Embedding benchmarks. Cost per passage.
BGE small, base and large on four rented GPUs at 128 and 512 tokens: passages per second and what a million passages cost, for a RAG pipeline you run yourself.
Measured
$0.032per 1M passages
L40S, bge-small, 128-token passages, at the run-day price.
Run 2026-10-29
BGE embedding throughput per dollar. Three model sizes, two lengths, four GPUs.
At run-day prices the L40S embedded a million passages for less than the other three cards in all six cases, from $0.032 per million 128-token passages with bge-small to $0.492 per million 512-token passages with bge-large. With bge-large at 512 tokens it ran 598 passages per second on the H100, 411 on the RTX 5090, 305 on the L40S and 234 on the RTX A6000.
Hardware
- RTX 5090 32 GB (16 vCPUs), L40S 48 GB (12 vCPUs), RTX A6000 48 GB (28 vCPUs), H100 PCIe 80 GB (20 vCPUs), one per machine
Software
- BAAI/bge-small-en-v1.5 (33M parameters), bge-base-en-v1.5 (109M), bge-large-en-v1.5 (335M), fp16
- sentence-transformers 5.7.0, transformers 4.57.1, Python 3.12; no ONNX, TensorRT or text-embeddings-inference
Method
- 4,096 passages per run of random vocabulary words, truncated to exactly 128 or 512 tokens
- Passages per second over at least 20 s of repeated model.encode(texts, batch_size=128) after a warm-up, tokenization included
| Model | GPU | $/hour, run day | Passages/s | Tokens/s | $ per 1M, run day | $ per 1M, today |
|---|---|---|---|---|---|---|
| bge-small-en-v1.5 | RTX 5090 | 1.15 | 6,415 | 821,094 | $0.050 | $0.037 |
| bge-small-en-v1.5 | L40S | 0.54 | 4,714 | 603,443 | $0.032 | $0.032 |
| bge-small-en-v1.5 | RTX A6000 | 0.65 | 3,681 | 471,130 | $0.049 | $0.043 |
| bge-small-en-v1.5 | H100 | 3.86 | 4,952 | 633,882 | $0.217 | $0.218 |
| bge-base-en-v1.5 | RTX 5090 | 1.15 | 4,077 | 521,843 | $0.078 | $0.057 |
| bge-base-en-v1.5 | L40S | 0.54 | 3,054 | 390,925 | $0.049 | $0.049 |
| bge-base-en-v1.5 | RTX A6000 | 0.65 | 2,273 | 290,931 | $0.079 | $0.069 |
| bge-base-en-v1.5 | H100 | 3.86 | 4,123 | 527,706 | $0.260 | $0.261 |
| bge-large-en-v1.5 | RTX 5090 | 1.15 | 1,748 | 223,731 | $0.183 | $0.134 |
| bge-large-en-v1.5 | L40S | 0.54 | 1,374 | 175,936 | $0.109 | $0.109 |
| bge-large-en-v1.5 | RTX A6000 | 0.65 | 875 | 111,987 | $0.206 | $0.181 |
| bge-large-en-v1.5 | H100 | 3.86 | 2,391 | 306,061 | $0.448 | $0.449 |
| Model | GPU | $/hour, run day | Passages/s | Tokens/s | $ per 1M, run day | $ per 1M, today |
|---|---|---|---|---|---|---|
| bge-small-en-v1.5 | RTX 5090 | 1.15 | 1,713 | 877,005 | $0.186 | $0.136 |
| bge-small-en-v1.5 | L40S | 0.54 | 1,193 | 610,867 | $0.126 | $0.126 |
| bge-small-en-v1.5 | RTX A6000 | 0.65 | 1,068 | 546,867 | $0.169 | $0.148 |
| bge-small-en-v1.5 | H100 | 3.86 | 1,428 | 731,085 | $0.751 | $0.753 |
| bge-base-en-v1.5 | RTX 5090 | 1.15 | 1,033 | 528,998 | $0.309 | $0.226 |
| bge-base-en-v1.5 | L40S | 0.54 | 763 | 390,605 | $0.197 | $0.197 |
| bge-base-en-v1.5 | RTX A6000 | 0.65 | 603 | 308,838 | $0.299 | $0.262 |
| bge-base-en-v1.5 | H100 | 3.86 | 1,156 | 591,974 | $0.927 | $0.929 |
| bge-large-en-v1.5 | RTX 5090 | 1.15 | 411 | 210,330 | $0.778 | $0.568 |
| bge-large-en-v1.5 | L40S | 0.54 | 305 | 156,109 | $0.492 | $0.492 |
| bge-large-en-v1.5 | RTX A6000 | 0.65 | 234 | 120,064 | $0.770 | $0.675 |
| bge-large-en-v1.5 | H100 | 3.86 | 598 | 306,330 | $1.792 | $1.797 |
What stood out
- For 10 million 512-token chunks with bge-base that is about $2.0 on the L40S, $3.0 on the RTX A6000, $3.1 on the RTX 5090 and $9.3 on the H100, at run-day prices.
- bge-small ran at 4,952 passages per second on the H100 and 6,415 on the RTX 5090: small models wait on tokenization and Python between batches, so the CPU side sets the speed.
Keep in mind
- The machines differ in CPU count (12 to 28 vCPUs) and CPU model, which matters most for the small models.
- Plain sentence-transformers in fp16, no dedicated embedding server; a dedicated server would raise the small-model numbers.
- Real corpora have mixed lengths, so short real-world text runs faster than these fixed-length 512-token numbers.
Costs
How the costs are worked out. Run day and today, side by side.
Run day
The cost the write-up printed: the hourly price the machine was launched at that day, against what it measured. Prices move daily, so each run-day figure carries its date. Every run was a single run.
Today
The run-day figure moved by today's price: the "from" price per GPU-hour shown on /pricing, over the run-day rate per GPU-hour. A GPU that is not in stock right now reads "Not in stock" rather than a stale price. Prices read 2026-11-04 04:00 UTC.
GPUs
The GPUs on this page. From price per GPU-hour, today.
Benchmarks