Independent benchmarks for production deployments. Throughput, latency, and capacity tested across hardware configurations.
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 4x RTX Pro 6000 Blackwell | 384GB | 638tok/s | 1 - 10 | 1K - 1M | 11 users | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 4x RTX Pro 6000 Blackwell | 384GB | 392tok/s | 1 - 10 | 1K - 1M | ~16 users | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 2x RTX Pro 6000 Blackwell | 192GB | 368tok/s | 1 - 8 | 1K - 1024K | 5 users | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 1x RTX Pro 6000 Blackwell | 196GB | 183tok/s | 1 - 4 | 1K - 256K | — | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 2x RTX Pro 6000 Blackwell | 196GB | 218tok/s | 1 - 5 | 1K - 256K | — | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 1x RTX Pro 6000 Blackwell | 96GB | 674tok/s | 1 - 10 | 1K - 256K | 13 users | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 1x RTX Pro 6000 Blackwell | 96GB | 129tok/s | 1 - 4 | 1K - 96K | 1 users | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 1x RTX Pro 6000 Blackwell | 96GB | 126tok/s | 1 - 4 | 1K - 128K | 2 users | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 1x RTX Pro 6000 Blackwell | 96GB | 262tok/s | 1 - 5 | 1K - 256K | 4 users | View |
| Hardware | VRAM | Peak Throughput | Concurrency Tested | Context Length Tested | Chatbot User Capacity (32K) | Full Report |
|---|---|---|---|---|---|---|
| 1x RTX Pro 6000 Blackwell | 96GB | 178tok/s | 1 - 5 | 1K - 512K | 6 users | View |
Each configuration runs 50+ test scenarios across different context lengths and concurrency levels up to what the model and hardware can support. This includes measuring standard metrics such as throughput and latency, as well as conducting capacity tests that increase concurrent requests until performance thresholds are exceeded.
Prompts use calibrated token counts. No prompt caching or speculative decoding unless specifically noted.