A practical rental guide
How to choose a GPU
Start with the workload and the memory it needs. Then compare the cost of finishing the job, not just the price of one GPU hour.
1. Decide whether you need a GPU at all
For hosted text generation, a model API avoids managing a container, downloading weights, and paying for idle GPU time. Free Qwen inference is a useful starting point when its model and API fit your application.
Rent a GPU when you need to run a particular model, train or fine-tune, change the runtime, process data locally in the workspace, or execute a custom CUDA workload. A rental gives you a container with the resources listed in the offer.
2. Budget memory before comparing speed
Model weights are only part of the memory requirement. Inference also needs runtime buffers and a KV cache; longer contexts and more simultaneous requests consume more memory. Training adds activations, gradients, and optimizer state.
| Precision | Bytes per weight | Weights alone |
|---|---|---|
| 16-bit | 2 | 54 GB |
| 8-bit | 1 | 27 GB |
| 4-bit | 0.5 | 13.5 GB |
These are not minimum GPU requirements. Quantization metadata and other tensors add overhead, and your runtime must support the chosen format. Check the model's deployment instructions and the listing's usable VRAM, not only the GPU's advertised capacity.
For practical memory tuning, see the vLLM optimization guide. Benchmark your own context length and batch size.
3. Choose a useful starting point
| GPU or family | What to evaluate | Watch out for |
|---|---|---|
| RTX 4090 | Smaller models, quantized inference, image workloads, and development within its memory budget. | Memory headroom can be the limiting factor before compute speed. |
| RTX PRO 6000 | Larger single-GPU workloads and more room for model weights and context. | The Blackwell Max-Q variant is different from older RTX 6000 generations and other power variants. |
| H200 | Memory-intensive inference and workloads that benefit from HBM bandwidth. | Check the exact form factor, interconnect, GPU count, and current availability. |
| Blackwell | A GPU architecture family, not a single interchangeable rental product. | Compare exact models and runtime support; architecture alone does not predict job speed. |
Do not assume that several GPUs' memory acts like one large GPU. Your software must explicitly support distributing the model or job, and communication overhead can dominate. The current nafnaf raw-rental flow reserves one GPU per rental.
4. Check the whole machine and the whole bill
- Container RAM and CPU: host totals are not your container allowance. Model loading and preprocessing may need substantial system memory.
- Storage: budget for model weights, datasets, checkpoints, and temporary files. Approved storage can remain billable while a workspace is paused.
- Network: measured upload and download rates are observations, not guaranteed throughput. Large downloads can cost time and metered transfer fees.
- Uptime: compare the observed period as well as the percentage. A short observation window is not proof of long-term reliability.
- Runtime: confirm driver, CUDA, framework, and workload-template compatibility before booking.
Estimate GPU rate × runtime + storage + metered transfer. A faster GPU can cost less per completed job even at a higher hourly rate. Run a small representative benchmark before committing to a long job, and export important results before deleting the workspace.