Cost of Renting an NVIDIA?H100 on AWS or Google Cloud

As demand for high-performance AI infrastructure continues its rapid climb, the **NVIDIA?H100-**powered by the Hopper architecture-has become a gold standard for training large language models (LLMs), deep learning systems, and HPC workloads. But what does it cost to rent an H100 in top public clouds like AWS and Google Cloud? This knowledgebase will walk you through the pricing, instance options, cost breakdowns, and practical guidance for Go4hosting clients considering H100 rentals.

1. Why NVIDIA?H100?

NVIDIA's H100 represents the pinnacle of Hopper?based AI tech-with enormous tensor core throughput, massive HBM3 memory, and blazing interconnect speeds. Organizations using it see model training 3-4* faster than last-gen systems like the A100.

2. AWS: Renting H100 via EC2 P5 Instances

On AWS, H100 GPUs are available in the P5 series of EC2 instances:

• p5.48xlarge: Contains 8 H100 GPUs, 192 vCPUs, 2?TB RAM, and 30?TB NVMe.
• On-demand pricing in regions like Mumbai and N. Virginia is roughly $3.93 per GPU?hour-totaling about $31.46/hr per instance
• That equates to $5.56/hr per GPU on an 8?GPU instance

• Spot pricing offers significant savings-often 50-75% off on-demand; Vantage stats show average spot at $25.17/hr ($3.15 per GPU).
• Reserved instances/Savings Plans (1?3 year terms) can reduce costs dramatically-monthly per?GPU goes down to $1.89-$2.05/hr equivalent.
• June 2025 price cuts on GPU instances (P4/P5/P5en families) may further lower rates.

Cost Summary (On-Demand):

Cost With Spot & Savings:

• Spot: as low as $20/hr per instance ($2.50 per GPU)
• With 3?year reserved/Savings Plan: $1.90-$2.10/hr per GPU

3. Google Cloud: A3 Instances with H100 GPUs

Google offers H100 GPUs within the A3 family (like A3?MEGA / A3?HIGH):

• Spot (preemptible) pricing:
• A3?HIGH (H100): $2.253/hr per GPU 
• A3?MEGA (H100): $2.379/hr per GPU
• On-demand (on-demand VMs) are around $3.00/hr per GPU for PCIe H100 and $6.00/hr for dual?GPU setups.
• No long-term reserved options-sustained usage discounts apply, but not as deep as AWS reserved models.

GCP Cost Summary:

4. AWS vs GCP: Price Comparison

• On-demand: AWS charges $3.93/hr per GPU; GCP is $3.00/hr.
• Spot/preemptible: AWS $2.50/hr; GCP $2.25/hr.
• Long-term savings: AWS offers deeper cuts with 1-3 year commitments ($1.90-$2.10/hr). Google Cloud sustained use gives modest yet noticeable discounts.

5. Total Cost of Ownership (TCO): Rent vs Own

• Cloud rental: Spot rates around $2.25/hr/GPU offer flexibility without upfront investment.
• Purchase: H100 cards alone cost $25k-30k; full cloud server cost $40k+, plus colocation, power, network, maintenance.
• If running 24/7, buying may become economical beyond 2-3 years, but cloud rental remains ideal for scaling, experimentation, or on-demand bursts.

6. Use-Cases and Budgeting Guidance

a) Short-Term Training or Testing

• Use preemptible/spot instances for testing proofs-of-concept or model validation.
• GCP gives best price-per-hour; AWS offers larger-scale flexibility (8 GPUs).

b) Medium-Term or Burst Projects

• Combine on-demand and spot windows.
• AWS server Savings Plans can cut continuous use to $1.90-$2.10/hr.
• GCP sustained usage can further reduce on-demand over prolonged use.

c) 24/7 Training at Scale

• AWS reserved instances are most cost-effective.
• Buying hardware becomes viable only after sustained multi-year use.

7. Other Options Beyond AWS & GCP

• Specialty cloud providers like Lambda Labs offer H100 for $1.99/hr, and CoreWeave around $2.23/hr, but often require pre-approval.
• These vendors can offer 10-30% lower costs compared to hyperscalers-great for heavy AI workloads.

8. Tips for Cost Optimization

1. Spot/preemptible rentals: Opt for spot to keep hourly below $2.5.
2. Savings/reservation: Commit 1-3 years on AWS if you need continuous access.
3. Region selection: Prices vary-Mumbai, Ohio, Oregon, Tokyo often cheaper.
4. Multi-cloud strategy: Combine AWS and GCP based on spot availability and pricing.
5. Evaluate data transfer/storage costs: GPU rental is only part of your TCO.
6. Consider alternate vendors: Pre-approved rentals may offer 30-40% savings.

9. Conclusion for Go4hosting Clients

• On-demand H100 rental: AWS $3.90/hr; GCP $3.00/hr.
• Spot/preemptible rates: AWS $2.50/hr; GCP $2.25/hr.
• AWS long-term savings: $1.90-$2.10/hr per GPU with 1-3-year commitments.
• Specialty providers: $1.99/hr for H100-if scale and access allow.

At Go4hosting, we help our clients choose the right GPU rental strategy-whether staying with public clouds or leveraging colocation or hybrid setups. Our experts can guide you through:

• instance type selection (e.g., p5 vs a3)
• balancing spot vs on-demand use
• cost modelling and TCO analysis
• architecture design for multi-cloud compute bursts

10. Should You Rent or Buy?

• Rent if you need flexibility, burst capacity, or testing.
• Buy only if you plan full-time, long-haul usage for years under your control.

Bottom line for clients: With public cloud H100 rates between $2.25-$3.90/hr, purchasing remains ideal only at significantly higher utilization and workload duration. Go4hosting can assist in building the optimal plan-blending public cloud rental, server colocation, and hybrid GPU infrastructure-to hit your performance needs and budget.

Was this answer helpful? 0 0