返回资讯中心
外部精选
软件工程
#开发者实时流

We kept renting GPUs after getting 7 quotes to buy them

We came close to buying our own GPU servers.

Hacker News 最新pencildiver2026/10/19 · 27 天前30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

We came close to buying our own GPU servers.

We had a spreadsheet that said it would save us money. We had quotes from seven vendors in Madrid: two hardware resellers, four colocation providers and a network carrier. We had a rack picked out. Then we finished the model, and it said that buying two servers would take until year nine to beat renting them.

This post covers how we got to that answer. It explains why we looked at buying in the first place, how we went to market, what each proposal said, how the model works, and what we would tell another team facing the same decision. Every number in it comes from a real quote or from our internal cost model. Where we have made an assumption, we say so.

We pay $7,200 a month to rent two GPU servers. That works out to about $2.47 per GPU-hour, all-in, which is already near the bottom of the market.

Buying the cheapest equivalent would cost $354,100 upfront. That includes 21% Spanish VAT (IVA), which we cannot recover. Running the servers in a Madrid data center would cost about $39,850 a year.

The savings are real but small. About $46,550 a year, against $375,700 of cash spent before the first saving arrives.

Payback takes just over eight years. The hardware has a three-year warranty and a standard five-year useful life. At year five we would still be $143,000 behind.

No single change fixes it. A cheaper data center, a bigger fleet, or recovering the IVA all help. None of them brings payback inside five years.

The biggest variable is what you pay to rent today. At the 2026 on-demand median for an H100, the same purchase pays back in under five years. At our rate it does not.

Agora runs most of its AI on its own models. Query planning, reranking and product embeddings are served from GPUs we control, and the search models are retrained every day on the previous day's logs. We wrote about that system in "Building a reinforcement learning system for e-commerce search". Self-hosting is what makes that loop possible. It also makes GPUs one of our largest infrastructure costs.

Today that work runs on two rented servers, each with two NVIDIA H100 PCIe GPUs. The bill is $7,200 a month, or $86,400 a year. It covers the GPUs, the rest of the server, hosting, power and hardware maintenance. Divided across four GPUs running every hour of the month, that is about $2.47 per GPU-hour.

Rental prices were rising. For most of 2024 and 2025, renting a GPU got cheaper every quarter. That stopped. SemiAnalysis reported that one-year H100 rental contracts rose almost 40%, from $1.70 per GPU-hour in October 2025 to $2.35 by March 2026. On-demand prices followed. Nebius listed its H100 at $2.95 an hour in May, according to Yahoo Finance, and its pricing page shows $4.50 from October 1, 2026. The median on-demand H100 across 40 providers was $3.42 an hour at the end of September, 15% more than a year earlier, according to GetDeploying data cited by DataStorage.com.

Our usage was going to grow. When we asked for quotes, we told every data center the same thing: we run two GPU servers today, about 3 to 5 kW, and expect to grow to several racks in the next 18 to 24 months. If we were going to own hardware, it made sense to decide before that growth, not after.

The conventional wisdom says owning is cheaper. It often is. 37signals moved its applications off the public cloud onto about $700,000 of Dell servers and recouped the whole purchase within a year, according to ITPro. If it worked for them, we wanted to know whether it would work for us.

We started by writing down what we needed, in one paragraph, and sent the same paragraph to every vendor:

Start with half a rack to a full rack in Madrid.

Two GPU servers today, about 3 to 5 kW, growing to several racks over 18 to 24 months.

Power density per rack matters more than floor space.

We own and operate the hardware. We need space, power, cooling and connectivity.

Madrid was an easy choice. It is where our team is, it has a dense cluster of carrier-neutral data centers, and it keeps the hardware close enough that someone can drive over when something breaks.

We split the problem into three markets, because they are bought separately and priced differently:

Hardware. The servers and GPUs, bought outright.

Colocation. A rack, power, cooling and physical security in someone else's building.

Connectivity. Internet transit and cross-connects into that building.

We built the cost model first, in June, using estimated prices. Then, as each quote came in, we replaced an estimate with a real number. That order turned out to matter. The model told us which inputs were worth chasing, and the quotes showed us how wrong our estimates had been.

Over about four months we collected seven proposals:

Colocation, MAD-1, 5 kW cabinet with internet

Colocation, Madrid campus, 2 kW and 5 kW racks

Dedicated internet access at Iron Mountain and Digital Realty

Colocation, MD6, 5 kVA secure cabinet with internet and cloud fabric

Colocation, 3, 5 and 7 kW racks, plus transit

Two Dell GPU servers, with managed colocation at Digital Realty

All four colocation providers quoted a single rack with 5 kW of power. Put side by side, the monthly prices are closer than we expected.

All prices exclude IVA. A few things stood out.

Power is most of the bill. In every quote, power and space is more than half of the monthly cost, and up to two thirds. Electricity is bundled into the per-kW price, so the number you choose for power is the number that sets your bill. Data4 put it plainly: the monthly fee follows the kW you contract, not the number of racks.

Terms matter more than the headline price. The spread between the cheapest and most expensive monthly total is about €760. The difference between a 12-month and a 36-month minimum term, or between flat pricing and 5% a year compounded, matters more over the life of the contract. Iron Mountain told us it had seen escalation as low as 3% for larger footprints, and offered a 3-6-9-11 month ramp so we would not pay for full power from day one. Neither was in the written quote.

GPU density forces decisions early. Data4 offers single-phase racks up to 7 kW. Above that, the rack moves to three-phase power, and the €4,850 setup fee is charged again because the rack's electrical installation is replaced. Alhambra's proposal recommended a 7 kW rack for its two Dell servers. If your GPUs will outgrow single-phase within the contract, it is cheaper to plan for that on day one.

Every quote had gaps. Data4 did not price cross-connects or remote hands. Digital Realty's internet came from Colt, and its quote priced six cross-connects when we asked for two. Iron Mountain's quote said 36 months while its sales team told us 24 was the minimum. None of these were large, but each one meant another email before we could treat a number as final.

On connectivity, Colt quoted 1 Gbps of dedicated internet access on 36-month terms, with 99.9% availability and a four-hour time to repair: $663 a month at Iron Mountain and $760 at Digital Realty plus a cross-connect. Usage-based options were slightly more. Iron Mountain and Equinix bundled their own internet into the colocation quote.

For the model, we used Digital Realty as the base case. Data4 was cheaper on paper, but its quote was missing items we would need. Digital Realty was close behind, had the shortest minimum term, and did not quote an escalator.

We asked both resellers for the same thing: a like-for-like replacement for what we rent, with two H100 PCIe GPUs per server and room to add two more. Neither quoted it.

SIXE proposed two Supermicro SYS-221GE-NR servers, each with two NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs (96 GB each), two 60-core Xeon Platinum 8580 processors, 1 TB of DDR5 and two free GPU slots. Each server costs €129,500 before IVA, with a three-year next-business-day warranty. They substituted the GPU because H100 PCIe cards were not in stock.

Alhambra IT proposed two Dell PowerEdge XE7740 servers, each with two NVIDIA H200 NVL GPUs (141 GB each), two 64-core Xeon 6 processors and 1 TB of DDR5. Each server costs €194,932 with a three-year warranty, or €209,321 with five. Alhambra would also manage the colocation at Digital Realty's Madrid campus, quoting €2,145 a month for a 5 kW rack and €3,088 for 7 kW.

Neither option is a straight swap for what we run, and the difference is easy to miss if you only compare memory.

The RTX PRO 6000 has more memory than an H100 PCIe, 96 GB against 80. But it uses GDDR7 rather than HBM, and its bandwidth is about 1.6 TB/s, according to PNY's datasheet, against 2.0 TB/s for the H100 PCIe. When a language model generates text, each new token requires reading the model's weights from memory, so bandwidth often sets the speed. For some of our workloads, the cheaper card could be slower than the one we have.

The H200 NVL is a large step up: 141 GB of HBM3e and 4.8 TB/s, according to the Alhambra proposal. It also costs 50% more per server. We would be paying for capacity we do not need yet.

The terms were as important as the specs:

SIXE wanted 100% of the payment at order. No staged or net terms. They said leasing might be possible.

The quote was valid for 12 days, from September 18 to September 30.

Lead times ranged from two to three weeks for configurations in stock to five or six months for others.

The price excluded transceivers, DAC cables, switches, rack installation, the operating system and software licences.

Our June model assumed about €75,241 per server, based on public H100 server pricing at the time. SIXE's quote came in at €129,500, 72% higher.

Some of that gap is our own doing. The quote has more memory than we specified (1 TB against 768 GB) and a different GPU. But much of it is the market. Memory makers have shifted capacity toward high-bandwidth memory for AI accelerators, and conventional server memory has been short all year. TrendForce expects conventional DRAM contract prices to rise another 10 to 15% in the fourth quarter of 2026, and Micron expects the shortage to get worse in 2027 and 2028, according to CIO.

This matters for anyone doing the same exercise. Rising rental prices are a reason to look at buying. But the same shortage that pushes rents up also pushes server prices up. The gap you are hoping to capture is smaller than either number suggests on its own.

The model is a plain spreadsheet with five tabs. We kept it simple on purpose, so anyone at the company could change an input and see the result.

Assumptions. Every input in one place: number of servers, GPUs per server, server price, rack costs, colocation, tax, our current rental bill, and the migration overlap. Inputs are blue. A toggle picks which hardware quote and which colocation quote feed the rest.

Quotes. Each hardware and colocation quote as received, with notes on what is missing.

Hardware. The one-off purchase, including installation and IVA.

Running costs. Colocation, cross-connects, bandwidth, remote hands and support, per month and per year.

Buy versus rent. Nine years of cumulative cost for each path, and the difference.

Euros are converted at 1.12 dollars. The base case uses SIXE hardware and Digital Realty colocation with Colt internet.

IVA is a real cost for us. Search Agora, Inc. is a Delaware corporation with no Spanish VAT registration. Non-EU businesses can sometimes reclaim Spanish VAT, but Spain requires reciprocity, and the European Commission's summary of Spain's rules lists reciprocity with Canada, Monaco, Japan, Norway, Switzerland and Israel only. In April 2026 the Spanish tax authority confirmed that US businesses can claim refunds in a narrow set of cases where reciprocity is not required, according to DLA Piper. Buying servers for our own use is not one of them. So the $61,455 is money we would not get back. This is our reading, not tax advice; check your own position.

Migration is not free. Moving production inference and daily tra

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →