返回资讯中心
外部精选
软件工程
#开发者实时流

Fly.io Platform Wide Rolling Host Reboots for PwnEscape

I’m debating redeploying, but worried we’ll end up on a host not yet rebooted (and then be in the same boat).

Hacker News 最新nethunters2026/10/17 · 2026/10/176 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

I’m debating redeploying, but worried we’ll end up on a host not yet rebooted (and then be in the same boat).

If you’re seeing a lot longer outage than the promised 15–20 minutes, then it’s probably worth redeploying (in my opinion). Running servers don’t always reboot cleanly, and fixing the rare cases that didn’t come back to life on their own may not be the highest priority, given that the Fly.io Machines platform treats single-host outages as more or less normal.

(Indeed, data that exists only on a single volume can be lost permanently, which is why flyctl always nags people when they’re in that situation, etc.)

I am one of the many frustrated now. I wish the comms were more proactive now as we have been down for at least 75 minutes now.

Unfortunately I’m unable to destroy our volume and restore it from our litestream backup because the control plane is reporting that our volume is still attached to the instance, even though that instance has been destroyed.

❯ fly -c fly.prod.toml machines list

No machines are available on this app cosmic-production

❯ fly -c fly.prod.toml volumes destroy vol_rn80lpxyo8e23ker Warning! Every volume is pinned to a specific physical host. You should create two or more volumes per application. Deleting this volume will leave you with 0 volume(s) for this application, and it is not reversible. Learn more at https://fly.io/docs/volumes/overview/ ? Are you sure you want to destroy this volume? Yes Error: failed destroying volume: failed to destroy volume vol_rn80lpxyo8e23ker: failed_precondition: volume is currently bound to machine: 807de9ae545d28

Update! I was finally able to destroy the volume and redeploy a new volume and instance, so we are back up for the moment. No idea if Fly is still scheduling instances to non-rebooted hosts, but we’ll see…

Hm… Can you move to another region? fly deploy doesn’t try to reuse volumes that are in a different region, if I remember correctly.

(There’s also the --with-new-volumes flag on fly scale count, although I’m not sure how well that would work here…)

Just updated my message above. I was finally able to delete the volume. Switching region was my next option if that continued to fail though. Here’s to runbooks and litestream generally working as advertised. I’m still disappointed in the comms though. And if our uptime alarm goes off again, I’m going to be further disappointed. Our Fly bill isn’t huge but it ain’t zero either.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →