Repository navigation
fix(sfcompute): keep starting nodes listed as pending on SSH 404 - #163
Merged
Merged
Conversation
mmoulikk
force-pushed
the
BREV-13508/keep-starting-sfcompute-nodes-listed
branch
from
November 13, 2026 05:51
5247e92 to
39d0ee2
Compare
mmoulikk
marked this pull request as ready for review
November 13, 2026 12:43
drewmalin
approved these changes
Nov 13, 2026
drewmalin
deleted the
BREV-13508/keep-starting-sfcompute-nodes-listed
branch
November 13, 2026 21:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
While a new SF Compute node is starting, SF Compute already lists it as
running, butGET /integrations/brev/v1/instances/{id}/sshreturns404("SSH details not currently available") for a few seconds.ListInstancesleft the node out of the list, and dev-plane reads a missing node asterminated. The environment then moved toinstance_lostand could end up interminated: setup never ran, Delete was hidden in the console, and the VM kept running and billing (BREV-13508).Root cause
getSSHInforeturned every SSH lookup error to its caller.ListInstancesskipped any node whose lookup failed, andGetInstancereturned the error. A404right after creation means the SSH details aren't ready yet, not that the node is gone.Fix
getSSHInfo: a404within 45 minutes of creation (sshDetailsPendingWindow) means the node is still starting. The node stays listed aspending,GetInstanceno longer errors, and it logssfcv2: SSH details not available yet, reporting instance as pending.sfcInstanceToBrevInstance: arunningnode without an SSH hostname is reported aspending, since Brev can't reach it yet.pendingdoesn't change an environment's state in dev-plane, and time inpendingisn't billed.Test
instance_test.go, against a fake SF Compute API:TestIsSSHDetailsPending: a404inside the window counts as pending; a404after the window, a500, or a connection error doesn't.TestListInstancesKeepsNewInstanceWithoutSSHDetailsAsPending: a new node with an SSH404is listed aspending, a ready node isrunningwith its SSH address, and an old node with an SSH404is still left out.TestGetInstanceReportsNewInstanceWithoutSSHDetailsAsPending:GetInstancereturnspendinginstead of an error.Staging (dev-plane
v0-brev-13508-0, Nov 13)404and were kept aspendinginstead of dropped (logs).instance_lost.dau23x56n(logs) andcoyxnym6e(logs) hit the404, stayed inbuilding, and reachedrunningafter one build retry while SSH came up. The other 9 were deleted on purpose mid-build, as part of the test.