Take advantage of suspending breakpoint provided by delve

[Lslightly]

What version of Go, VS Code & VS Code Go extension are you using?

Version Information

• Run go version to get version of Go from the VS Code integrated terminal.
  • go version go1.24.0 linux/amd64
• Run gopls -v version to get version of Gopls from the VS Code integrated terminal.
  • golang.org/x/tools/gopls v0.18.1
• Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.
  • 1.98.0 6609ac3d66f4eade5cf376d1cb76f13985724bcb x64
• Check your installed extensions to get the version of the VS Code Go extension
  • 0.46.1
• Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.
  - the dlv is the version not released yet in https://github.com/Lslightly/delve/tree/followExecByDefault which enable target follow-exec -on by default and fixed some other bugs(see service/debugger: fix FindLocation with child processes by aarzilli · Pull Request #3937 · go-delve/delve).

Environment

GOBIN: undefined
toolsGopath:
gopath: /home/lqw/go
GOROOT: /home/lqw/go1.24.0
PATH: /home/lqw/.vscode-server/bin/6609ac3d66f4eade5cf376d1cb76f13985724bcb/bin/remote-cli:/home/lqw/.rvm/gems/ruby-3.0.0/bin:/home/lqw/.rvm/gems/ruby-3.0.0@global/bin:/home/lqw/.rvm/rubies/ruby-3.0.0/bin:/home/lqw/mygit/delve:/home/lqw/go/bin:/home/lqw/go1.24.0/bin:/home/lqw/scripts:/home/lqw/cmake/bin:/home/lqw/.local/bin:/home/lqw/miniconda3/bin:/home/lqw/miniconda3/condabin:/home/lqw/.opam/default/bin:/home/lqw/.elan/bin:/home/lqw/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/lib/wsl/lib:/home/lqw/.rvm/bin

Tools

go:	/home/lqw/go1.24.0/bin/go: go version go1.24.0 linux/amd64

gopls:	/home/lqw/go/bin/gopls	(version: v0.18.1 built with go: go1.24.1)
gotests:	not installed
gomodifytags:	not installed
impl:	not installed
goplay:	not installed
dlv:	/home/lqw/go/bin/dlv	(version: v0.0.0-20250311114909-570f5725d595+dirty built with go: go1.24.0)
staticcheck:	/home/lqw/go/bin/staticcheck	(version: v0.6.0 built with go: go1.24.0)

Go env

Workspace Folder (delve): /home/lqw/mygit/delve

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE='on'
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/lqw/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/lqw/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build4247711127=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/home/lqw/mygit/delve/go.mod'
GOMODCACHE='/home/lqw/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/lqw/go'
GOPRIVATE=''
GOPROXY='https://goproxy.cn,direct'
GOROOT='/home/lqw/go1.24.0'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/lqw/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/lqw/go1.24.0/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.24.0'
GOWORK=''
PKG_CONFIG='pkg-config'

Share the Go related settings you have added/edited

Run Preferences: Open Settings (JSON) command to open your settings.json file.
Share all the settings with the go. or ["go"] or gopls prefixes.

"go.alternateTools": {
    "dlv": "<modified dlv mentioned above>"
}

Describe the bug

A clear and concise description of what the bug.
A clear and concise description of what you expected to happen.

Bug: The suspended argument is always false when CreateBreakpoint is invoked in delve server through RPC call by vscode-go debug adapter. The consequence is that when the location of a breakpoint is not found in the delve debugger, the breakpoint will be deleted in service/debugger/debugger.go#L792-L797 as shown in the following code, which makes vscode frontend fail to set the breakpoint. Current interface not fully utilizes the ability of suspending breakpoints in delve, which can limit vscode-go debug adapter's ability in cases like multiple process debugging even if FollowExec is enabled.

if suspended {
	logflags.DebuggerLogger().Debugf("could not enable new breakpoint: %v (breakpoint will be suspended)", err)
} else {
	delete(d.target.LogicalBreakpoints, lbp.LogicalID)
	return nil, err
}

What I expected to see: suspended is always sent true so that when the location of a breakpoint is not found, the breakpoint can still be suspended. In the future, the breakpoint will be valid.

The interfaces in both sides are:

• vscode-go extension/src/debugAdapter/goDebug.ts:

  vscode-go/extension/src/debugAdapter/goDebug.ts

  Line 2189 in 1a21ede

  ?.callPromise('CreateBreakpoint', [

• delve:
  • argv = reflect.New(mtype.ArgType) at https://github.com/go-delve/delve/blob/570f5725d595dc698d257f91326e677b9eda89df/service/rpccommon/server.go#L238
  • function.Call([]reflect.Value{argv, reflect.ValueOf(ctl)}) at https://github.com/go-delve/delve/blob/570f5725d595dc698d257f91326e677b9eda89df/service/rpccommon/server.go#L295
  • s.debugger.CreateBreakpoint(&arg.Breakpoint, arg.LocExpr, arg.SubstitutePathRules, arg.Suspended) at https://github.com/go-delve/delve/blob/570f5725d595dc698d257f91326e677b9eda89df/service/rpc2/server.go#L263

Current interface in delve is:

• https://github.com/go-delve/delve/blob/570f5725d595dc698d257f91326e677b9eda89df/service/api/types.go#L74-L141
• https://github.com/go-delve/delve/blob/570f5725d595dc698d257f91326e677b9eda89df/service/rpc2/server.go#L243-L249

Code of delve

// Breakpoint addresses a set of locations at which process execution may be
// suspended.
type Breakpoint struct {
	// ID is a unique identifier for the breakpoint.
	ID int `json:"id"`
	// User defined name of the breakpoint.
	Name string `json:"name"`
	// Addr is deprecated, use Addrs.
	Addr uint64 `json:"addr"`
	// Addrs is the list of addresses for this breakpoint.
	Addrs []uint64 `json:"addrs"`
	// AddrPid[i] is the PID associated with by Addrs[i], when debugging a
	// single target process this is optional, otherwise it is mandatory.
	AddrPid []int `json:"addrpid"`
	// File is the source file for the breakpoint.
	File string `json:"file"`
	// Line is a line in File for the breakpoint.
	Line int `json:"line"`
	// FunctionName is the name of the function at the current breakpoint, and
	// may not always be available.
	FunctionName string `json:"functionName,omitempty"`
	// ExprString is the string that will be used to set a suspended breakpoint.
	ExprString string

	// Breakpoint condition
	Cond string
	// Breakpoint hit count condition.
	// Supported hit count conditions are "NUMBER" and "OP NUMBER".
	HitCond string
	// HitCondPerG use per goroutine hitcount as HitCond operand, instead of total hitcount
	HitCondPerG bool

	// Tracepoint flag, signifying this is a tracepoint.
	Tracepoint bool `json:"continue"`
	// TraceReturn flag signifying this is a breakpoint set at a return
	// statement in a traced function.
	TraceReturn bool `json:"traceReturn"`
	// retrieve goroutine information
	Goroutine bool `json:"goroutine"`
	// number of stack frames to retrieve
	Stacktrace int `json:"stacktrace"`
	// expressions to evaluate
	Variables []string `json:"variables,omitempty"`
	// LoadArgs requests loading function arguments when the breakpoint is hit
	LoadArgs *LoadConfig
	// LoadLocals requests loading function locals when the breakpoint is hit
	LoadLocals *LoadConfig

	// WatchExpr is the expression used to create this watchpoint
	WatchExpr string
	WatchType WatchType

	VerboseDescr []string `json:"VerboseDescr,omitempty"`

	// number of times a breakpoint has been reached in a certain goroutine
	HitCount map[string]uint64 `json:"hitCount"`
	// number of times a breakpoint has been reached
	TotalHitCount uint64 `json:"totalHitCount"`
	// Disabled flag, signifying the state of the breakpoint
	Disabled bool `json:"disabled"`

	UserData interface{} `json:"-"`

	// RootFuncName is the Root function from where tracing needs to be done
	RootFuncName string
	// TraceFollowCalls indicates the Depth of tracing
	TraceFollowCalls int
}

type CreateBreakpointIn struct {
	Breakpoint api.Breakpoint

	LocExpr             string
	SubstitutePathRules [][2]string
	Suspended           bool
}

Describe the solution you'd like
A clear and concise description of what you want to happen.

add suspended in RPC pack. vscode-go always send suspended=true to delve.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

nothing yet.

Additional context
Add any other context or screenshots about the feature request here.

This issue is blocking #3712. #3712 is a promising enhancement.

Related PR: go-delve/delve#3900

[Lslightly]

After trying to add suspended in the RPC data in legacy delve adapter as follows in delve and goDebug.ts, I found that CreateBreakpoint will return some unwanted breakpoint.

type CreateBreakpointIn struct {
	Breakpoint api.Breakpoint

	LocExpr             string
	SubstitutePathRules [][2]string

	// Suspended decides whether the breakpoint is suspended when its location
	// is not found when creating it
	Suspended bool `json:"suspended"`
}

this.delve?.isApiV1 ? breakpointIn : { Breakpoint: breakpointIn, suspended: true }

unwanted breakpoint:

---

Reproduce the above screenshot:

git clone -b fixSuspended https://github.com/Lslightly/delve.git
git clone -b fixSuspended https://github.com/Lslightly/delve-dbg-child-proc.git
cd delve-dbg-child-proc
go build -gcflags "-N -l" . # avoid optimization and inlining
cd utils
go build -gcflags "-N -l" . # avoid optimization and inlining
cd ../../
git clone --depth 2 -b fixSuspended https://github.com/Lslightly/vscode-go.git
cd delve
# build delve
make build
# backup old dlv in your GOPATH
cp $(go env GOPATH)/bin/dlv $(go env GOPATH)/bin/dlv_backup
# use new dlv
ln -s $(pwd)/dlv $(go env GOPATH)/bin/dlv
# install vscode-go dependencies
cd ../vscode-go/extension
npm ci
# Then press launch F5 as docs/contribution.md says.
# open delve-dbg-child-proc in [Extension Development Host] vscode instance
# launch debug session in delve
# launch debug session in [Extension Development Host] vscode instance

Notice that in the settings.json in Lslightly/delve-dbg-child-proc, legacy adapter is used.

[Lslightly]

The cause of unwanted breakpoint: (legacy debug adapter)

[firelizzard18]

Just FYI, I don't think the legacy adapter is being actively developed.

[Lslightly]

Just FYI, I don't think the legacy adapter is being actively developed.

Thanks. I also see the caller of CreateBreakpoint at service/dap/server.go#L1525, i.e. the DAP service of delve. Since suspended is hard coded as false, I choose to change the legacy mode first, which is easier to change the interface.

got, err = s.debugger.CreateBreakpoint(bp, "", nil, false)

Possibly changing false to true, combining with a fake breakpoint service/debugger/debugger.go#L812-L821, is sufficient.

	if suspended {
		// Create a breakpoint with enough information although its location is not found to cheat the frontend
		// Not completed yet.
		// Possibly notifying the client through err is a better solution
		// ErrSuspended can be a part of RPC interface
		createdBp.FunctionName = setbp.FunctionName
		createdBp.File = setbp.File
		createdBp.Line = setbp.Line
		createdBp.ExprString = setbp.ExprString
	}

---

Besides, @firelizzard18 Thanks for your detailed explanation of what goDebug.ts is. And here is the discussion link in gophers slack for future developers(possibly fewer in legacy adapter 😄 ).

[firelizzard18]

@Lslightly Do you know what happens if the breakpoint can't be set? If s.debugger.CreateBreakpoint returns a specific error in that case, would it be sufficient to detect that and call it again with suspended set to true?

[Lslightly]

Do you know what happens if the breakpoint can't be set?

If not suspended, delve will return a SetBreakpointsResponse whose body contains breakpoints []dap.Breakpoint(Breakpoint) like the following:

[]go-dap.Breakpoint len: 1, cap: 1, [{Id: 0, Verified: false, Message: "could not find file /home/lqw/mygit/archived/delve-dbg-child-proc/utils/build.go", Source: *github.com/google/go-dap.Source nil, Line: 0, Column: 0, EndLine: 0, EndColumn: 0, InstructionReference: "", Offset: 0}]

So there is the missing point! There is difference between go-dap Breakpoint definition and Breakpoint spec. reason field is missing. I update go-dap and open the PR in go-dap.

If s.debugger.CreateBreakpoint returns a specific error in that case, would it be sufficient to detect that and call it again with suspended set to true?

The error message is not special.

But if the user of vscode-go allows suspending breakpoints (or by default), then initial configuration to allow suspending can be sent to delve so that delve can return pending as reason when a breakpoint is not found.

[firelizzard18]

In CreateBreakpoint:

if suspended {
	logflags.DebuggerLogger().Debugf("could not enable new breakpoint: %v (breakpoint will be suspended)", err)
}

That's the only place suspended is referenced. So simply hard-coding true might be enough.

[firelizzard18]

I spent some time trying to make this work. So far, I got it to the point where suspended breakpoints appear in vscode: https://github.com/firelizzard18/delve/tree/dap-suspended-breakpoints. The DAP SetBreakpointRequest handler hard-codes suspend to true and has some extra hackery to send the correct location data back to vscode.

However, they don't completely work.

• Build this plugin.
• Set a breakpoint in main.go before the plugin is loaded.
• Set a breakpoint in plugin.go.
• Launch it with the debugger.
• The main.go breakpoint is active, the plugin.go breakpoint is active.
• Step past loading the plugin.
• The plugin.go breakpoint should become active but doesn't.
• But if you step past the plugin breakpoint, it does stop.

Presumably the same would be true for follow-exec, but it seems like a good idea to get it to work for plugins first, since those are already supported in DAP mode. Delve is detecting when a plugin is loaded and un-suspending relevant breakpoints, however for this to work correctly delve needs to notify vscode. I believe the solution is to send a Breakpoint event. Unfortunately I need to get back to my day job. @Lslightly I gave you access to my delve fork - if you have the time and energy to look into this (issuing a Breakpoint event), maybe we can solve it together. If we can get this to work for plugins (and get that merged into delve), then I'm hoping adding support for follow-exec will be easier.

To debug delve, I open delve in one window and the target (the plugin thing) in another, run the delve launch config below, then run the target debug config which attaches to the previously launched delve instance. If you have them both in the same window, things get weird - if you set a breakpoint in delve it will also try to set that breakpoint in the target and vice versa.

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Delve",
            "type": "go",
            "request": "launch",
            "mode": "auto",
            "program": "${workspaceFolder}/cmd/dlv",
            "cwd": "/home/firelizzard/src/misc/hermes/example",
            "args": ["dap", "--listen=127.0.0.1:37653", "--log-dest=3"]
        },
    ]
}