unreasonably large executable size after including dd-trace-go

[steabert]

Hi!

I'm trying to move from using OpenTelemetry (via go.opentelemetry.io/otel/trace) for custom tracing to using dd-trace-go/v2/ddtrace/tracer in my application (a server which runs in a kubernetes cluster with a datadog agent present).

Unfortunately, the executable size increases by ~9MB compared to using the otel libraries. This seems unreasonable for a library which is supposed to create some spans and send them off to the agent. Trying to look into it myself I found that part of the issue was that it pulls in gogo/protobuf, so I assume that's used as a wire format for agent communication, fair enough. However that cannot account for the entire size.

Does anyone know why this library increases executable size by ~9MB ?

[darccio]

@steabert Hi, thanks for reaching out. dd-trace-go includes multiple products. The Go toolchain should be able to prune most of the library and dependencies' code if only importing ddtrace/tracer.

We did a quick test with a net/http server:

package main

import (
        "fmt"
        "net/http"

        "github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
        httptrace "github.com/DataDog/dd-trace-go/contrib/net/http/v2"
)

func main() {
        tracer.Start()
        defer tracer.Stop()

        mux := httptrace.NewServeMux()
        mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
                fmt.Fprintf(w, "Hello world!")
        })
        http.ListenAndServe(":3000", nil)
}

This is the breakdown as generated by github.com/Zxilly/go-size-analyzer:

Regarding gogo/protobuf, we are going to check it because it's not a direct dependency.

[darccio]

@steabert You can use grpcnotrace (from Google's gRPC library) to optimize the size. The reason is that gRPC pulls golang.org/x/net/trace which uses some reflect methods that disable compiler optimizations.

We can confirm that using github.com/aarzilli/whydeadcode:

❯ go build -ldflags=-dumpdep -o manually-instrumented-go-http-puppy |& whydeadcode
text/template.(*state).evalField reachable from:
         text/template.(*state).evalFieldChain
         text/template.(*state).evalCommand
         text/template.(*state).evalPipeline
         text/template.(*state).walk
         text/template.(*Template).execute
         html/template.(*Template).Execute
         golang.org/x/net/trace.RenderEvents
         golang.org/x/net/trace.Events
         golang.org/x/net/trace.Events·f
         golang.org/x/net/trace.init.0
         golang.org/x/net/trace..inittask
         go:main.inittasks
         _

github.com/modern-go/reflect2.safeStructType.MethodByName reachable from:
         type:github.com/modern-go/reflect2.safeStructType
         github.com/modern-go/reflect2.(*frozenConfig).wrapType
         github.com/modern-go/reflect2.(*frozenConfig).Type2
         github.com/modern-go/reflect2.(*frozenConfig).TypeOf
         github.com/json-iterator/go.init
         github.com/json-iterator/go..inittask
         go:main.inittasks
         _

❯ go build -tags grpcnotrace -ldflags=-dumpdep -o manually-instrumented-go-http-puppy |& whydeadcode
github.com/modern-go/reflect2.safeStructType.MethodByName reachable from:
         type:github.com/modern-go/reflect2.safeStructType
         github.com/modern-go/reflect2.(*frozenConfig).wrapType
         github.com/modern-go/reflect2.(*frozenConfig).Type2
         github.com/modern-go/reflect2.(*frozenConfig).TypeOf
         github.com/json-iterator/go.init
         github.com/json-iterator/go..inittask
         go:main.inittasks
         _

In our test, we saw a 25% reduction in binary size.

[steabert]

Thanks for the info!

The code I had already uses grpc, but maybe those functions weren't called. I'll see if grpcnotrace makes a difference, and run https://github.com/aarzilli/whydeadcode as well.

[darccio]

@steabert Did you get good results? I must say that we are investigating some venues to reduce our footprint.

[darccio]

We did some efforts this quarter, and found new venues to further reduce it. We've removed unmaintained dependencies and documented the usage of known build tags that can yield more than 5% reduction.