Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go tool -n can point to a file that doesn't exist #72824

Open
iwahbe opened this issue Mar 12, 2025 · 7 comments · May be fixed by #72902
Open

cmd/go: go tool -n can point to a file that doesn't exist #72824

iwahbe opened this issue Mar 12, 2025 · 7 comments · May be fixed by #72902
Labels
BugReport Issues describing a possible bug in the Go implementation. GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.

Comments

@iwahbe
Copy link

iwahbe commented Mar 12, 2025

Go version

go version go1.24.1 darwin/arm64

Output of go env in your module/workspace:

AR='ar'
CC='cc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='c++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN='/Users/ianwahbe/go/bin'
GOCACHE='/Users/ianwahbe/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/ianwahbe/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/fg/_1q36r4j6yx0rwz2fbhjd5y40000gn/T/go-build3052038314=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/Users/ianwahbe/go/src/github.com/example/empty/go.mod'
GOMODCACHE='/Users/ianwahbe/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/ianwahbe/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.24.1/libexec'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/ianwahbe/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.24.1/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.24.1'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

I was using go tool -n github.com/iwahbe/helpmakego in my Makefile to work around #71733 like this:

HELPMAKEGO := $(shell go tool -n github.com/iwahbe/helpmakego)

# Use the tool as intended - details don't really matter here
$(shell ${HELPMAKEGO} ...)

I noticed that sometimes $(shell ${HELPMAKEGO} ...) would fail unexpectedly, saying that there was no file at ${HELPMAKEGO}.

I can replicate this behavior locally:

$ mkdir repro
$ go mod init
$ go get -tool github.com/iwahbe/helpmakego
$ go tool -n github.com/iwahbe/helpmakego | tee should_exist && rm $(cat should_exist)
/Users/ianwahbe/Library/Caches/go-build/c5/c503c5059f1c48a29b1dc68749074f475aaf82c717ef355e7079f7e9b4116c2b-d/helpmakego
$ go tool -n github.com/iwahbe/helpmakego | tee should_exist && rm $(cat should_exist)
/var/folders/fg/_1q36r4j6yx0rwz2fbhjd5y40000gn/T/go-build1160981703/b001/exe/helpmakego
rm: /var/folders/fg/_1q36r4j6yx0rwz2fbhjd5y40000gn/T/go-build1160981703/b001/exe/helpmakego: No such file or directory
$ go tool -n github.com/iwahbe/helpmakego | tee should_exist && rm $(cat should_exist)
/Users/ianwahbe/Library/Caches/go-build/c5/c503c5059f1c48a29b1dc68749074f475aaf82c717ef355e7079f7e9b4116c2b-d/helpmakego
$ go tool -n github.com/iwahbe/helpmakego | tee should_exist && rm $(cat should_exist)
/var/folders/fg/_1q36r4j6yx0rwz2fbhjd5y40000gn/T/go-build1221207145/b001/exe/helpmakego
rm: /var/folders/fg/_1q36r4j6yx0rwz2fbhjd5y40000gn/T/go-build1221207145/b001/exe/helpmakego: No such file or directory

You will see that every other time go tool -n github.com/iwahbe/helpmakego it points to a file that doesn't exist.

What did you see happen?

$ go tool -n github.com/iwahbe/helpmakego
/var/folders/fg/_1q36r4j6yx0rwz2fbhjd5y40000gn/T/go-build1221207145/b001/exe/helpmakego

What did you expect to see?

I expect that any time go tool -n ... is run, it will point to a runnable executable.

If that isn't true, then go help tool should document what conditions are necessary to ensure that go tool -n ... will point to an executable file.

@gabyhelp gabyhelp added the BugReport Issues describing a possible bug in the Go implementation. label Mar 12, 2025
@seankhliao
Copy link
Member

seems like when it builds a tool it prints the location of it in the temporary work directory, rather than the final location in the build cache

@ianlancetaylor
Copy link
Member

CC @matloob @samthanawalla

@ianlancetaylor ianlancetaylor added GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 12, 2025
@KirtanSoni
Copy link

I figured out the cause on the bug. would u mind if i take this issue up and propose a PR

@KirtanSoni KirtanSoni linked a pull request Mar 17, 2025 that will close this issue
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/658395 mentions this issue: cmd/go: respect -n flag in 'go tool' command

@matloob
Copy link
Contributor

matloob commented Mar 17, 2025

The behavior in your repro log doesn't match what I'd expect: specifically my expectation is that we return the temp file location the first time we run go tool (where the tool is not yet cached) and the location to the cached path afterwards. Here's a log from my repro:

matloob@matloob-mac Desktop % mkdir repro
matloob@matloob-mac Desktop % go mod init example.com/repro
go: creating new go.mod: module example.com/repro
go: to add module requirements and sums:
	go mod tidy
matloob@matloob-mac Desktop % go get -tool github.com/iwahbe/helpmakego
go: downloading github.com/iwahbe/helpmakego v0.1.0
go: added github.com/inconshreveable/mousetrap v1.1.0
go: added github.com/iwahbe/helpmakego v0.1.0
go: added github.com/spf13/cobra v1.8.1
go: added github.com/spf13/pflag v1.0.5
go: added golang.org/x/mod v0.22.0
matloob@matloob-mac Desktop % go tool -n github.com/iwahbe/helpmakego
/var/folders/k3/3s_zz9nx1ms089s9vjb70xw8004_sl/T/go-build1617190989/b001/exe/helpmakego

matloob@matloob-mac Desktop % 
matloob@matloob-mac Desktop % go tool -n github.com/iwahbe/helpmakego
/Users/matloob/Library/Caches/go-build/11/1102170b7a6b4ae7d914933cb74aac17d22032c6b28f695dfdafa976774e1f5c-d/helpmakego
matloob@matloob-mac Desktop % rm /Users/matloob/Library/Caches/go-build/11/1102170b7a6b4ae7d914933cb74aac17d22032c6b28f695dfdafa976774e1f5c-d/helpmakego

matloob@matloob-mac Desktop % go tool -n github.com/iwahbe/helpmakego
/var/folders/k3/3s_zz9nx1ms089s9vjb70xw8004_sl/T/go-build3727914114/b001/exe/helpmakego
matloob@matloob-mac Desktop % /var/folders/k3/3s_zz9nx1ms089s9vjb70xw8004_sl/T/go-build3727914114/b001/exe/helpmakego
zsh: no such file or directory: /var/folders/k3/3s_zz9nx1ms089s9vjb70xw8004_sl/T/go-build3727914114/b001/exe/helpmakego

What's happening is that the first time we build the binary, we build the executable to a temporary directory and then execute the command from the built location. (This is what go run did before executable caching was added) Every subsequent time, we fetch the executable from the cache and run it from there. If we decide to, we can make the example work by setting the 'built' field to be the cache location of the executable in the last condition in cmd/go/internal/work.(*Builder).updateBuildID.

I'm a bit unsure of doing this because the cache is supposed to be an implementation detail of the go command and it might not be a good idea for tools to depend on the path of the file in the cache. It is better to run go build -o to fetch the binary from the cache to a location controlled by the user, and then run the binary from there.

Of course if we decide that the file returned by -n isn't always available, we should update the documentation to indicate that the tool may not necessarily be available from the location returned by go tool -n.

@KirtanSoni
Copy link

KirtanSoni commented Mar 17, 2025

@matloob I wanted to propose the same change considering a different implementation (behind the -n flag), to point the fresh cache to a.built, and poison the target. Another workaround I can propose is that we can print out an error if the cache misses, as it will be futile to try and run the build that doesn't exist. This way we can isolate the implementation in go tool.

Also in my opinion, with the -n flag, that output is not usable when run without a cache. I would propose to keep the implementation of -n similar to the other go commands, which would make it consistent and easier to work with.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BugReport Issues describing a possible bug in the Go implementation. GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants