Go modules is the new dependency management system for Go language based projects. I just spent the last three days converting the Groups.io codebase to use Go modules. It’s a large code base, and we had vendored ~75 dependencies over the past five years of development. The code base pre-dates any of the pre-existing dependency management systems, so we were flying a bit blind when it came to which version of each dependency we were using. There are several more comprehensive guides to converting to Go modules; this is just a set of my notes. I may update them as I get more experience with modules.
I wanted to do the conversion incrementally, to give me more opportunity to test things and make sure I wasn’t breaking the build. What I did was create the empty go.mod file, and then within it, I added replace directives for each dependency we had vendored. The replace directive pointed into the vendor directory, like this:
github.com/jackc/pgx => ./vendor/github.com/jackc/pgx
Once I had done that, I started running
go build. That would complain about missing go.mod files in each of the dependent directories. I would dutifully go in and create these missing go.mod files one by one. I continued doing this until there were no more errors.
Now that I had a working build, I started the review process. I went through each dependency and, by seeing when we vendored it, I could review the upstream changes. For dependencies that had no changes I was uncomfortable with, I would then remove the replace directive in the go.mod file, letting Go manage that dependency.
Local Changes And Deleted Repos
We have made changes to several of these dependencies (that are not appropriate for upstream pull-requests), and we need to keep those changes. Also, a few dependences that we had vendored no longer exist, because their creators have deleted the repositories. I created a new top-level directory,
internal. I then moved those dependencies from
internal, and updated the relevant
replace directive to point into
Keeping A /vendor Directory
After all that work, we had a
go.mod file with a large number of require directives, and about 14 replace directives. I removed the existing vendor directory from the repository. A downside quickly became apparent, however. Within Sublime Text, like most editors, I can hover over a type or function call and Sublime will list the various files where it may have been defined. This is very helpful during development. But without a vendor directory, Sublime couldn’t locate any of the moduled dependencies. It can be handy to have all the source code on hand/available to the editor when doing development. To generate a /vender directory from your
go.mod file, use the command
go mod vendor. To then use the /vendor directory for builds, add the parameter
-mod=vendor when building.
Another advantage of keeping a /vendor directory is if you run the godoc tool locally. If you include the query parameter
?m=all when accessing the godoc webpage, it will include the /vendor directory (as well as the /internal directory, if you have one) when generating documentation.
go install gotcha
With go modules, the
GOPATH environment variable is no longer needed. If you run
go install without a GOPATH, it apparently does nothing, as it doesn’t know where to put the generated binary. To get go install to work again, I had to set the
GOBIN environment variable.