After reading this document, you should be able to produce Debian packages for Go software.

General notes

Team Maintenance

All Go packages are team-maintained in the pkg-go team. This has multiple advantages:

  1. There are no single points of failure, if one maintainer cannot continue maintaining a package, another maintainer can easily help out or take over.

  2. All packages can share the same technical standards and workflows.

  3. When working within a team anyway, it is much easier to get help on any technical issue.

A package maintained within the team should have the name of the team either in the Maintainer field or in the Uploaders field. We use tracker.debian.org’s team functionality and spell the team like this: Debian Go Packaging Team <team+pkg-go@tracker.debian.org>. This enables the team to have an overview of its packages on the DDPO website and ensures that all packages maintained by the team are automatically added to the Debian Package Tracker team.

Putting the team in Maintainers is a strong statement that fully collaborative maintenance is preferred. Anyone can commit to the git repository and upload as needed. A courtesy email to Uploaders can be nice but not required.

If for some reason, a package needs tighter control by its original maintainers, putting the team in Uploaders is a weak statement of collaboration. Help in maintaining the package is appreciated, commits to the git repository are freely welcomed, but before uploading, please contact the Maintainer for the green light.

Packaging in git

All Go packages are maintained in git and must be buildable gbp build-package using git-buildpackage.

We use Salsa to store the git repositories.

Git setup for automatically authenticating

When using git-buildpackage’s vcsgit feature (e.g. gbp clone vcsgit:golang-text), the following configuration results in git automatically rewriting URLs to be authenticated:

git config --global url."git.debian.org:/git/".insteadOf "https://anonscm.debian.org/git/"
git config --global url."git@salsa.debian.org:".insteadOf "https://salsa.debian.org/"

Using dh-make-golang

Instead of copying the debian/ directory from some random other package, please use dh-make-golang when you start packaging a new Go library/program.

Also dh-make-golang can create Salsa projects, and configure the CI automatically. Guest accounts on Salsa can’t create projects in go-team group, they can only do it with this approach. Please run dh-make-golang create-salsa-project to see the usage.

Version numbers

Many Go libraries (and also some actual Go programs) don’t have version numbers in the traditional sense, but live in a version control repository of some kind.

In case your upstream does not use version numbers, the Debian package version will look like this:

0.0~git20130606.b00ec39-1

  • The 0.0 in the beginning is used to allow upstream to adopt version numbers at any point in time and also to make it clear that no traditional version number is used for this package.

  • The second part is the version control system, e.g. git, hg, svn.

  • Afterwards, a date follows in the format YYYYMMDD.

  • After the dot, the version control system revision follows, to make it clear which commit was packaged, as many repositories have multiple commits on a given day.

  • The last part after the dash is the Debian version number.

In case you make more than one snapshot per day, you can append a snapshot number after the date, e.g. 0.0~git20130606.2.b00ec39-1. This should rarely be necessary.

changelog: UNRELEASED

During the time when you still work on a package, i.e. before it is ready to upload, please put UNRELEASED into the distribution field in debian/changelog (dch -v <debian_version> will do it automatically). When the package is ready for uploading, change it to unstable (dch -r).

If you change something that has to be noted in debian/changelog, just add a line to the current entry (dch -a). The [firstname lastname] markers added by dch are okay to give credit to non-upload-permitted contributors (also for the initial changelog entry).

Important NOTES to other group members may be placed at the top of the current changelog entry of packages that are not yet ready for upload (e.g. why a package is still UNRELEASED, etc.).

“upstream” branch should contain upstream git history

TODO: describe the rationale

Migration

  1. Add upstream repository as a new remote upstm. Then fetch its history.

    git remote add upstm https://github.com/paultag/go-topsort
git fetch upstm

  2. Delete/Rename the old upstream branch, and checkout a new upstream branch.

    git branch -m upstream old/upstream
git checkout -b upstream upstm/master

  3. Merge upstream history into the packaging branch, at the same point where the previous upstream was, usually based on release tags if upstream does releases; otherwise, the Debian release number will contain the commit ID.

    git diff <upstream_commit_id> old/upstream  # should be no differences
git checkout master
git merge --allow-unrelated-histories <upstream_commit_id>

Packaging new upstream release

When packaging new upstream release, you can fetch upstream remote and merge it into upstream branch.

If upstream does tag release, you can merge the new tag into upstream branch, and create a new tag with upstream/<version> format, which will be used by gbp.

git fetch upstm
git checkout upstream
git merge <upstream_tag>
git tag -a upstream/<version>
git checkout master
git merge upstream/<version>

If upstream doesn’t have tags, you can merge the latest commit into upstream branch, and create the upstream/<version> tag by commit date and id.

git fetch upstm
git checkout upstream
git merge upstm/master
TAG=$(git log --date=format:%Y%m%d --pretty=upstream/0.0_git%cd.%h -1)
git tag -a "$TAG"
git checkout master
git merge "$TAG"

Drop pristine-tar branches

Rationale

The appeal of using pristine-tar was that a byte-for-byte equal orig tarball could be easily (and automatically) generated, preventing rejected uploads and aiding future historians with reproducibly rebuilding older versions. Further, using pristine-tar results in a self-contained git repository which makes for a simple mental model (as opposed to the git repository + orig tarball in parent directory model).

In reality, pristine-tar branches weren’t consistently updated across our repositories for a number of reasons, resulting in despised additional maintenance effort for little benefit (uploads may or may not be rejected, depending on the repository state).

Hence, even though the new gbp push command and a consistent setting of pristine-tar=True in debian/gbp.conf might help improve consistency, we decided to abandon pristine-tar altogether.

Old workflow

  • Obtain a byte-for-byte equal orig .tar.gz file: pristine-tar list and pristine-tar checkout

  • Build against a byte-for-byte equal orig .tar.gz file: enable gbp buildpackage’s pristine-tar option, e.g. via --git-pristine-tar or pristine-tar=True in debian/gbp.conf.

New workflow

  • Obtain a byte-for-byte equal orig .tar.gz file: origtargz(1)

  • Build against a byte-for-byte equal orig .tar.gz file: use origtargz before building

Note
 Using --git-upstream-tree=TAG (the default) is not sufficient to obtain a byte-for-byte equal orig .tar.gz file. In stapelberg’s tests, only 5% of pkg-go’s git repositories would match the orig .tar.gz in the archive.
Tip

Configure gbp-clone(1) to automatically run origtargz(1) by adding the following to ~/.gbp.conf:

[clone]
postclone=origtargz

Migration

  1. Remove any pristine-tar-related settings from this repository’s gbp config files, defaulting to pristine-tar=False:

    for f in .gbp.conf debian/gbp.conf .git/gbp.conf
do
    [ -e "$f" ] && sed -i '/^pristine-tar/d' "$f"
done

  2. Delete the pristine-tar branch: git push origin :pristine-tar

TODO: verify step 2 works

Newer worflow

We use uscan to obtain an "orig .tar.gz" file and gbp import-orig for getting it into the git repository. All in one command:

gbp import-orig --uscan

Auto-format debian/control

Rationale

Common formatting increases consistency between packages maintained by pkg-go and auto-formatting frees up time previously spent on manual formatting.

Old workflow

Manually format control files such as debian/control.

New workflow

Use wrap-and-sort(1) from the devscripts package in the root of the Debian package directory:

wrap-and-sort --wrap-always --trailing-comma

The command line arguments result in a format which produces minimal diffs whenever new values are added or old values are removed.

TODO: is there any editor integration for wrap-and-sort yet?

Tip

To auto-format before committing, create the following hook at .git/hooks/pre-commit:

#!/bin/sh
wrap-and-sort --wrap-always --trailing-comma

Migration

  1. Run wrap-and-sort --wrap-always --trailing-comma

Adopt DEP-14 branch naming

Rationale

Consistency in our branch naming makes it easier for team-internal and team-external contributors to understand/interact with our packaging repositories.

Old workflow

The default branch of a package was named master.

New workflow

See DEP-14 for the full text. In a nutshell:

  • The default branch of a package should be named debian/sid.

  • git HEAD should point to debian/sid.

  • The upstream git history should live in a branch named upstream.

    TODO: wait until discussion in #812721 came to an agreement

  • Packages which release into stable releases should use the codename of the target distribution, e.g. debian/bookworm.

Migration

  1. TODO: describe the branch rename.

  2. git symbolic-ref HEAD refs/heads/debian/sid

Derive debian/changelog from git history

Rationale

Some pkg-go repositories used the old workflow, some used the new workflow — this was just a personal preference of whoever last touched the repository.

Unifying this difference eases contributions to any pkg-go repository.

Old workflow

  • Document your changes in a pending debian/changelog entry with the special suite name UNRELEASED.

  • Document your changes in your git commit(s).

New workflow

  • Document your changes in your git commit(s).

  • Before uploading the package to Debian, use gbp dch -R --commit.

TODO(stapelberg): install https://paste.debian.net/hidden/c3a81ddf/ (git update hook which declines old-workflow d/changelog updates) in all pkg-go repositories

Migration

TODO: remove pending d/changelog entries from existing packages.

Binary-only packages

A binary-only package is a package that contains a program written in Go, but no source code. An example is docker, which is written in Go, but does not offer an API (thus no source code).

Naming Conventions

The source package should be named like the upstream project, i.e. docker, you do NOT need to call it golang-docker.

Similarly, the resulting binary package(s) should NOT contain the golang- prefix.

Use dh-golang

Install dh-golang from Debian unstable so that you are using the newest version. The buildds are using the unstable version, too, so this is important.

dh-golang comes with an example debian/rules file: https://salsa.debian.org/go-team/packages/dh-golang/-/blob/debian/sid/example/rules

You will need to change the value of XS-Go-Import-Path in debian/control to correspond to your program’s upstream package name. This is usually what you would go get when installing it manually. dh-golang needs that information so that it can run go install.

dh-golang sets up a build environment that contains all the libraries that are available in /usr/share/gocode/src, so you need to add Build-Depends to your package. As an example, Debian Code Search depends on golang-github-lib-pq-dev and others: https://github.com/Debian/dcs/blob/master/debian/control#L5

Library (or binary + library) packages

Libraries written in Go are packaged for Debian with the only purpose of building other Go programs for Debian. They are specifically not available for users in their regular development workflow. For that, users should use go get. The rationale behind this decision is:

  • By using go get you are forced to set up the environment variable $GOPATH which you need for other go commands (e.g. go build, go test, etc.).

  • Debian packages install files to system locations, which users cannot modify. That means that users would not be able to upgrade packages with the canonical go get -u <package>. Even worse, when using sudo to forcibly modify the system files, it still would not work since no VCS information is contained in the Debian packages.

Naming Conventions

We derive Debian package names from the import path by replacing all slashes with dashes and using a canonical identifier instead of the hostname.

E.g., for github.com/stapelberg/godebiancontrol (which contains "go" already), the resulting Debian package name is golang-github-stapelberg-godebiancontrol-dev.

We use the -dev suffix to keep the namespace clean for shipping shared libraries in the future, which would then be shipped as golang-github-stapelberg-godebiancontrol.

In general, you should use the import path for deriving a name, not the actual package name. Ideally, those are the same anyway.

Here are a couple examples:

Import path Debian package name

github.com/stapelberg/websocket

golang-github-stapelberg-websocket-dev

code.google.com/p/go.net/websocket

golang-googlecode-go.net-websocket-dev

golang.org/x/oauth2

golang-golang-x-oauth2-dev

google.golang.org/appengine

golang-google-appengine-dev

File locations

All files should be installed into /usr/share/gocode/src/, which corresponds to $GOPATH/src. As an example, for github.com/lib/pq (golang-github-lib-pq-dev), one of the files is /usr/share/gocode/src/github.com/lib/pq/buf.go.

See https://packages.debian.org/sid/all/golang-github-lib-pq-dev/filelist for a full example file list.

Dependencies

Your library package, e.g. golang-github-lib-pq-dev, needs to have all the other Go libraries it depends on in its Depends line. The dependencies need to be available at build time to run the tests (if any) and at installation time so that other packages can be built.

Upstream package moves

Occasionally, upstream packages might move from one code hosting provider to a different one, as was the case with code.google.com being discontinued and many projects moving to GitHub.

Such a move should be mentioned in debian/changelog and a compatibility symbolic link should be installed using debian/links (see https://salsa.debian.org/go-team/packages/golang-go.crypto/commit/4e0a285a0989331edb5ff580797ab3574197534d for an example). Since the location of the package is also contained in the Debian package’s name, it should be renamed (see https://salsa.debian.org/go-team/packages/golang-go.net/commit/189085288e608ca2720b32d551227d330a561123 for an example).