Bazel is a fast building system open sourced by Google. It has go support but it does not have a detailed instruction on how to integrate CGO with the bazel build system. Building everything from scratch with cc_library and manually written go_library is do-able. However, we do prefer the amazing gazelle and sometimes it is hard to use prebuilt libraries with CGO build without digging up in the source code in the gazelle.

When writing the post, the latest versions of each tools are listed as following:

Toolchain	Version
Golang	v1.16
bazelbuild/bazel	4.0.0
bazelbuild/rules_go	v0.27.0
bazelbuild/bazel-gazelle	v0.23.0

I'm using an opus format encoding, decoding library, hraban/opus, as an example which shows the most cases I would like to cover. The whole source code in the guide is available at github:

https://github.com/adamcavendish/bazel-cgo-demo

Step 1: Initialize the repository

When starting off integrating a golang repository with Bazel, we shall start with bazelbuild/rules_go and bazelbuild/bazel-gazelle to setup the WORKSPACE file and BUILD.bazel file in the root path. It is covered with the official READMEs in both repositories.

Install Dependent Packages

CGO requires gcc and opus library requires libopus and libopusfile.

On Debian / Ubuntu:

sudo apt install build-essential libopus-dev libopusfile-dev

On CentOS / Fedora:

sudo dnf install gcc opus-devel opusfile-devel

Digging up patching functionality in Gazelle

bazel-gazelle go_repository() source code

def patch(ctx):
    """Implementation of patching an already extracted repository"""
    bash_exe = ctx.os.environ["BAZEL_SH"] if "BAZEL_SH" in ctx.os.environ else "bash"
    for patchfile in ctx.attr.patches:
        command = "{patchtool} {patch_args} < {patchfile}".format(
            patchtool = ctx.attr.patch_tool,
            patchfile = ctx.path(patchfile),
            patch_args = " ".join([
                "'%s'" % arg
                for arg in ctx.attr.patch_args
            ]),
        )
        st = ctx.execute([bash_exe, "-c", command])
        if st.return_code:
            fail("Error applying patch %s:\n%s%s" %
                 (str(patchfile), st.stderr, st.stdout))
    for cmd in ctx.attr.patch_cmds:
        st = ctx.execute([bash_exe, "-c", cmd])
        if st.return_code:
            fail("Error applying patch command %s:\n%s%s" %
                 (cmd, st.stdout, st.stderr))

As we find in the source code, it allows us to apply patches to the go_repository generated rules. Therefore, it is possible for us to still use the gazelle and apply a patch to the generated rule to allow us adding some C/C++ dependencies.

In the first step, after cloning the bazel-cgo-demo, let's git checkout step-1 to follow the guide.

The step-1 is a repository with simple rules_go and gazelle setup. After running the following commands:

bazel run //:gazelle
bazel build //cmd/demo:demo

You will see a lot of errors as following:

ERROR: /root/.cache/bazel/_bazel_root/6ae983b61e8c834296feefc10b130c72/external/in_gopkg_hraban_opus_v2/BUILD.bazel:3:11: GoCompilePkg external/in_gopkg_hraban_opus_v2/opus_v2.a failed: (Exit 1): builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix linux_amd64 -src external/in_gopkg_hraban_opus_v2/decoder.go -src external/in_gopkg_hraban_opus_v2/encoder.go ... (remaining 38 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix linux_amd64 -src external/in_gopkg_hraban_opus_v2/decoder.go -src external/in_gopkg_hraban_opus_v2/encoder.go ... (remaining 38 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
external/in_gopkg_hraban_opus_v2/decoder.go:14:10: fatal error: opus.h: No such file or directory
 #include <opus.h>
          ^~~~~~~~
compilation terminated.
compilepkg: error running subcommand external/go_sdk/pkg/tool/linux_amd64/cgo: exit status 2
Target //cmd/demo:demo failed to build

It means that bazel cannot find the header opus.h. It is installed by the libopus system package and shall be available at /usr/include/opus/opus.h location.

It is very useful to use bazel build -s //cmd/demo:demo to show the actual executed command in the sandbox for easy reproduction.

Note: code is available at bazel-cgo-demo, branch step-1.

Step 2: Declare the System libopus Pre-built Dependency

On the next step, we'll need to declare the system libopus pre-built dependency with bazel's cc_import and cc_library combination.

In the WORKSPACE file in the repository root, we can declare the global system headers and system libraries using new_local_repository to allow system headers and libraries to be seen by bazel.

# in WORKSPACE file
new_local_repository(
    name = "libopus",
    build_file = "@//thirdparty/opus:BUILD.opus",
    path = "/usr",
)

Note the build_file flag for new_local_repository(), it is a file specified about how to build the repository. The symbol @ without any package name specifies the current repository, the repository that contains the current WORKSPACE file, aka. The bazel-cgo-demo repository.

Now, let's create the file BUILD.opus in the path thirdparty/opus/ directory with the following content:

load("@rules_cc//cc:defs.bzl", "cc_library", "cc_import")

# The headers can be added in hdrs param in cc_import() or cc_library().
# However, if cc_library() wrapper is used, it is suggested to add in the cc_library().
# The header files are included in the package manager. For debian related systems, we can use
# $ dpkg -L libopus-dev
# $ dpkg -L libopusfile-dev
# to list the installed files.
cc_library(
    name = "libopus",
    hdrs = [
        "//:include/opus/opus.h",
        "//:include/opus/opus_defines.h",
        "//:include/opus/opus_multistream.h",
        "//:include/opus/opus_projection.h",
        "//:include/opus/opus_types.h",
        "//:include/opus/opusfile.h",
    ],
    strip_include_prefix = "include/opus", # allow the code to #include <opus.h> directly
    deps = [               # the sequence of deps is important because it impacts the sequence of linking
        "//:libopusfile_private",
        "//:libopusurl_private",
        "//:libopus_private",
    ],
    visibility = ["//visibility:public"],
)

# Each cc_import() shall contain only one library.
# If multiple libraries are needed, we must use the cc_library() as a wrapper
# to depend on the required multiple libraries.
cc_import(
    name = "libopus_private",
    static_library = "//:lib/x86_64-linux-gnu/libopus.a",
    shared_library = "//:lib/x86_64-linux-gnu/libopus.so",
    visibility = ["//visibility:private"],
)

cc_import(
    name = "libopusfile_private",
    static_library = "//:lib/libopusfile.a",
    shared_library = "//:lib/libopusfile.so",
    visibility = ["//visibility:private"],
)

cc_import(
    name = "libopusurl_private",
    static_library = "//:lib/libopusurl.a",
    shared_library = "//:lib/libopusurl.so",
    visibility = ["//visibility:private"],
)

Now try to build the libopus repository that we introduced in the WORKSPACE.

bazel build @libopus//:libopus
# INFO: Analyzed target @libopus//:libopus (5 packages loaded, 27 targets configured).
# INFO: Found 1 target...
# Target @libopus//:libopus up-to-date (nothing to build)
# INFO: Elapsed time: 0.464s, Critical Path: 0.00s
# INFO: 1 process: 1 internal.
# INFO: Build completed successfully, 1 total action

It shall succeed and indicates that we have no syntax errors in the BUILD.opus file.

Note: in each directory level in thirdparty/opus we need to add a BUILD.bazel to make it recognized as bazel packages.

Note: code is available at bazel-cgo-demo, branch step-2.

Step 3: Patch the go_repository() Rule for in_gopkg_hraban_opus_v2 to Depend on libopus

In the step 2, we have enabled bazel to see the libopus related headers and libraries. However, we haven't declared that the in_gopkg_hraban_opus_v2 package needs libopus as a dependency.

Review the generated package build file by gazelle

First substep would be review the generated BUILD.bazel by gazelle. How can we find the location of the BUILD.bazel generated for package in_gopkg_hraban_opus_v2 ? Use the following bazel query to output the location of the generated BUILD.bazel:

bazel query @in_gopkg_hraban_opus_v2//:all --output=location

It will show us the locations of each rule:

# /root/.cache/bazel/_bazel_root/6ae983b61e8c834296feefc10b130c72/external/in_gopkg_hraban_opus_v2/BUILD.bazel:26:8: go_test rule @in_gopkg_hraban_opus_v2//:opus_v2_test
# /root/.cache/bazel/_bazel_root/6ae983b61e8c834296feefc10b130c72/external/in_gopkg_hraban_opus_v2/BUILD.bazel:20:6: alias rule @in_gopkg_hraban_opus_v2//:go_default_library
# /root/.cache/bazel/_bazel_root/6ae983b61e8c834296feefc10b130c72/external/in_gopkg_hraban_opus_v2/BUILD.bazel:3:11: go_library rule @in_gopkg_hraban_opus_v2//:opus_v2
# Loading: 1 packages loaded

Note: each external dependency package shall locate in the external folder and it is declared by gazelle by go_repository() in deps.bzl.

Note: the bazel building sanbox in Linux systems shall locate mostly in a path similar to ~/.cache/bazel/_bazel_root/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.

It can be changed with a bazel flag --output_base, i.e. bazel --output_base=/tmp/my-bazel-builds/.

Now copy the generated BUILD.bazel to our thirdparty directory.

cp ~/.cache/bazel/_bazel_root/6ae983b61e8c834296feefc10b130c72/external/in_gopkg_hraban_opus_v2/BUILD.bazel thirdparty/in_gopkg_hraban_opus_v2/in_gopkg_hraban_opus_v2.orig
cp thirdparty/in_gopkg_hraban_opus_v2/in_gopkg_hraban_opus_v2.orig thirdparty/in_gopkg_hraban_opus_v2/in_gopkg_hraban_opus_v2.need

Create the patch

Edit the thirdparty/in_gopkg_hraban_opus_v2/in_gopkg_hraban_opus_v2.need file to the following:

load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")

go_library(
    name = "opus_v2",
    srcs = [
        "callbacks.c",
        "decoder.go",
        "encoder.go",
        "errors.go",
        "opus.go",
        "stream.go",
        "stream_errors.go",
        "streams_map.go",
    ],
    cgo = True,
    cdeps = ["@libopus//:libopus"], # keep
    importpath = "gopkg.in/hraban/opus.v2",
    visibility = ["//visibility:public"],
)

alias(
    name = "go_default_library",
    actual = ":opus_v2",
    visibility = ["//visibility:public"],
)

go_test(
    name = "opus_v2_test",
    srcs = [
        "decoder_test.go",
        "encoder_test.go",
        "opus_test.go",
        "stream_test.go",
        "utils_test.go",
    ],
    data = glob(["testdata/**"]),
    embed = [":opus_v2"],
)

The edited line is the cdeps = ["@libopus//:libopus"], # keep. The comment # keep will tell gazelle not to remove cdeps.

Use the diff command to generate the patch file.

diff -au thirdparty/in_gopkg_hraban_opus_v2.orig thirdparty/in_gopkg_hraban_opus_v2.need > thirdparty/in_gopkg_hraban_opus_v2.patch

Modify the patch file (the first two lines) as the following:

--- BUILD.bazel
+++ BUILD.bazel
@@ -13,6 +13,7 @@
         "streams_map.go",
     ],
     cgo = True,
+    cdeps = ["@libopus//:libopus"], # keep
     importpath = "gopkg.in/hraban/opus.v2",
     visibility = ["//visibility:public"],
 )

Apply the patch in go_repository() rule

Now edit the deps.bzl file to use the patch in the go_repository as following:

load("@bazel_gazelle//:deps.bzl", "go_repository")

def go_dependencies():
    go_repository(
        name = "in_gopkg_hraban_opus_v2",
        importpath = "gopkg.in/hraban/opus.v2",
      	patches = ["//thirdparty/in_gopkg_hraban_opus_v2:in_gopkg_hraban_opus_v2.patch"], # keep
        sum = "h1:sxrRNhZ+cNxxLwPw/vV8gNsz+bbqRQiZHBYBJfpyNoQ=",
        version = "v2.0.0-20201025103112-d779bb1cc5a2",
    )

Re-run the build:

bazel build @in_gopkg_hraban_opus_v2//:all

Now you should be able to see the ogg linking errors.

/usr/bin/ld: external/libopus/lib/libopusfile.a(opusfile.o): in function `op_get_next_page':
(.text+0x21b): undefined reference to `ogg_sync_pageseek'
/usr/bin/ld: (.text+0x26f): undefined reference to `ogg_sync_buffer'
/usr/bin/ld: (.text+0x28b): undefined reference to `ogg_sync_wrote'
/usr/bin/ld: external/libopus/lib/libopusfile.a(opusfile.o): in function `op_fetch_headers_impl':
(.text+0x36a): undefined reference to `ogg_page_serialno'
/usr/bin/ld: (.text+0x374): undefined reference to `ogg_stream_reset_serialno'
/usr/bin/ld: (.text+0x37f): undefined reference to `ogg_stream_pagein'
/usr/bin/ld: (.text+0x38c): undefined reference to `ogg_stream_packetout'
/usr/bin/ld: (.text+0x3e2): undefined reference to `ogg_page_bos'
/usr/bin/ld: (.text+0x40b): undefined reference to `ogg_page_serialno'
/usr/bin/ld: (.text+0x44e): undefined reference to `ogg_page_serialno'
/usr/bin/ld: (.text+0x4be): undefined reference to `ogg_page_serialno'
/usr/bin/ld: (.text+0x4d1): undefined reference to `ogg_page_bos'
/usr/bin/ld: (.text+0x588): undefined reference to `ogg_page_serialno'
/usr/bin/ld: (.text+0x5a9): undefined reference to `ogg_stream_packetout'
/usr/bin/ld: (.text+0x5e1): undefined reference to `ogg_stream_packetout'
/usr/bin/ld: (.text+0x617): undefined reference to `ogg_stream_pagein'
/usr/bin/ld: (.text+0x627): undefined reference to `ogg_stream_pagein'
# ... lines, lines of linking errors ...
collect2: error: ld returned 1 exit status
compilepkg: error running subcommand /usr/bin/gcc: exit status 1
INFO: Elapsed time: 2.118s, Critical Path: 1.07s
INFO: 3 processes: 3 internal.
FAILED: Build did NOT complete successfully

Note: code is available at bazel-cgo-demo, branch step-3.

Step 4: Add ogg Libraries for Linking

For libogg, we only needs the library for linking. Headers are not needed.

Let's do almost the same as how we introduced libopus to bazel.

1. Add a new_local_repository to the WORKSPACE
2. Create BUILD.ogg in thirdparty directory

In the WORKSPACE, let's add:

new_local_repository(
    name = "libogg",
    build_file = "@//thirdparty/ogg:BUILD.ogg",
    path = "/usr",
)

Add the following to the BUILD.ogg:

load("@rules_cc//cc:defs.bzl", "cc_import")

cc_import(
    name = "libogg",
    static_library = "//:lib/x86_64-linux-gnu/libogg.a",
    shared_library = "//:lib/x86_64-linux-gnu/libogg.so",
    visibility = ["//visibility:public"],
)

We can find that there's no more cc_library wrapper for the cc_imports in the BUILD.ogg. It is because we do not need to include the headers of ogg nor having multiple libraries to depend on.

Finally, let's add the libogg dependency to libopus so when linking against libopus, bazel will automatically know that libogg is needed in advance.

# file: thirdparty/opus/BUILD.opus
cc_library(
    name = "libopus",
    # ... hidden lines ...
    deps = [
        "//:libopusfile_private",
        "//:libopusurl_private",
        "//:libopus_private",
        "@libogg//:libogg", # <<<< this is the line added.
    ],
    visibility = ["//visibility:public"],
)
# ... hidden lines ...

Note: the sequence is still important. We need to make sure libogg is linked first, then comes the libopus other libraries.

Let's re-do the build for the package in_gopkg_hraban_opus_v2:

bazel build @in_gopkg_hraban_opus_v2//:all

Now the in_gopkg_hraban_opus_v2 build shall succeed.

Let's try building the demo.

bazel build //cmd/demo:demo

Now the demo shall build successfully too.

Note: code is available at bazel-cgo-demo, branch step-4.

Run Demo

After building successfully, we can now run the demo with the following params:

./bazel-bin/cmd/demo/demo_/demo testdata/sample.opus testdata/sample.wav