Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Bazel config for building CEF binary distribution #3757

Open
magreenblatt opened this issue Jul 27, 2024 · 11 comments
Open

Add Bazel config for building CEF binary distribution #3757

magreenblatt opened this issue Jul 27, 2024 · 11 comments
Labels
Distribution Related to binary distribution enhancement Enhancement request

Comments

@magreenblatt
Copy link
Collaborator

magreenblatt commented Jul 27, 2024

Is your feature request related to a problem? Please describe.
Bazel is a popular alternative to CMake for building applications. We should provide sample configurations for Bazel as part of the CEF binary distribution.

Describe the solution you'd like
Build the CEF sample apps in a binary distribution using Bazel and the default platform toolchain.

Additional context
Bazel default platform toolchains are tested to work for Windows x64, MacOS ARM64 and x64 (cross-compile from ARM64), and Linux x64. They are known to be broken for Windows x86 (cross-compile from x64). Other platforms/architectures are untested and may require custom toolchain definitions (Linux examples here).

@magreenblatt magreenblatt added enhancement Enhancement request Distribution Related to binary distribution labels Jul 27, 2024
@magreenblatt
Copy link
Collaborator Author

magreenblatt commented Jul 27, 2024

Initial Bazel support is considered complete (working) in version 127.1.5+g35f74cc+chromium-127.0.6533.89.

Building with Bazel works as follows:

  1. Install Bazelisk.
  2. Download/extract a CEF binary distribution that includes Bazel support.
  3. Build and run using Bazel:
$ cd /path/to/cef_binary_*
$ bazel build //tests/cefclient 

# On Mac/Linux:
$ bazel run //tests/cefclient

# On Windows:
$ bazel run //tests/cefclient/win:cefclient.exe

Platform-specific instructions are included in the binary distribution README.txt file.

See cef-project for an example Bazel project that utilizes the CEF binary distribution as an external dependency.

Notes:

  • Add -c dbg for a Debug build
  • Add --cpu=darwin_x86_64 for a Mac x64 cross-compile build on ARM64 host
  • Arguments can be passed to the app when using run by adding -- [...] at the end

@magreenblatt
Copy link
Collaborator Author

@magreenblatt
Copy link
Collaborator Author

magreenblatt commented Jul 27, 2024

Remaining work:

  • (DONE) Figure out how to consume the CEF binary distribution (as repository or module) from a third-party Bazel project. Update cef-project to demo this.
  • Windows: Add configuration for USE_ATL.

Related to C++ compiler/toolchain configuration:

  • Determine if additional compiler flags should be added in bazel/[win|mac|linux]/variables.bzl, or if we should leave those up to the toolchain.
  • Consider adding a REPO.bazel file (design doc).
  • Linux: Add per-compiler/version/feature config, maybe as flags in .bazelrc.
  • Linux: Maybe find a a better way to solve rpath configuration so we can remove the patchelf package dependency.
  • Windows: Figure out how PDB files are supposed to work with opt builds (see here).

@magreenblatt
Copy link
Collaborator Author

magreenblatt commented Jul 28, 2024

Figure out how to consume the CEF binary distribution (as repository or module) from a third-party Bazel project.

(For background see https://bazel.build/external/overview)

The http_archive rule allows us to download an archive file, and optionally override the associated BUILD and/or WORKSPACE file. We'll need a convenient way to collect download URLs and sha256 values for all platform builds of a particular release. For example, see #3757 (comment).

magreenblatt added a commit that referenced this issue Jul 29, 2024
Add support for building the CEF binary distribution using Bazel
and the default platform toolchain. Tested to work for Windows
x64, MacOS ARM64 and x64 (cross-compile from ARM64), and
Linux x64. Windows x86 (cross-compile from x64) is known to
be broken, see bazelbuild/bazel#22164.

Includes minor changes to tests directory structure to meet
Bazel build requirements.
magreenblatt added a commit that referenced this issue Jul 31, 2024
Add support for building the CEF binary distribution using Bazel
and the default platform toolchain. Tested to work for Windows
x64, MacOS ARM64 and x64 (cross-compile from ARM64), and
Linux x64. Windows x86 (cross-compile from x64) is known to
be broken, see bazelbuild/bazel#22164.

Includes minor changes to tests directory structure to meet
Bazel build requirements.
@magreenblatt
Copy link
Collaborator Author

We'll need a convenient way to collect download URLs and sha256 values

The Spotify builder now provides <file>.sha256 for all binaries beginning with version 127.1.2.

@magreenblatt
Copy link
Collaborator Author

magreenblatt commented Jul 31, 2024

The http_archive rule allows us to download an archive file, and optionally override the associated BUILD and/or WORKSPACE file.

Unfortunately this approach has a number of limitations. For example:

  • We can download a platform/arch-specific archive as name="cef_<platform>" and create target-specific aliases from @cef to @cef_<platform> using select({}), but we can't create similar aliases for macros or rules (*.bzl files) included in the distribution. Consequently, there is no way to load("@cef//...", ...) and have it redirect to load("@cef_<platform>//...", ...). See Cannot load .bzl via alias label bazelbuild/bazel#10092 for background. This would require us to manually copy/commit the scripts from the distribution into the top-level client repository, or move them to a separate CEF artifact/repository (without platform/arch-specific name).
  • We can't have select({}) evaluate to strings as input to WORKSPACE load-related rules/macros (e.g. for downloading a platform/arch-specific archive as name="cef"). See https://stackoverflow.com/a/71199926 for background.

We will therefore provide a separate script for downloading the CEF binary distribution and writing a version.bzl file, to be run manually before bazel build. The WORKSPACE configuration then becomes:

load(":third_party/cef/version.bzl", "CEF_VERSION", "CEF_PLATFORM")

local_repository(
    name = "cef",
    path = "third_party/cef/cef_binary_{}_{}".format(CEF_VERSION, CEF_PLATFORM),
)

@magreenblatt
Copy link
Collaborator Author

magreenblatt commented Jul 31, 2024

We can download a platform/arch-specific archive as name="cef_<platform>" and create target-specific aliases from @cef to @cef_<platform> using select({})

Here's an example of this approach for future reference:

## `WORKSPACE` contents:

local_repository(
    name = "cef",
    path = "bazel/cef",
)
load("@cef//:repos.bzl", "install_cef_repos")
install_cef_repos()


## `bazel/cef/BUILD.bazel` contents:

# Alias all targets to the platform/arch-specific repo.
alias(
    name = "cef_wrapper",
    actual = select({
        "@//:macos_arm64": "@cef_macosarm64//:cef_wrapper",
        # ...
    }),
)

# ...


## `bazel/cef/repos.bzl` contents:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

CEF_DOWNLOAD_URL = "https://cef-builds.spotifycdn.com/"
CEF_VERSION = "127.1.2+g1cd2424+chromium-127.0.6533.73"

def add_cef_repo(
        platform,
        sha256,
        version = CEF_VERSION,
        url = CEF_DOWNLOAD_URL):
    file_name = "cef_binary_{}_{}".format(version, platform)
    http_archive(
        name = "cef_{0}".format(platform),
        sha256 = sha256,
        strip_prefix = file_name,
        url = "{}{}".format(url, file_name),
    )

def install_cef_repos():
    """
      Set up definitions of all repos for each supported platform/arch.
    """
    add_cef_repo(platform = "macosarm64", sha256 = "d2e9ead...")
    # ...

@magreenblatt
Copy link
Collaborator Author

We will therefore provide a separate script for downloading the CEF binary distribution

There's also a middle ground where the script writes the platform but Bazel still performs the download:

## `cef_config.bzl` contents (written by script):

CEF_PLATFORM = "macosarm64"


## `WORKSPACE` contents:

load("//:cef_config.bzl", "CEF_PLATFORM")
load("//:bazel/cef_loader.bzl", "load_cef_repo")
load_cef_repo(CEF_PLATFORM)


## `bazel/cef_loader.bzl` contents:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

CEF_DOWNLOAD_URL = "https://cef-builds.spotifycdn.com/"
CEF_VERSION = "127.1.2+g1cd2424+chromium-127.0.6533.73"
CEF_FILE_SHA256 = {
    # Map of each platform name to sha256 hash.
    "macosarm64": "d2e9ead6...",
    # ...
}

def load_cef_repo(platform
                  name = "cef",
                  version = CEF_VERSION,
                  url = CEF_DOWNLOAD_URL):
    """
    Download, extract and load a CEF binary distribution.
    """
    file_name = "cef_binary_{}_{}".format(version, platform)

    http_archive(
        name = name,
        sha256 = CEF_FILE_SHA256[platform],
        strip_prefix = file_name,
        url = "{}{}.tar.bz2".format(url, file_name),
    )

magreenblatt added a commit that referenced this issue Aug 1, 2024
Unqualified target labels don't resolve correctly when the
disribution is loaded as an external repo. Mapping, if necessary,
can be performed using the `repo_mapping` parameter to
http_archive() or local_repository().
magreenblatt added a commit that referenced this issue Aug 1, 2024
Unqualified target labels don't resolve correctly when the
disribution is loaded as an external repo. Mapping, if necessary,
can be performed using the `repo_mapping` parameter to
http_archive() or local_repository().
magreenblatt added a commit that referenced this issue Aug 1, 2024
Unqualified target labels don't resolve correctly when the
disribution is loaded as an external repo. Mapping, if necessary,
can be performed using the `repo_mapping` parameter to
http_archive() or local_repository().
magreenblatt added a commit that referenced this issue Aug 2, 2024
- Headers that are included by .rc files need to be supplied to
  `declare_exe` via the `resources_deps` attribute (passed as the `deps
  attribute to `compile_rc`). The headers must be part of a cc_library
  target via either the `hdrs` or `srcs` attribute.
- File paths for CEF resources are prefixed with "external/<repo>"
  when CEF is loaded as an external repo. Update `copy_filegroups` to
  work with these paths.
magreenblatt added a commit that referenced this issue Aug 2, 2024
- Headers that are included by .rc files need to be supplied to
  `declare_exe` via the `resources_deps` attribute (passed as the `deps
  attribute to `compile_rc`). The headers must be part of a cc_library
  target via either the `hdrs` or `srcs` attribute.
- File paths for CEF resources are prefixed with "external/<repo>"
  when CEF is loaded as an external repo. Update `copy_filegroups` to
  work with these paths.
magreenblatt added a commit that referenced this issue Aug 2, 2024
- Headers that are included by .rc files need to be supplied to
  `declare_exe` via the `resources_deps` attribute (passed as the `deps
  attribute to `compile_rc`). The headers must be part of a cc_library
  target via either the `hdrs` or `srcs` attribute.
- File paths for CEF resources are prefixed with "external/<repo>"
  when CEF is loaded as an external repo. Update `copy_filegroups` to
  work with these paths.
magreenblatt added a commit to chromiumembedded/cef-project that referenced this issue Aug 3, 2024
Bazel can be used to build CEF-based applications. CEF support for
Bazel is considered experimental. For current development status
see chromiumembedded/cef#3757 . For
updated build and usage instructions see the README.md.
@magreenblatt
Copy link
Collaborator Author

Bazel build support has been added to cef-project, see instructions.

magreenblatt added a commit that referenced this issue Aug 5, 2024
Fixes "file not found with <angled> include" errors.
magreenblatt added a commit that referenced this issue Aug 5, 2024
Fixes "file not found with <angled> include" errors.
magreenblatt added a commit that referenced this issue Aug 5, 2024
Fixes "file not found with <angled> include" errors.
magreenblatt added a commit that referenced this issue Aug 6, 2024
- Add `declare_[cc|objc]_library` macros to configure common `copts`
  and `local_defines` (where supported) on `[cc|objc]_library`
  targets. This limits the scope of defines to the specific target
  without inheritance by dependent targets.
- `objc_library` does not currently support `local_defines` so we
  use `copts` instead of MacOS.
- Use `**kwargs` to pass all other arguments to the actual cc
  target declaration.
magreenblatt added a commit that referenced this issue Aug 6, 2024
- Add `declare_[cc|objc]_library` macros to configure common `copts`
  and `local_defines` (where supported) on `[cc|objc]_library`
  targets. This limits the scope of defines to the specific target
  without inheritance by dependent targets.
- `objc_library` does not currently support `local_defines` so we
  use `copts` instead of MacOS.
- Use `**kwargs` to pass all other arguments to the actual cc
  target declaration.
magreenblatt added a commit that referenced this issue Aug 6, 2024
- Add `declare_[cc|objc]_library` macros to configure common `copts`
  and `local_defines` (where supported) on `[cc|objc]_library`
  targets. This limits the scope of defines to the specific target
  without inheritance by dependent targets.
- `objc_library` does not currently support `local_defines` so we
  use `copts` instead of MacOS.
- Use `**kwargs` to pass all other arguments to the actual cc
  target declaration.
magreenblatt added a commit that referenced this issue Aug 9, 2024
Copy the CEF framework into the app bundle but do not link it. See
https://groups.google.com/g/cef-announce/c/Fith0A3kWtw/m/6ds_mJVMCQAJ
for background.

Use `**kwargs` to pass all other arguments to the actual
`macos_application` target declaration.
magreenblatt added a commit that referenced this issue Aug 9, 2024
Copy the CEF framework into the app bundle but do not link it. See
https://groups.google.com/g/cef-announce/c/Fith0A3kWtw/m/6ds_mJVMCQAJ
for background.

Use `**kwargs` to pass all other arguments to the actual
`macos_application` target declaration.
magreenblatt added a commit that referenced this issue Aug 9, 2024
Copy the CEF framework into the app bundle but do not link it. See
https://groups.google.com/g/cef-announce/c/Fith0A3kWtw/m/6ds_mJVMCQAJ
for background.

Use `**kwargs` to pass all other arguments to the actual
`macos_application` target declaration.
@magreenblatt
Copy link
Collaborator Author

The Bazel fastbuild configuration is a bit of an oddity. From the docs:

fastbuild means build as fast as possible: generate minimal debugging information (-gmlt
-Wl,-S), and don’t optimize. This is the default. Note: -DNDEBUG will not be set.

CEF provides Debug and Release configs. CEF currently treats fastbuild the same as opt (Release) because, on Windows, fastbuild sets /MT (non-Debug runtime library) due to --features=static_link_msvcrt (code here). However, the Release config also defines NDEBUG which conflicts with the above fastbuild docs.

We therefore recommend that users avoid fastbuild when integrating CEF into existing Bazel configs. Instead, explicitly select the dbg (Debug) or opt (Release) config.

@magreenblatt
Copy link
Collaborator Author

We may revisit the cc_import shared_library config that currently copies libcef.dll as a transitive dependency. This seems to cause issues with complex dependency graphs, and it may be better to instead explicitly link libcef.lib and explicitly copy libcef.dll.

magreenblatt added a commit that referenced this issue Nov 5, 2024
Using cc_import + interface_library/shared_library to link libcef.lib causes
libcef.dll to be copied as a transitive dependency, leading to issues with
complex Bazel configs. Instead, we explicitly link libcef.lib in the binary
target (cc_binary + linkopts/additional_linker_inputs) and explicitly copy
libcef.dll to the target directory.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Distribution Related to binary distribution enhancement Enhancement request
Projects
None yet
Development

No branches or pull requests

1 participant