Skip to content

Documenting how platform builds work, and will work #693

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ ALL=1 make -C opt clean build

Note: in order to use the PyTorch backend on Linux, at least `gcc 4.9.2` is required.

[See this document](opt/build/README.md) for building AI dependencies at platforms.

### Running the server

You will need a redis-server version 5.0.7 or greater. This should be
Expand Down
9 changes: 9 additions & 0 deletions opt/build/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# RedisAI dependency builds

Platform dependency build systems are located in this folder. Dependencies are to be pre-built, and published to S3. To do so, they rely *(ultimately)* on running **make build publish** in a given directory. The goal is for this to be true on all target platforms (x86_64, arm64), though at this time it's only true for: tensorflowlite.

## Background

Items are built in docker images, for the target platform whenever possible. If needed (i.e a future planned MacOS build) items are built on the dedicated hardware. There are design wrinkles to each build. Though the ideal is to build a base docker (see the [automata repository](https://github.com/redislabsmodules/automata). That base docker is then used as the base build system injector for the dependency itself. A docker image is built from the base docker, accepting externalized variables such as the dependency version. Compilation of external requirements takes place in a build file, mounted inside the docker image.

Ideally a per-platform Docker file (i.e Dockerfile.x64, Dockerfile.arm) will exist in the underlying folder, assuming building within a docker is tenable.
7 changes: 7 additions & 0 deletions opt/build/tflite/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# tensorflowlite (tflite)

### arm64

The arm build of tflite currently occurs on **jetson arm devices** only, as portions of the root filesystem of the Jetson device are mounted during the build. Given the symlinks that exist on the device between things in /usr/lib to /etc/alternatives, and in turn to /usr/local/cuda, which is itself a symlink to /usr/local/cuda-10.2, this is the current philosophy.

WThe *build_arm* target in the [Makefile](Makefile) describes the process in detail. The code to build the base docker build image can be found in the [automata repository](https://github.com/RedisLabsModules/automata/tree/master/dockers/buildsystem/bazelbuilder). The *bazelbuilder* image is published to the [redisfab dockerhub repositories](https://hub.docker.com/r/redisfab/).