Cloud Deployment

One of the most common usages of IBeam would be to deploy it as a Docker container on a remote server, using cloud providers such as AWS, GCP or DigitalOcean.

The following is a simple guide that should help you get started with the remote deployment.

Note: some users got in touch with me to help them get IBeam deployed on the cloud, which I now do for a fee. If you'd rather have me set this all up for you, get in touch at [email protected]. Naturally, if you don't want to hire me and would like to do this yourself, I'm still happy to answer your questions in the Issues.

Digital Ocean

This guide was contributed by @kls06541 👏

Create a server

Create a new DigitalOcean droplet with the following settings.

1. Region: New York (or change to whichever you find most suitable)
2. Choose an Image -> Marketplace: Docker
3. Droplet Type: Basic
4. CPU Options: Regular, 2GB RAM (other things shouldn’t matter)
5. Choose Authentication Method: Password (I tried working with SSH keys, but couldn’t get these to work properly. The setup with ‘root’ is straightforward enough).
6. Hostname: Type in a meaningful host name, for example ‘ibeam01’

Note: Setting up a server with 'root' permissions is not considered a safe practice. This document should serve as a basic quickstart guide, and it is encouraged each user looks into minimising security risks.

Set up the server

SSH into the server.

If you succeed with private SSH keys then great, otherwise go with ‘Access Console -> Launch Droplet Console’.

Run following as root on the server:

1. nano setup.sh
2. Copy-paste the contents of setup.sh found in the Appendix A at the bottom of this document.
3. You save the file by pressing CTRL+X, then typing ‘Y’ and hitting ENTER.
4. . setup.sh [YOUR IBKR ACCOUNT NAME] [YOUR IBKR ACCOUNT PASSWORD]

You should now have the following directory structure:

root
└───ibeam_files
    ├───inputs_directory
    │   └───conf.yaml
    ├───outputs
    └───env.list

Double check the configuration files:

1. /root/ibeam_files/env.list - Check with the current version of IBeam which variables may need to be set. Check that your credentials are correct.
2. /root/ibeam_files/inputs_directory/conf.yaml - Ensure that the IPs listed in the 'allow' are corresponding with the IPs from which you’ll be making requests from.

Start IBeam

Make sure you’ve completed the IBeam setup above and that all files are present in the /root/ibeam_files directory.

Using startup.sh

The setup.sh script should create a startup.sh wrapper script which facilitates starting IBeam correctly configured.

To run it call (note the dot at the start):

. starter.sh

startup.sh accepts a number of arguments (defaults in parenthesis):

• -e –env (env.list): environment variables file
• -t –tag (latest): IBeam image tag to be used
• -i –ibeam_files (/root/ibeam_files): directory of IBeam files
• -n –name (ibeam): name of the IBeam container

For example:

. starter.sh -t 0.4.4 -e env_secondary.list

Will start IBeam with tag ‘0.4.4’ using ‘env_secondary.list’ environment file.

Manually

Alternatively, you can start IBeam directly using the following Docker command:

docker run -d --env-file /root/ibeam_files/env.list --name ibeam -p 5000:5000 -v /root/ibeam_files/inputs_directory/:/srv/inputs -v /root/ibeam_files/outputs:/srv/outputs:rw voyz/ibeam:latest

Logs

To observe the logs of IBeam, open a new Terminal and call:

docker logs -f ibeam

Verify connection

To verify whether the IBeam login worked, run:

curl -X GET "https://localhost:5000/v1/api/tickle" -k

In the output of that command, you should see a value that says authenticated:true - if you can see that, it's all good. If it's false or you cannot see authenticated, or the command fails, then you're not logged in and something failed.

Restart IBeam

To restart IBeam (not the server) run:

docker stop ibeam
docker start ibeam

Restart Server

To restart the DigitalOcean server, run:

reboot

Verify IBeam is started by running:

docker ps

You should see ‘Up X minutes’ in the STATUS column

Then, wait for a few minutes and call the following to verify the successful connection:

curl -X GET "https://localhost:5000/v1/api/tickle" -k

Virtual Display

Optionally, you may create a virtual display that should allow you to remotely connect to the server using a RDP protocol.

To do so, run:

1. sudo apt-get update
2. sudo apt-get install -y xrdp
3. sudo ufw allow 3389/tcp
4. sudo apt-get install -y xfce4
5. echo xfce4-session > ~/.xsession
6. sudo systemctl enable xrdp
7. sudo systemctl start xrdp

Now you should be able to use Remote Desktop Connection, providing the IP address of the new server and root credentials.

Gnome system monitor

Gnome system monitor can help with monitoring system resources:

sudo apt-get install gnome-system-monitor

It may be a good idea to keep the system monitor open on the server and make sure it doesn’t run out of memory.

Web browser

Additionally install a web browser if needed, such as Firefox:

sudo apt install firefox

Appendix A

setup.sh:

#!/bin/bash

if [[ $# -ne 2 ]]; then
  echo "Usage: $0 <IBKR account name> <IBKR account password>"
  exit 1
fi

echo Setup starting



# create IBeam configuration directories

echo Creating IBeam configuration directories
mkdir /root/ibeam_files
mkdir /root/ibeam_files/outputs
chmod 777 /root/ibeam_files/outputs
mkdir /root/ibeam_files/inputs_directory


# create environment variable files

echo Creating environment vars file at /root/ibeam_files/env.list
cat <<EOF >  /root/ibeam_files/env.list
IBEAM_ACCOUNT=$1
IBEAM_PASSWORD=$2
IBEAM_OUTPUTS_DIR=/srv/outputs
EOF



# create conf.yaml

echo Creating Gateway configuration file at /root/ibeam_files/inputs_directory/conf.yaml
cat <<EOF > /root/ibeam_files/inputs_directory/conf.yaml
ip2loc: "US"
proxyRemoteSsl: true
proxyRemoteHost: "https://api.ibkr.com"
listenPort: 5000
listenSsl: true
ccp: false
svcEnvironment: "v1"
sslCert: "vertx.jks"
sslPwd: "mywebapi"
authDelay: 3000
portalBaseURL: ""
serverOptions:
  blockedThreadCheckInterval: 1000000
  eventLoopPoolSize: 20
  workerPoolSize: 20
  maxWorkerExecuteTime: 100
  internalBlockingPoolSize: 20
cors:
  origin.allowed: "*"
  allowCredentials: false
webApps:
  - name: "demo"
    index: "index.html"
ips:
  allow:
    - 10.*
    - 192.*
    - 131.216.*
    - 127.0.0.1
    - 0.0.0.0
    - 172.17.0.*
  deny:
    - 212.90.324.10
EOF



# add IBeam autostart service

echo Creating autostart service file at /etc/systemd/system/ibeam_autostart.service 
cat <<EOF > /etc/systemd/system/ibeam_autostart.service
[Unit]
Description=IBeam container
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker start -a ibeam
ExecStop=/usr/bin/docker stop -t 2 ibeam

[Install]
WantedBy=default.target
EOF

sudo systemctl enable ibeam_autostart.service



# Create the starter file

echo Creating IBeam starter file at starter.sh
cat <<EOF > starter.sh
#!/bin/bash

ENV="env.list"
TAG="latest"
IBEAM_FILES="/root/ibeam_files"
NAME="ibeam"

while [[ \$# -gt 0 ]]; do
  key="\$1"
  case \$key in
    -e|--env)
      ENV="\$2"
      shift # past argument
      shift # past value
      ;;
    -t|--tag)
      TAG="\$2"
      shift # past argument
      shift # past value
      ;;
    -i|--ibeam_files)
      IBEAM_FILES="\$2"
      shift # past argument
      shift # past value
      ;;
    -n|--name)
      NAME="\$2"
      shift # past argument
      shift # past value
      ;;
    *)
      # unknown option
      echo "Unknown option: \$1"
      ;;
  esac
done

echo "IBeam environment file: $ENV"
echo "IBeam tag: $TAG"
echo "IBeam container name: $NAME"
echo "IBeam files directory: $IBEAM_FILES"

echo "Attempting to remove existing ibeam container"
docker rm -f ibeam

echo "Starting IBeam..."

docker run -d --env-file "\$IBEAM_FILES/\$ENV" --name "\$NAME" -p 5000:5000 -v "\$IBEAM_FILES/inputs_directory/:/srv/inputs" -v "\$IBEAM_FILES/outputs:/srv/outputs:rw" voyz/ibeam:\$TAG
EOF

MY_IP=$(ip -o route get to 8.8.8.8 | sed -n 's/.*src \([0-9.]\+\).*/\1/p')
cat << EOF



Setup completed!

Next steps:
1. Remotely connect to this server using its IP address: $MY_IP
2. Verify the configuration files in '/root/ibeam_files' directory
3. Start IBeam using the 'starter.sh' file
EOF