Skip to content

Latest commit

 

History

History
69 lines (55 loc) · 4.72 KB

README.md

File metadata and controls

69 lines (55 loc) · 4.72 KB
Raw

OddEyeCam is a practical inside-out mobile device tracking system to support body-centric peephole interaction. OddEyeCam tracks a user's body using a WFoV RGB camera and obtains precise depth information using an NFoV depth camera from a smartphone close to the body. This project was published in UIST 2020.
More info about this work can be found in:
[Project Video] [Paper]


Some usage examples of OddEyeCam

How to use

Step 0: Camera Calibration

OddEyeCam combines WFoV RGB camera and NFoV depth camera. For this fusion process, we need to calibrate the cameras. For more details, please check the calibration folder. Before you calibrate two cameras, be sure to fix two cameras to each other. We provide 3D printing models that we used for prototyping. Please see 3D_printing folder.

Step 1: Install Requirements

  • Pose Estimation Module (OpenPose)
    We used OpenPose for body tracking. Please compile OpenPose and place model, library files in core/thirdparty/openpose/ folder. If you are using Python3.6 on Windows, we provide these files to save your time. Please download them here.
  • Android App
    Our system needs an Android device because OddEyeCam 1) receives accelerometer data from the device and 2) visualizes the result on the app. Please download OddEyeCam Android application package (APK). Or, you can also modify and build this app. For more details, please check the android folder.

Step2: Run OddEyeCam on a Desktop

All files you need to run OddEyeCam on the desktop are in the core and util folders. oddeyecam_demo.py uses these essential files to show demo.

python oddeyecam_demo.py

After you run oddeyecam_demo.py, you will see the line "[Server] Server is opened!". The program is waiting for the connection of an Android device.


After you run oddeyecam_demo.py. System is waiting for the connection of an Android device.

Step3: Connect with Android Device

Please press the button on the OddEyeCam Android app. After the connection, you will see on the command, and then camera windows will appear.


After the connection. Camera views are shown on the desktop's screen (left). Position data are shown on the mobile phone display (right).

If you want to stop camera streaming, then press the esc button on the camera windows.

Step4: Demo Applications

If you want to use example application, please press button. Then you will see six applications:

  • Drag and Drop between Apps
  • Body-Centric Folder
  • Large Image Viewer
  • One-hand Tagging
  • Getting Direction
  • Marking Menus

Why 'OddEye'Cam?

The name OddEyeCam was inspired by an odd-eyed cat. An odd-eyed cat is a cat with one blue eye and one eye either green, yellow, or brown. Analogous to the cat with two eyes of different colors, we combine the following two types of cameras with different characteristics: an RGB camera with a wide field of view (WFoV) and a depth camera with a narrow field of view (NFoV). OddEyeCam finds body keypoints from a WFoV RGB image and uses par- tial depth information from a depth camera to track the body and estimate the device position relative to the body.

Btw, I love cats!