Asymmetrical Sample Apartment (#25)

* Fix Browser to send and recieve Mouse and Keyboard data

* Add Camera Controls for Better Asym Control

* Add Null Check

* Update Tracked Alias in Bow Sample

* Shrink Display Res and Make Input Non Static

* Shirk Default Stream Size

* Add Apartment Sample and Update Change and Project files

* Adding Unity Versions to README & Cloud Docs

* Add Apartment Sample to README

Author: vmohan7

CHANGELOG.md

@@ -4,6 +4,24 @@ All notable changes to com.unity.renderstreaming package will be documented in t
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+## [0.5.1] - 2021-09-20
+
+### Added
+
+- Added a **NEW** Sample : Asymmetrical Apartment for testing asymmetrical multiplayer
+- Add CameraControls.cs to allow for the Camera to be controlled in 2D experiences
+- Added Supported Versions to README : https://github.com/FusedVR/VRStreaming/issues/24
+- Added enableKeyboardTouchControls to VRInputManager to control whether to use Camera Controls
+
+### Changed
+
+- Fixed HDRPVR Sample to reference correct Render Streaming Prefab
+- Fixed Mouse & Camera Input from the Web Client to be correctly sent to Unity SDK using RemoteInput class
+- Removed WebXR controls from client and instead send input to be handled by server 
+- Fixed Overriding Prefab Positional & Rotational Data : https://github.com/FusedVR/VRStreaming/issues/19
+- Rotation from headset is applied in local rotation instead of global rotation
+
+
 ## [0.5.0] - 2021-09-13
 
 ### Added

Images~/apartment.png

@@ -20,6 +20,12 @@ For an overview of this package, please refer to this video tutorial on the Fuse
 5. On the Browser, click "**Connect to Cloud VR Streaming Server**"
 6. Once you see the video feed, click "**Enter Virtual Reality**"
 
+# Unity Versions
+
+Tested & Verfied with Unity 2019.4 LTS. Limited testing has been done with Unity 2020.3 LTS and seems to be working as intended there as well. 
+
+Please also refer to [Unity Render Streaming](https://github.com/Unity-Technologies/UnityRenderStreaming) for their list of supported Unity versions.  
+
 # Configurations
 
 On the **Render Streaming Service** Game Object, there are a few configuration options to make it easier to customize your Render Streaming Game.
@@ -40,7 +46,10 @@ At this time, the WebXR Client sends the resolution required to render per eye b
 
 # Cloud Support
 
-Limited testing and Proof of Concepts have been done with deploying a standalone version of the Unity Render Streaming app onto a Cloud or Edge Service provider via either a Windows or Ubuntu Virtual Machine. More testing is required to be done to enable this support. 
+This project has been tested to work on AWS EC2 leveraging Nvidia's Gaming AMIs. See the following videos for documentation on how to setup your Unity VR Streaming project and get it deployed on the Cloud.
+
+- Windows Server with Tesla T4 GPU Accelerated Graphics : https://youtu.be/UFsbQ8YlboU
+- Ubuntu 18.04 with Tesla T4 GPU Accelerated Graphics : https://youtu.be/3p0tzqD-s-c
 
 # GPU Recommendations
 
@@ -50,14 +59,14 @@ It is strongly recommended to utlize a NVIDIA GPU as these GPUs support Hardware
 
 Controller Input is captured via the [A-Frame Tracked-Controls component](https://aframe.io/docs/1.2.0/components/tracked-controls.html) and then sent over the data channel to the Unity SDK. This data protocol is adapted from the Unity Broadcast system, which was also capable of sending Keyboard, Mouse, Touch, and Gamepad data back to Unity via the RemoteInput.cs script in Unity. As such, VR data is sent from the client to Unity as Data Array buffers, defined in bytes. 
 
-The first byte of the data Array Byffer refers to the input mode to determine how to parse the data as an id. The following IDs were reservered for Web Input by the Unity Render Streaming system. 
+The first byte of the data Array Byffer refers to the input mode to determine how to parse the data as an id. The following IDs were reservered for Web Input by the Unity Render Streaming system and integrated with their Remote Input class. 
 
 - ID 0 = Keyboard
 - ID 1 = Mouse
 - ID 2 = MouseWheel
 - ID 3 = Touch
 - ID 4 = UI (legacy)
-- ID 5 = Gamepad
+- ID 5 = Gamepad (currently not implemented)
 
 ID 6 is used for all WebXR Input specific to VR. Within VR Input, we specify 3 different modes for sending data, which are:
 - ID 0 = Positional and Rotational Data of the Headset and Hands
@@ -99,10 +108,17 @@ The first one uses the Unity HDRP Built In Template to show case streaming a sce
 
 ![High Definition Render Pipeline](https://raw.githubusercontent.com/FusedVR/VRStreaming/master/Images~/hdrpvr.png)
 
-The second provided sample shows integration with [VRTK](https://www.vrtk.io/) to showcase how to utilize the input that is streamed from the WebXR client into Unity. Using this input, we can build an Archery Sandbox. Simmply pick up the bow and then grab arrows from behind your back to begin shooting. Please note that to you use this sample you will need to manually also import the following [VRTK Tilia Packages](https://www.vrtk.io/tilia.html):
+The second provided sample shows integration with [VRTK](https://www.vrtk.io/) to showcase how to utilize the input that is streamed from the WebXR client into Unity. Using this input, we can build an Archery Sandbox. Simply pick up the bow and then grab arrows from behind your back to begin shooting. Please note that to you use this sample you will need to manually also import the following [VRTK Tilia Packages](https://www.vrtk.io/tilia.html):
 
 - **io.extendreality.tilia.camerarigs.trackedalias.unity**
 - **io.extendreality.tilia.interactions.interactables.unity**
 - **io.extendreality.tilia.interactions.snapzone.unity**
 
 ![VRTK Archery](https://raw.githubusercontent.com/FusedVR/VRStreaming/master/Images~/archery-sample.png)
+
+The third provided sample shows an example of creating a asymmetrical multiplayer experience, where up to 4 users can enter the application either in VR or in 2D as a laptop or mobile user to view an apartment. VR users will be able to teleport around using the enabled [VRTK](https://www.vrtk.io/) integration. Please note that to you use this sample you will need to manually also import the following [VRTK Tilia Packages](https://www.vrtk.io/tilia.html):
+
+- **io.extendreality.tilia.locomotors.teleporter.unity**
+- **io.extendreality.tilia.indicators.objectpointers.unity**
+
+![Apartment](https://raw.githubusercontent.com/FusedVR/VRStreaming/master/Images~/apartment.png)

README.md

@@ -2,7 +2,8 @@
     "name": "FusedVR.Streaming",
     "references": [
         "GUID:40a5acf76f04c4c8ebb69605e4b0d5c7",
-        "GUID:f12aafacab75a87499e7e45c873ffab8"
+        "GUID:f12aafacab75a87499e7e45c873ffab8",
+        "GUID:75469ad4d38634e559750d17036d5f7c"
     ],
     "includePlatforms": [],
     "excludePlatforms": [],

Runtime/FusedVR.Streaming.asmdef

@@ -72,4 +72,4 @@ MonoBehaviour:
   gameID: 
   maxConnections: 1
   playerPrefabs:
-  - {fileID: 3432969078835647384, guid: 234ee74f5f05b9541850ec55278bb3f5, type: 3}
+  - {fileID: 1305806924054047947, guid: 234ee74f5f05b9541850ec55278bb3f5, type: 3}

Runtime/Prefabs/Render Streaming Service.prefab

@@ -87,7 +87,7 @@ MonoBehaviour:
   m_Script: {fileID: 11500000, guid: 88dc1ef0a0e7d1b4a897f095b852ab38, type: 3}
   m_Name: 
   m_EditorClassIdentifier: 
-  streamingSize: {x: 1350, y: 1500}
+  streamingSize: {x: 972, y: 1080}
   cameras:
   - {fileID: 7977754495293142443}
   depth: 0
@@ -154,7 +154,7 @@ GameObject:
   serializedVersion: 6
   m_Component:
   - component: {fileID: 7977754496362167397}
-  - component: {fileID: 3432969078835647384}
+  - component: {fileID: 1305806924054047947}
   - component: {fileID: 7977754496362167396}
   - component: {fileID: 8526365356649225057}
   m_Layer: 0
@@ -171,7 +171,7 @@ Transform:
   m_PrefabInstance: {fileID: 0}
   m_PrefabAsset: {fileID: 0}
   m_GameObject: {fileID: 7977754496362167402}
-  m_LocalRotation: {x: -0, y: -0, z: -0, w: 1}
+  m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
   m_LocalPosition: {x: 0, y: 0, z: 0}
   m_LocalScale: {x: 1, y: 1, z: 1}
   m_Children:
@@ -181,7 +181,7 @@ Transform:
   m_Father: {fileID: 0}
   m_RootOrder: 0
   m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
---- !u!114 &3432969078835647384
+--- !u!114 &1305806924054047947
 MonoBehaviour:
   m_ObjectHideFlags: 0
   m_CorrespondingSourceObject: {fileID: 0}
@@ -212,6 +212,7 @@ MonoBehaviour:
   m_EditorClassIdentifier: 
   local: 0
   label: 
+  enableKeyboardTouchControls: 1
   VRCameras: {fileID: 0}
   VRPoseEvent:
     m_PersistentCalls:
@@ -401,7 +402,7 @@ MonoBehaviour:
   m_Script: {fileID: 11500000, guid: 88dc1ef0a0e7d1b4a897f095b852ab38, type: 3}
   m_Name: 
   m_EditorClassIdentifier: 
-  streamingSize: {x: 1350, y: 1500}
+  streamingSize: {x: 972, y: 1080}
   cameras:
   - {fileID: 7977754496754579055}
   depth: 0

Runtime/Prefabs/WebXR Streamer.prefab

@@ -28,7 +28,7 @@ public void ApplyData(VRInputManager.Source id, Vector3 position, Quaternion rot
             if (index == id) //check if the data is from the correct source
             {
                 transform.localPosition = new Vector3(position.x, position.y, -position.z); //note that z data is reversed on WebXR
-                transform.rotation = rotation; //apply rotation - note coordinate system change was already applied
+                transform.localRotation = rotation; //apply rotation - note coordinate system change was already applied
             }
         }
     }

Runtime/Scripts/ApplyVRData.cs

@@ -0,0 +1,149 @@
+using UnityEngine;
+using UnityEngine.InputSystem;
+
+namespace FusedVR.VRStreaming {
+
+    /// <summary>
+    /// Fly Camera Writen by Windexglow 11-13-10. And Modified for usage with VR Render Streaming
+    /// Original Source : https://gist.github.com/gunderson/d7f096bd07874f31671306318019d996
+    /// </summary>
+    public class CameraControls : MonoBehaviour {
+
+        #region Public Properties
+        public float mainSpeed = 10.0f;   // Regular speed
+        public float shiftAdd = 25.0f;   // Amount to accelerate when shift is pressed
+        public float maxShift = 100.0f;  // Maximum speed when holding shift
+        public float camSens = 0.15f;   // Mouse sensitivity
+        #endregion
+
+        #region Private Properties
+        private Mouse myMouse; // This Controllers Mouse Input
+        private Touchscreen myTouch; // This Controllers Touch Input
+        private Keyboard myKeyboard; // This Controllers Keyboard Input
+
+        private Pointer lastPointer; // For Mouse / Touch Input, what pointer was used last
+        private Vector2 lastPointerPosition = Vector2.zero; // The last position of the pointer
+        private bool isFirstPress = true; // whether the pointer has been previously pressed in the prior frame
+        private float totalRun = 1.0f; // speed control for the camera
+        #endregion
+
+        #region Public Methods
+        /// <summary>
+        /// Add a Unity Input Device (Mouse, Touch, Keyboard) to this controller to be used for Camera Controls
+        /// Used in the VR Input Manager to assign devices
+        /// </summary>
+        public void AddDevice(InputDevice device) {
+            switch (device) {
+                case Mouse mouse:
+                    myMouse = mouse;
+                    lastPointer = myMouse;
+                    break;
+                case Keyboard keyboard:
+                    myKeyboard = keyboard;
+                    break;
+                case Touchscreen touch:
+                    myTouch = touch;
+                    lastPointer = myTouch;
+                    break;
+            }
+        }
+
+        /// <summary>
+        /// Reset and Remove all Devices that have prior assigned
+        /// Used in the VR Input Manager to remove devices for the camera control on disconnect
+        /// </summary>
+        public void RemoveDevices() {
+            myMouse = null;
+            myKeyboard = null;
+            myTouch = null;
+            lastPointer = null;
+        }
+        #endregion
+
+        #region Camera Controls
+        void Update() {
+            // Mouse camera angle
+            if (myMouse != null && myMouse.leftButton.isPressed) {
+                if (isFirstPress) {
+                    lastPointerPosition = myMouse.position.ReadValue();
+                    isFirstPress = false;
+                }
+
+                UpdateMouseTouch(myMouse.position.ReadValue());
+            } else if (myTouch != null && myTouch.primaryTouch.press.isPressed ) {
+                if (isFirstPress) {
+                    lastPointerPosition = myTouch.primaryTouch.position.ReadValue();
+                    isFirstPress = false;
+                }
+
+                UpdateMouseTouch(myTouch.primaryTouch.position.ReadValue());
+            } else {
+                isFirstPress = true;
+            }
+
+            // Keyboard commands
+            Vector3 p = GetBaseInput();
+
+            if ( myKeyboard != null && myKeyboard.leftShiftKey.isPressed ) {
+                totalRun += Time.deltaTime;
+                p *= totalRun * shiftAdd;
+                p.x = Mathf.Clamp(p.x, -maxShift, maxShift);
+                p.y = Mathf.Clamp(p.y, -maxShift, maxShift);
+                p.z = Mathf.Clamp(p.z, -maxShift, maxShift);
+            } else {
+                totalRun = Mathf.Clamp(totalRun * 0.5f, 1f, 1000f);
+                p *= mainSpeed;
+            }
+
+            p *= Time.deltaTime;
+            transform.Translate(p);
+        }
+
+        // Updates the Rotation based on the Mouse 2D Position
+        void UpdateMouseTouch(Vector2 position) {
+            Vector3 delta = position - lastPointerPosition;
+            delta = new Vector3(-delta.y * camSens, delta.x * camSens, 0);
+            delta = new Vector3(transform.eulerAngles.x + delta.x, transform.eulerAngles.y + delta.y, 0);
+            transform.eulerAngles = delta;
+
+            lastPointerPosition = position;
+        }
+
+        // Returns the basic values, if it's 0 than it's not active.
+        private Vector3 GetBaseInput() {
+            if (myKeyboard == null) {
+                return Vector3.zero;
+            }
+
+            Vector3 p_Velocity = new Vector3();
+
+            // Forwards
+            if (myKeyboard.wKey.isPressed)
+                p_Velocity += new Vector3(0, 0, 1);
+
+            // Backwards
+            if (myKeyboard.sKey.isPressed)
+                p_Velocity += new Vector3(0, 0, -1);
+
+            // Left
+            if (myKeyboard.aKey.isPressed)
+                p_Velocity += new Vector3(-1, 0, 0);
+
+            // Right
+            if (myKeyboard.dKey.isPressed)
+                p_Velocity += new Vector3(1, 0, 0);
+
+            // Up
+            if (myKeyboard.spaceKey.isPressed)
+                p_Velocity += new Vector3(0, 1, 0);
+
+            // Down
+            if (myKeyboard.leftCtrlKey.isPressed)
+                p_Velocity += new Vector3(0, -1, 0);
+
+            return p_Velocity;
+        }
+        #endregion
+
+    }
+}

Runtime/Scripts/CameraControls.cs

@@ -17,6 +17,7 @@ namespace FusedVR.VRStreaming
     /// The manager class for Controller Input Data i.e. Trigger, Grips Pressed, etc.
     /// This class provides an abstraction over the raw data ControllerDataEvent from VRInputManager 
     /// </summary>
+    [RequireComponent(typeof(VRInputManager))]
     public class ControllerInputManager : MonoBehaviour
     {
         #region Enums
@@ -71,18 +72,21 @@ public class VRAxisData : UnityEvent<Hand, Button, float, float> {
         public VRAxisData VRAxisEvent;
         #endregion
 
+        private VRInputManager input; // handles recieving the data
+
         #region Methods
         // Start is called before the first frame update
         void Start()
         {
-            VRInputManager.ButtonDataEvent += ButtonEvents; //start listening
-            VRInputManager.AxisDataEvent += AxisEvents; //start listening
+            input = GetComponent<VRInputManager>();
+            input.ButtonDataEvent += ButtonEvents; //start listening
+            input.AxisDataEvent += AxisEvents; //start listening
         }
 
         // OnDestroy is called when the game object is about to be destroyed in the scene
         private void OnDestroy() {
-            VRInputManager.ButtonDataEvent -= ButtonEvents; //end listening
-            VRInputManager.AxisDataEvent -= AxisEvents; //end listening
+            input.ButtonDataEvent -= ButtonEvents; //end listening
+            input.AxisDataEvent -= AxisEvents; //end listening
         }
 
         /// <summary>