Skip to content

BlockifyVR (Virtual Reality) #1732

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 44 commits into
base: master
Choose a base branch
from

Conversation

Brackets-Coder
Copy link
Contributor

@Brackets-Coder Brackets-Coder commented Oct 23, 2024

BlockifyVR

This is a Virtual Reality extension I've been working on since January 27th of 2023.

The end goal is to have:

  • Little to no performance overhead with the extension
  • Cross-platform compatibility for all major 6DOF headsets A-frame supports
  • An easy-to-use system that's coherent and efficient
  • Works well with any 3D engine
  • Bug-free enough for optimal user experience without frequent disruptions

Here are some things to note:

  • It includes a local copy of the A-frame JS library, version 1.7.0, MIT License. This is for VR Management, rendering, and ease of development as I'd have to write the entire thing from scratch using WebXR, WebGL, and the Gamepad API otherwise. Setting up the VR boilerplate and stereoscopy and other things would be a pain without it.
  • Gallery thumbnail, documentation, and sample projects are not currently available and will be worked on as I get nearer towards release
  • The rendering uses a plane fixed to the camera which directly mirrors the scratch stage with an unshaded texture each frame. This is relatively performant and works well, but there could be better solutions to look into. The plane is placed 1 meter away from the camera and A-frame uses stereoscopic rendering to reduce eye strain. Note that the plane automatically scales to the size of the camera (display) while attempting to maintain the ratio of the stage. I use this instead of drawing directly to the canvas because it won't let me and I would have to re-write the renderer that's already tailored for VR

@Brackets-Coder Brackets-Coder marked this pull request as ready for review October 24, 2024 19:02
@Brackets-Coder Brackets-Coder marked this pull request as draft October 24, 2024 19:02
@Brackets-Coder Brackets-Coder changed the title BlockifyVR (Virtual Reality) Extension New Extension: BlockifyVR (Virtual Reality) Oct 24, 2024
@Brackets-Coder Brackets-Coder changed the title New Extension: BlockifyVR (Virtual Reality) BlockifyVR (Virtual Reality) Oct 24, 2024
@SharkPool-SP SharkPool-SP added the pr: new extension Pull requests that add a new extension label Oct 24, 2024
@hammouda101010
Copy link

that looks cool

@hammouda101010
Copy link

that looks cool

It looks cool on the flat screen right now, but it's not so great when you try it. You'd probably get motion sickness. I'm still working on it.

sadly i dont think that's possible to stop this.

@Xeltalliv
Copy link
Contributor

Does this show the same scratch 3D perspective to both eyes?

@Brackets-Coder
Copy link
Contributor Author

Brackets-Coder commented Oct 29, 2024

Does this show the same scratch 3D perspective to both eyes?

A-frame has stereoscopic rendering built in and I've enabled it in the rendering settings, so no. This fixes some issues with visualizing depth, so the eye offset allows your brain to process the image with more clarity. Even if the matrices aren't used in the renderer, it should properly use the correct eye offset. Why do you ask, just out of curiosity? If you'd like I could add a block that disables or enables this feature, in case the Scratcher wants to build that framework themselves, which I'd doubt.

Keep in mind this is still a work in progress and most of the code is many months old so it's not the most efficient way to do VR.

@Xeltalliv
Copy link
Contributor

Why do you ask, just out of curiosity?

Yes. At one point I tried to extend my AR extension to support AR headsets and VR as well, but the multiple eye rendering has been the main thing holding me back. I was just curious to know how you solved it.

@Brackets-Coder
Copy link
Contributor Author

Brackets-Coder commented Oct 29, 2024

Why do you ask, just out of curiosity?

Yes. At one point I tried to extend my AR extension to support AR headsets and VR as well, but the multiple eye rendering has been the main thing holding me back. I was just curious to know how you solved it.

I believe that there can be two XRViewerPose matrices for each eye if available, according to Mozilla Docs. You might want to look into that again. It'd be great if Augmented Reality could support AR headsets such as the Oculus Quest 3 in addition to just mobile touchscreen devices. There should be a views array that can do this. An example can be seen here. Just a suggestion though. I simply think having AR headsets would be a nice update.

@Brackets-Coder

This comment was marked as resolved.

@Xeltalliv
Copy link
Contributor

Xeltalliv commented Oct 30, 2024

I'll take a look when I have time. Thought it will probably be hard because the only kind of VR I have access to is phone-based 3DOF no controller VR.

Also, do you mind giving some links, so that I can faster figure out where to look?

@Brackets-Coder

This comment was marked as resolved.

@Brackets-Coder
Copy link
Contributor Author

I'll take a look when I have time. Thought it will probably be hard because the only kinf of VR I have access to is phone-based 3DOF no controller VR.

Also, do you mind giving some links, so that I can faster figure out where to look?

As for VR testing, you wouldn't really need a VR headset. I've used Meta Quest's Immersive Web Emulator extension for testing when I didn't care to load up the entire headset.

@Drago-Cuven
Copy link

ah yes, I remember seeing a trailer for this on scratch. I wanted to help, but now I can't

@Drago-Cuven
Copy link

nvm I think I can
i'm Martinelplayz on scratch btw

@siskka7

This comment was marked as resolved.

@Thebloxers998
Copy link

could i try the extension?

Hi siskka7 it's me Thebloxers998

@siskka7
Copy link

siskka7 commented Nov 1, 2024

@Thebloxers998
Copy link

Thebloxers998

hmm https://github.com/TurboWarp/extensions/blob/2837a7adfbfc2e3aba49108003686affa1cb7beb/extensions/MasterMath/BlockifyVR.js

I'm not sure what you're "hmm"ing. It's just the source code.

Hi, nice extension

@Xeltalliv
Copy link
Contributor

I'll take a look when I have time.

I took a look and have no idea. It's all code specific to AFRAME. I don't know AFRAME. AFRAME is too big and complicated for me to try to debug, especially if the only tool in my disposal is WebXR API Emulator browser extension.

@Thebloxers998
Copy link

Nic3

@Thebloxers998
Copy link

Nice*

@GarboMuffin
Copy link
Member

GarboMuffin commented Feb 22, 2025

Recent versions of Chrome on Android added support for window.showOpenFilePicker and maybe Meta has not bothered to finish their side of the support...

@Brackets-Coder
Copy link
Contributor Author

Recent versions of Chrome on Android added support for window.showOpenFilePicker and maybe Meta has not bothered to finish their side of the support...

It was working previously, but now it's not, so it's likely just a browser issue... or maybe it's the latest A-frame v1.7.0 that came out yesterday and moves from CommonJS to ES modules? I don't really know but the extension seemed to have loaded properly so it didn't throw significant initialization errors, I just can't load a project.

@siskka7
Copy link

siskka7 commented Feb 23, 2025

@GarboMuffin and @CubesterYT sorry to ping you but this is a bit odd.

I attempted to test a backend code efficiency update to make this more reviewable when I discovered that "File > Load From Your Computer" doesn't work on the Oculus Quest browser. All other menu / navbar items work. I'm assuming this is a browser issue because it was working previously... any recommendations or should I just wait for another browser patch update? The tricky part is that development on the Oculus Browser is a big pain in the neck because it doesn't have any developer tools by default, so I don't have access to a JS console and no alerts popped up, so I don't know what's happening.

i hope meta did not do this on purpose beceause it may be on purpose.meta does not like modding or any other things that are we not suposed to do.

@Brackets-Coder
Copy link
Contributor Author

Brackets-Coder commented Mar 19, 2025

I've made some changes to try to make it more professional, object-oriented, follow best practices, etc. because some of this code was really old and I never bothered to fix it until now. Hopefully this should be easier to review. Note that I haven't tested this commit because I for some reason couldn't get file uploads to work. Some untested breaking changes may be present. Don't merge until I can confirm this is working properly.

Also this commit includes A-frame v1.7.0

@Brackets-Coder
Copy link
Contributor Author

@CubesterYT I'm trying to get this all finished up but I've come to the conclusion that this will be especially difficult to review considering the difference is too large to display for reviewing... what should I do about that? You can still manually view the file but you can't add comments to it and continue with the normal review process.

@CubesterYT
Copy link
Member

Don't worry about size, we still have ways to review it (VSCode for example)

@Drago-Cuven
Copy link

I think I broke this with the backend efficiency update but I'll have to check just to make sure. If it's not working then I can just go back to the old working version and optimize some things there or just merge the messy version but the mods probably wouldn't like that and tell me to cast and use switches instead of the horrific if else blocks

Take your time. I know you'll come up with the best possible solutions. It just takes time.

@Brackets-Coder
Copy link
Contributor Author

Brackets-Coder commented Jun 28, 2025

That moment you realize that you forgot to put the VR controls my block in the forever loop and that's the only reason why it hasn't been working for a month 😭

@Brackets-Coder
Copy link
Contributor Author

So everything's working now with the backend efficiency update. I've also enabled colorManagement: true so now everything should be on an sRGB linear workflow and consistent with what users see on their computer screens... a better WYSIWYG workflow. Basically I've just got to go through and add Scratch.Cast and this should be ready for merging like officially.

@CubesterYT how necessary is it that an extension like this have documentation when it appears on the gallery? How big of a problem will it be and how many people will complain on my Scratch profile about how to use it? Is that something I need to have finished before it's merged or can that come later with sample projects and extra resources?

@CubesterYT
Copy link
Member

I highly recommend docs for this, same with your Ammo3D extension

Big/complicated extensions users usually look for the most help

@Brackets-Coder
Copy link
Contributor Author

I highly recommend docs for this, same with your Ammo3D extension

Big/complicated extensions users usually look for the most help

Okay, just means later merge date. I was going to do docs for both anyway it was just a matter of when...

@Brackets-Coder Brackets-Coder marked this pull request as draft June 30, 2025 15:14
@Brackets-Coder
Copy link
Contributor Author

@CubesterYT or @yuri-kiss
I've figured out the source to one of my main issues. When entering a VR experience, controllers don't connect until the stage has been interacted with in some way (most often clicking). I'm just assuming this is a higher-level issue with the browser, but is there anyway to go around this limitation? Most often users will just press the green flag, "enter VR", and then load into the game without interacting with the stage.

@Brackets-Coder
Copy link
Contributor Author

@CubesterYT or @yuri-kiss I've figured out the source to one of my main issues. When entering a VR experience, controllers don't connect until the stage has been interacted with in some way (most often clicking). I'm just assuming this is a higher-level issue with the browser, but is there anyway to go around this limitation? Most often users will just press the green flag, "enter VR", and then load into the game without interacting with the stage.

@CubesterYT It seems this same issue isn't just in my extension, it's a browser-wide security limitation on every WebXR experience available to me. Will this be something the users will have to face on their own? I can't do anything about it, but it is a bit aggravating to not have controllers connect immediately upon entering VR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr: new extension Pull requests that add a new extension
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants