Using RenderTexture to create reflection make the render "choppy" (code included)

[rudiHammad]

Hi,
I am trying to figure out how to make the environment reflection. Basically I want all the elements to be rendered upside down (I am not worrying about the shader, just the reflection). So what I did is take a screenshot (renderTexture) using RenderTexture::create(...), and then passing it to a sprite. Finally the update method uses the renderTexture method (clear), (begin) and end(). Here's the code using the built-in content images if any one wants to try it.

#pragma once
#include "axmol.h"
#include "renderer/Shaders.h"
#include "renderer/backend/DriverBase.h"

USING_NS_AX;



class ShadersSceneAdvanced : public ax::Scene
{
    
public:
    bool init() override;
    void update(float delta) override;



private:
    ax::Sprite* spriteA;
    ax::Sprite* mirrorSprite;
    RenderTexture* renderTexture;
};


bool ShadersSceneAdvanced::init()
{

    if (!Scene::init())
    {
        return false;
    }
        
    auto visibleSize = _director->getVisibleSize();
    auto origin      = _director->getVisibleOrigin();
    auto safeArea    = _director->getSafeAreaRect();
    auto safeOrigin  = safeArea.origin;


    /////////////////////////////////////////////////////////////////////////////////////////////////
    // 1. Capture screen with RenderTexture
    auto frameSize = Director::getInstance()->getVisibleSize();
    renderTexture = RenderTexture::create(frameSize.width, frameSize.height);
    renderTexture->retain();
    

    // 2. Pass the capture to a sprite
    mirrorSprite = Sprite::createWithTexture(renderTexture->getSprite()->getTexture());
    
    mirrorSprite->setAnchorPoint(Vec2(0,0));
    auto mirroSpriteOffset = Vec2(200, -10);                                                    // offset to move the sprite where I want
    mirrorSprite->setTextureRect(Rect(mirroSpriteOffset.x, mirroSpriteOffset.y, 320, 360));     // Set the texture rectangle to only use a portion of the texture
    mirrorSprite->setFlippedY(false);                                                           //BUG?: set flippedY false flips y already
    mirrorSprite->getTexture()->setAliasTexParameters();
    this->addChild(mirrorSprite, 10);

    mirrorSprite->setPosition(mirroSpriteOffset);
    


    /////////////////////////////////////////////////////////////////////////////////////////////////

    auto backgroundSprite = Sprite::create("HelloWorld.png");
    backgroundSprite->setAnchorPoint(Vec2(0,0));
    backgroundSprite->getTexture()->setAliasTexParameters();
    backgroundSprite->setPosition(visibleSize.x/2 -100, visibleSize.y/2);
    addChild(backgroundSprite);

    spriteA = Sprite::create("CloseSelected.png");
    spriteA->setPosition(Vec2(visibleSize.x/2 - 100, visibleSize.y/2));
    spriteA->setAnchorPoint(Vec2(0, 0));
    spriteA->getTexture()->setAliasTexParameters();
    addChild(spriteA, 1);

    auto moveAction = MoveBy::create(5, Vec2(100, 0));
    auto sequenceAction = Sequence::create(moveAction, moveAction->reverse(), nullptr);
    auto repeatAction = RepeatForever::create(sequenceAction);
    spriteA->runAction(repeatAction);
    

    scheduleUpdate();

    return true;
}


void ShadersSceneAdvanced::update(float delta)
{ 
    renderTexture->clear(0,0,0,0);
    renderTexture->begin();
    Director::getInstance()->getRunningScene()->visit();
    renderTexture->end();
}

As you can see in the video, everything works as expected, but the reflected images doesn't move as smoothly as the image above.
Also, I don't know if that's a bug or not, but the image is flipped when the flip is set to false (mirrorSprite->setFlippedY(false)). Shouldn't this be if set as true?

2024-12-18.17-28-14.mp4

thanks

[rh101]

Also, I don't know if that's a bug or not, but the image is flipped when the flip is set to false (mirrorSprite->setFlippedY(false)). Shouldn't this be if set as true?

It's not a bug. The RenderTexture is flipped in the Y axis, so to draw it in the correct orientation you would need to call Sprite::setFlipped(true). In your case, you want it to be upside down, so there is no reason to call setFlippedY(false), since the default should be false.

[rh101]

Would it be more efficient to render the entire scene to a RenderTexture (not to the screen), then use that same texture to display the normal and mirrored orientation?

[rh101]

@rudiHammad Demo project is here: https://github.com/rh101/axmol-reflection-demo

[rudiHammad]

Cool, thanks for that.
I tried your code and it looks smooth, but as you anticipated, it is related to setAliasTexParameters(). This is what I tested in you code.
If I do movingSprite->getTexture()->setAliasTexParameters(); the same choppy move happens. But if instead of doing that, I do
_rt->getSprite()->getTexture()->setAliasTexParameters();, then the movement looks smooth, and the sprite sharp as I want (no antialising).

[rudiHammad]

@rh101

Right now, based on your existing code, it is rendering to the screen, then rendering everything a second time to the render texture.

Can you expand on this please? I'm trying to understand where and when the first and second render occur, as don't see why rendering is happening twice.

[rh101]

Can you expand on this please? I'm trying to understand where and when the first and second render occur, as don't see why rendering is happening twice.

I just checked the code in your initial post again, and realised that you're not calling Director::getInstance()->getRenderer()->render(); after the renderTexture->end(); call. That would have been part of what I was referring to, but also the fact that your original code is calling scene->visit() again.

Each loop, the void Director::mainLoop(), as the name implies, runs in a loop. Inside this is a call to drawScene();, which calls _glView->renderScene(_runningScene, _renderer);. It does the following:

void GLView::renderScene(Scene* scene, Renderer* renderer)
{
...
    scene->render(renderer, Mat4::IDENTITY, nullptr);
}

As you can see, it starts off the rendering of the scene, which essentially means it ends up calling Scene::visit(...), and then looping through all child visit() methods.

Now, what happens next in your code is this:

void ShadersSceneAdvanced::update(float delta)
{ 
    renderTexture->clear(0,0,0,0);
    renderTexture->begin();
    Director::getInstance()->getRunningScene()->visit();
    renderTexture->end();
}

You end up calling Director::getInstance()->getRunningScene()->visit();, which re-renders the entire scene all over again, but this time to the RenderTexture, calling visit() etc. on every single child node of that scene.

What is happening here is that you've gone through the scene tree twice (or node tree, it's not really relevant what the root node is).

So, what I suggested earlier is to avoid this double iteration of the entire scene tree is by simply doing it once, with the render target being the RenderTexture. You then create sprites from the RenderTexture texture, one for normal orientation, and another flipped, to give you the mirror effect. This also ends up with a single draw call for the final output, since they're both using the exact same texture.

The original root node that is rendered to the RenderTexture is never added as a child of the running scene, so it never goes through the normal render process. This method only traverses the scene tree once, not twice.

Does that make sense?

[rudiHammad]

Totally, thanks for the explanation.
It's not related though to the main issue in this post, correct? Since setting setAliasTexParameters() on the moving sprite of your code and the same thing happens. Can you try that in your code and see if you get the same feeling that I do?
Because I thought that it might be related to the double rendering, but it seems that it's related to the aliasTexParameters. Isn't that strange?

Question:
RenderTexture* _rt; and then _rt->retain() would be the same than ax::RefPtr<ax::RenderTexture> _rt;? Looking at what they do, at the end both increase the reference count.

[rh101]

It's not related though to the main issue in this post, correct? Since setting setAliasTexParameters() on the moving sprite of your code and the same thing happens. Can you try that in your code and see if you get the same feeling that I do? Because I thought that it might be related to the double rendering, but it seems that it's related to the aliasTexParameters. Isn't that strange?

I did notice the same jerky movement, which became smooth when the call to setAliasTexParameters() was removed from the moving sprite. I haven't looked into setAliasTexParameters to see what it does, so I can't really comment on why it affect it like that.

Question: RenderTexture* _rt; and then _rt->retain() would be the same than ax::RefPtr<ax::RenderTexture> _rt;? Looking at what they do, at the end both increase the reference count.

ax::RefPtr is a wrapper, and will automatically call retain(), and also automatically call release() when the variable goes out of scope or is set to nullptr.

[rudiHammad]

After more testing, I believe that the problem comes from using Sprite::createWithTexture and is not related to setAntiAliasTexParameters() nor setAliasTexParameters(). Using setAntiAliasTexParameters() might look smoother but I feel that it just disguise the jerky move making it less noticeable.
Because if we compare the move of a standard scene, meaning just adding the sprite and move it, that move is much smooth than the sprites in the _normalView even with the Anti Aliasing.

In the videos bellow, the sprite on the right is added as a child of the scene to compare it with the one at the _dummyNode.
This video has antialising on (the default). Your nose should be 1 inch away from the monitor to notice that the one right is more
stable.
https://github.com/user-attachments/assets/3fd5592f-fcaf-4dfc-bec1-a21c1d3c3c71

In this video, where Iset the antialiasing off, it is easier to see the difference.
https://github.com/user-attachments/assets/8f37d095-f3b5-44b9-a42e-894e4a92c64e

I believe the issue occurs in both cases, but it becomes more noticeable when setAliasTexParameters() is used.

Anyway, enough time spent on this topic. Hopefully this can be fixed.

[rh101]

Sprite::createWithTexture

I doubt that this would be the cause of the issue, because if I understand it correctly, what that method does is simply use a specific texture for a sprite. It doesn't have anything to do with the movement.

What is more likely to be the cause is the way the movement is being captured into the RenderTexture.

[rh101]

@rudiHammad Can you try to comment out spriteA->getTexture()->setAliasTexParameters(), and see if it the mirrored animation becomes smoother?

[rudiHammad]

Sure. Just tried that. Maybe it's my head but the blurriness of the sprite with antialiasing makes me feel sick. I really can't tell if it is smoother or not.