[Done] Rendering opaque and transparent separately

[niello]

Hi.

I want to render GUI in two phases:
1. Render opaque GUI with Z-write enabled
2. Render scene
3. Render transparent GUI
to avoid rendering significant part of the scene occluded by opaque GUI.
Can I do this with 0.8 release without touching CEGUI core, if I use my own custom renderer?
Or is this planned for 1.0 release? If not, it would be nice to add this feature.
Thanks.

[Ident]

How do you plan to separate the opaque GUI and the transparent GUI: Are those two entirely different layouts that can live in separate CEGUI::GUIContext objects?

[YaronCT]

@niello: Which renderer do u use, and if it's ogre, which version and which render system inside ogre?

[niello]

@Ident: no, I just want to separate them in terms of rendering, keeping them in one context. Instead of calling UIContext::draw() I want to make two calls, one for rendering opaque GUI elements and one for transparent. With possible external rendering state changes in between.

@YaronCT: I use my own custom renderer, so I have full control on it.

[Ident]

How would you define what is opaque and what is not? Would that be a custom per-window decision? Then you can use a custom window property for that. Otherwise if you can specify it via the alpha value then you could use this (instead of a property) in the draw call to check for opaqueness. Add a parameter to the draw call to let the info trickle down if we are in the opaque-render-mode or the other one and then only render the ones you want in each pass, depending on the opaqueness-check. That's the best way I can think of doing this.

I dont think anyone ever tried to do what you are doing and that's why we dont have that. Also it does not seem obvious to me which elements you wanna render in which pass becaus textures can contain transparency too.

I am also still not quite sure about the purpose behind this. Mind to show an example? I would love to see one.

[niello]

It would be sufficient for me to manually set opacity property, although ideally system should treat GUI element as opaque if its render state has alpha-blending disabled (no explicit "texture has alpha" flag nor fade/transparency effect applied).
No working example exists, but idea is simple. Imagine typical game GUI where we have a set of informational windows at the bottom part of the screen. In RPG there can be message window, quick slots, mini-map and others. Some of them occlude 3D scene behind them, except all of them are semi-transparent, which is not my case. With a third-person camera and distance-based LOD, the part of scene occluded by those GUI will contain the most detailed objects, possibly many of them, and the size of this area is hundreds of pixels in both X and Y dimensions. I don't want to render these detailed objects at all (via occlusion culling) or partially (using depth test), because their potentially heavy pixels end up invisible. So I must render opaque GUI in the first place, filling my Z-buffer.
Can't say what performance impact it would have, but I expect noticeable improvement. So I need to implement this feature before I can conclude is it useful or just another stupid optimization that doesn't work

[YaronCT]

@Ident: Sounds like a useful feature to me. I think it's common in 3D engines to render the opaque objects b4 the translucent ones, and I think it would b useful if we support it too. And I don't think it should b very hard to implement automatically - if it's a texture we can just check whether it contains a not fully opaque pixel.

[YaronCT]

@niello: Out of curiousity, y do u use your own renderer?

[niello]

Examining a texture to find alpha pixels may be suboptimal in case CEGUI uses / plans to use alpha-testing. Also texture with alpha, rendered without alpha-blending, will result in opaque GUI element. Anyway these are implementation details.
I use my own renderer because I develop my own engine. I enjoy architecture work too much to use 3rd-party one, and I like to know that any part of my code is open for exploring and modification. Renderer supports both D3D9 and D3D11 through a virtual interface.

[Ident]

It would be sufficient for me to manually set opacity property

I think that might be the most reasonable approach.

although ideally system should treat GUI element as opaque if its render state has alpha-blending disabled

The CEGUI GUI elements do not have explicit alpha-blending states.

(no explicit "texture has alpha" flag nor fade/transparency effect applied).

There is no "texture has alpha" flag in direct3d or opengl. Even if you have an alpha channel this might be a texture importer thing and be unused, or it might be that the source image just has all alpha bits on 1.0 or 255. My point is that it would be tedious and unreliable to base anything on texture information and if something is flagged wrong and you want to fix that manually, well, you couldn't in the case that it is all automatised. And as you said, the texture isnt the only source of transparency, alpha property on windows is also there.

So ideally, I believe, a new window property would come into play here. Adding a member to CEGUI::Window for this purpose would break ABI but we can work with a regular (not quite as efficient) property here and do it right on default branch.

We would then need to overload the draw function. I would add an enum mode for the rendering types. There should be ALL, OPAQUE, and NON-OPAQUE (or whatever). I just checked CEGUI code and I think this is feasible and not a large change as yaron said, although we will need to do this somewhat dirtily in v0-8 because all functions need to be copied that will have the new enum, which will need to trickle down all the way to window granularity.

[Ident]

Examining a texture to find alpha pixels may be suboptimal in case CEGUI uses / plans to use alpha-testing. Also texture with alpha, rendered without alpha-blending, will result in opaque GUI element. Anyway these are implementation details.

I was still answering while you wrote that but I see that we agree on this.

[niello]

Sure an automated approach must not replace a manual one. And I agree that adding automated one in our case doesn't worth effort. If you really implement it in v0-8 I will test it and report results. Statistics from popular platforms like Ogre may be a lot more useful though.

[Ident]

Sure an automated approach must not replace a manual one. And I agree that adding automated one in our case doesn't worth effort. If you really implement it in v0-8 I will test it and report results. Statistics from popular platforms like Ogre may be a lot more useful though.

Having worked with Ogre for quite a bit I would not say that statistics on our scenario are more useful there. If you have a slim engine specialised on your use purpose then the perf. difference we see in your case is worth much more to me than that of Ogre.

[YaronCT]

@Ident: If it's too dirty to add it to branch "v0-8" I think we better do it in branch "v0", which is pretty stable I think and @niello can use that.

[Ident]

I am working on an untested version for niello, based on v0-8 right now.