shaderc compile through api with #include I'm using shaderc to compile glsl code in to spirv module and pass to pipelie in REAL-TIME. I want to use #include to reuse some of the shader code but shaderc is having problems recognizing it. Any one know how to solve this? (I'm NOT using command line to compile shader upfront.)

The error I got from shaderc is:

standard.vert:6: error: '#include' : #error unexpected include directive for header name: GlobalInclude.glsl

Here are my shaders and client codes:

standard.vert

#version 450
#extension GL_ARB_separate_shader_objects : enable
#extension GL_KHR_vulkan_glsl : enable
#extension GL_GOOGLE_include_directive : enable

#include "GlobalInclude.glsl"

layout(location = 0) in vec3 inPosition;
layout(location = 1) in vec3 inColor;
layout(location = 2) in vec2 inTexCoord;

layout(location = 0) out vec3 fragColor;
layout(location = 1) out vec2 fragTexCoord;

void main() {
  gl_Position = passUBO.proj * passUBO.view * objectUBO.model * vec4(inPosition, 1.0);
  fragColor = inColor;
  fragTexCoord = inTexCoord;
}

standard.frag

#version 450
#extension GL_ARB_separate_shader_objects : enable
#extension GL_KHR_vulkan_glsl : enable
#extension GL_GOOGLE_include_directive : enable

#include "GlobalInclude.glsl"

layout(set = PASS_SET, binding = TEXTURE_SLOT(0)) uniform sampler2D texSamplerAlbedo;

layout(location = 0) in vec3 fragColor;
layout(location = 1) in vec2 fragTexCoord;

layout(location = 0) out vec4 outColor;

void main() {
  //outColor = vec4(fragTexCoord, 0.0, 1.0);
  outColor = texture(texSamplerAlbedo, fragTexCoord);
}

GlobalInclude.glsl

#define FRAME_SET 0//per frame
#define SCENE_SET 1//per scene
#define PASS_SET 2//per pass
#define OBJECT_SET 3//per draw call

#define UNIFORM_BUFFER_BINDING_START 0
#define TEXTURE_BINDING_START 1

#define TEXTURE_SLOT(NUMBER) TEXTURE_BINDING_START + NUMBER

layout(set = SCENE_SET, binding = UNIFORM_BUFFER_BINDING_START + 0) uniform SceneUniformBufferObject {
  uint time;
} sceneUBO;

layout(set = PASS_SET, binding = UNIFORM_BUFFER_BINDING_START + 0) uniform PassUniformBufferObject {
  mat4 proj;
  mat4 view;
  uint passNum;
} passUBO;

layout(set = OBJECT_SET, binding = UNIFORM_BUFFER_BINDING_START + 0) uniform ObjectUniformBufferObject {
  mat4 model;
} objectUBO;

layout(set = FRAME_SET, binding = UNIFORM_BUFFER_BINDING_START + 0) uniform FrameUniformBufferObject {
  uint frameNum;
} frameUBO;

Compile shader using shaderc API:

bool Shader::CreateShaderFromFile(shaderc_shader_kind kind, bool optimize)
{
  // read shader string
  if (!ReadShaderFromFile())
      return false;

  shaderc::Compiler compiler;
  shaderc::CompileOptions options;

  // like -DMY_DEFINE=1
  //options.AddMacroDefinition("MY_DEFINE", "1");
  if (optimize) options.SetOptimizationLevel(shaderc_optimization_level_size);

  std::cout << "output 1:" << shaderString << std::endl;

  // preprocess
  shaderc::PreprocessedSourceCompilationResult preprocessed = compiler.PreprocessGlsl(shaderString, kind, fileName.c_str(), options);

  if (preprocessed.GetCompilationStatus() != shaderc_compilation_status_success)
  {
      throw std::runtime_error(preprocessed.GetErrorMessage());
      return false;
  }

  shaderString = { preprocessed.cbegin(), preprocessed.cend() };

  std::cout << "output 2:" << shaderString << std::endl;

  // compile
  shaderc::SpvCompilationResult module = compiler.CompileGlslToSpv(shaderString, kind, fileName.c_str(), options);

  if (module.GetCompilationStatus() != shaderc_compilation_status_success)
  {
      throw std::runtime_error(module.GetErrorMessage());
      return false;
  }

  shaderBytecode = { module.cbegin(), module.cend() };// not sure why sample code copy vector like this

  return true;
}

It is your responsibility to support includes by instantiating an IncluderInterface instance and calling SetIncluder(...) on the options instance.

The interface includer API consists of the following methods:

// Handles shaderc_include_resolver_fn callbacks.
virtual shaderc_include_result* GetInclude(const char* requested_source,
                                           shaderc_include_type type,
                                           const char* requesting_source,
                                           size_t include_depth) = 0;

// Handles shaderc_include_result_release_fn callbacks.
virtual void ReleaseInclude(shaderc_include_result* data) = 0;

It will therefore also be your responsibility to implement these functions and interpret the #include directives from the GLSL into shaderc_include_result instances that will be used by shaderc to complete the preprocessing of the GLSL.


Deciphering Affine/Projective Transformation Code I've spent about 2 days trying to understand this piece of code (from here) that applies an affine/projective transformation to an image. I will link bits of the code that I'm struggling to understand and add comments on what I think it's trying to do. In particular, the author's original comments I'll label as //.

//      COMPUTE NEW BASIS
// X1, Y1 : upleft corner
// X2, Y2 : upright corner
// X3, Y3 : downleft corner
// sx     : x-size of output image
// sy     : y-size of output image
x12 = (X2-X1)/(float)(*sx);
y12 = (Y2-Y1)/(float)(*sx);
x13 = (X3-X1)/(float)(*sy);
y13 = (Y3-Y1)/(float)(*sy);

Q1. I can see that this part as the comment says, finds a new normalized basis but I'm unsure exactly why we need to find one to model an affine transformation?

// x4, y4 : downright corner (for projective transform)

if (y4)
{
    xx=((x4-X1)*(Y3-Y1)-(y4-Y1)*(X3-X1))/((X2-X1)*(Y3-Y1)-(Y2-Y1)*(X3-X1));
    yy=((x4-X1)*(Y2-Y1)-(y4-Y1)*(X2-X1))/((X3-X1)*(Y2-Y1)-(Y3-Y1)*(X2-X1));
    a = (yy-1.0)/(1.0-xx-yy);
    b = (xx-1.0)/(1.0-xx-yy);
}
else
{
    a=b=0.0;
}

After doing some internet search, I manage to find equations from here that the xx and yy equations solve for the "intersection point of two line segments in 2 dimensions". An image will be linked below.

START IMAGE.

END IMAGE.

Q2. I don't understand how he uses the xx and yy equations to solve for a & b?

Q3. I am unsure of what a and b are. My best guess is that they are the bottom row of a projective transformation matrix shown below:

$$
\begin{bmatrix}
a_{11} & a_{12} & a_{13} \\
a_{21} & a_{22} & a_{23} \\
a & b & 1 \\
\end{bmatrix}
$$

This is because the code sets a = b = 0.0 if we're not doing a projective transform, which should then make it an affine transformation matrix.

Now is where the main loop happens:

for (x=0;x<sx;x++)
    for (y=0;y<sy;y++) {
        fx = (float)x + 0.5;
        fy = (float)y + 0.5;
        d = a*fx/(float)(*sx)+b*fy/(float)(*sy)+1.0;
        xx = (a+1.0)*fx/d;
        yy = (b+1.0)*fy/d;
        xp = X1 + xx*x12 + yy*x13;
        yp = Y1 + xx*y12 + yy*y13;
    }

Q4. Why does he add 0.5 to the original x and y coordinate? My best guess is that the code's trying to map the centre of each pixel rather than the edge to the new image:

Q5. In

d = a*fx/(float)(*sx)+b*fy/(float)(*sy)+1.0;
xx = (a+1.0)*fx/d;
yy = (b+1.0)*fy/d;

what are they trying to accomplish here? My best guess is that d is the third component of the output homogeneous coordinate:

$$
\begin{bmatrix}
xx\\
yy\\
d\\
\end{bmatrix}
$$

calculated from the dot product of the third row of the (now normalised) transformation matrix from earlier and the input homogeneous vector:

$$
\begin{bmatrix}
\frac{a}{sx} & \frac{b}{sy} & 1 \\
\end{bmatrix}
\begin{bmatrix}
fx\\
fy\\
1\\
\end{bmatrix}
=
a*fx/sx+b*fy/sy+1.0
$$

Then the division

xx = (a+1.0)*fx/d;
yy = (b+1.0)*fy/d;

would just transform the homogeneous coordinates back to Euclidean coordinates, corresponding to the output image pixel location. I noted that if a=b=0.0 (so not projective), then (a+1.0)=(b+1.0)=d=1.0. I'm however not sure about why we have to multiply by (a+1.0) or (b+1.0). But again, I do not know if I'm on the right track or not as I do not have anyone to help me with this code.

Q7. The final operation

xp = X1 + xx*x12 + yy*x13;
yp = Y1 + xx*y12 + yy*y13;

I suppose is literally the dot product of the first two rows of the transformation matrix with the Euclidean coordinates as so:

$$
\begin{bmatrix}
x12 & x13 & X_{1} \\
y12 & y13 & Y_{1}
\end{bmatrix}
\begin{bmatrix}
xx\\
yy\\
1\\
\end{bmatrix}
=
\begin{bmatrix}
xp\\
yp\\
\end{bmatrix}
$$

Which brings me to my question, how does this achieve an affine transformation?

I suspect my lack of formal training could be why I'm not understanding how the code models an affine transformation. I'd appreciate some light shed on this piece of code!

Cheers!

Your understanding of the matrix structure in Q3 is correct. This code just does not construct a matrix explicitly and the matrix multiplication is applied implicitly. I think this part might cause your confusion.
Instead of deciphering the code, I would rather derive the transform and compare it with the code.

The affine (6 degrees of freedom) and projective matrix (8 dof) map points to points. So you can build a system of equations for each pair of points in different frames. You will need to solve this system for the components of the matrix (Q1).

Let $M$ be the transform matrix, then

$$
\begin{align}
M (0,0,1) &= (X_1,Y_1,1) \\
M (1,0,1) &= (X_2,Y_2,1) \\
M (0,1,1) &= (X_3,Y_3,1) \\
\end{align}
$$
Each of above lines gives you two equations. Now you have 6 equations and the affine matrix can be derived.
For the projective matrix, we need extra two equations using the point $(X_4,Y_4)$:

$$
\begin{align}
H_4 &= M (1,1,1) \\
\frac{H_4}{H_4.z} &= (X_4,Y_4,1)
\end{align}
$$
Note that I have to divide the homogeneous part for the transformed point.

Solve this linear system you can get an expression for your matrix $M$. Perform $M (f_x,f_y,1)$ you will see how it transforms the point. I believe this solves your Q2, Q3, Q5 and Q7.

For Q4, I think it is just a convention of pixel position and your understanding should be correct.

Appendix: maple code to verify the answer

with(ListTools):
# affine matrix
m:=<m11,m21,0|m12,m22,0|m13,m23,1>;
# Build equations
E:=[convert(m.<0,0,1> - <X1,Y1,1>,list),convert(m.<1,0,1> - <X2,Y2,1>,list), convert(m.<0,1,1> - <X3,Y3,1>,list)];
# E=[[m13 - X1, m23 - Y1, 0], [m11 + m13 - X2, m21 + m23 - Y2, 0],[m12 + m13 - X3, m22 + m23 - Y3, 0]]
# Solve E=0
sol:=eliminate(Flatten(E),indets(m));
# Plug the solution back to m:
subs(sol[1],m);
                           [-X1 + X2    -X1 + X3    X1]
                           [                          ]
                           [-Y1 + Y2    -Y1 + Y3    Y1]
                           [                          ]
                           [   0           0        1 ]

# projective matrix
p:=<m11,m21,a|m12,m22,b|m13,m23,1>;
# Compute transformed points
U0:=p.<0,0,1>;
U1:=p.<1,0,1>;
U2:=p.<1,1,1>;
U2:=p.<0,1,1>;
U3:=p.<1,1,1>;
# Build equations
E:=[convert(U0/U0[3]-<X1,Y1,1>,list),convert(U1/U1[3]-<X2,Y2,1>,list),convert(U2/U2[3]-<X3,Y3,1>,list),convert(U3/U3[3]-<X4,Y4,1>,list)];
# Sove E=0
solp:=eliminate(Flatten(E),indets(p));
# copy their code for xx and yy:
xx:=((X4-X1)*(Y3-Y1)-(Y4-Y1)*(X3-X1))/((X2-X1)*(Y3-Y1)-(Y2-Y1)*(X3-X1));
yy:=((X4-X1)*(Y2-Y1)-(Y4-Y1)*(X2-X1))/((X3-X1)*(Y2-Y1)-(Y3-Y1)*(X2-X1));
# Verify our a and b, compare ours with their expressing using xx and yy
subs(solp[1],a);
subs(solp[1],b);
simplify(subs(solp[1],a)-(yy-1)/(1-xx-yy));
# It is 0 as expected
simplify(subs(solp[1],b)-(xx-1)/(1-xx-yy));
# It is also 0


Radiative Transfer Equation for Photorealistic Rendering I've recently become interested in photorealistic rendering, and I've been looking at the different rendering philosophies. I read this Disney Research bachelor's thesis, which states both the radiative transfer equation (RTE)

$(\vec{w}\cdot\nabla)L(\vec{x},\vec{\omega}) = -(\sigma_{a}+\sigma_{s})L(\vec{x},\vec{\omega})+Q(\vec{x},\vec{\omega})+\sigma_{s} \int_{\Omega}L(\vec{x},\vec{\omega})\rho(\vec{\omega},\vec{\omega}')d\vec{\omega}'$

and that

  Exact BSSRDFs follow trivially from exact solutions to the RTE.
  Unfortunately, closed form solutions of the RTE for general 3D
  geometry, illumination and materials are rare, which motivates
  analysis on simplified, lower dimensional domains.

But most shaders use BSSRDF's and(/or?) the Rendering Equation instead, even high-end shaders like Disney's Hyperion, even though the Rendering Equation supposedly doesn't take into account sub-surface scattering, polarization, etc..

So my two questions are a) what is the reason for not using the radiative transfer equation for photorealistic rendering (or an approximation like the skin-shaders used for Big Hero and Moana), and b) how are the parameters like the phase functions and coefficients found for different materials like skin, hair, water, metal, glass?

I suspect the answer to (a) is simply performance. Full volumetric path tracing / photon mapping based on the RTE can certainly be done (and I'm sure it sometimes is), but it's very expensive and requires enormous numbers of samples to converge to a noise-free result.

This is especially true when the scattering medium is very dense, like in human skin and hair, or materials like milk, plastic, marble, or jade. In such materials, the light scatters so quickly that it's essentially isotropic after a couple of millimeters, so a BSSRDF is a very good approximation to the true volumetric scattering, and far cheaper to evaluate. (Performance matters even for offline rendering!) A mixture of volumetric simulations and empirical data might be used to derive and validate the BSSRDF.

As for how the coefficients are measured, one technique I've read about is to shine a narrow, bright beam of light (like a laser pointer) onto the material, then measure the light scattered back out using photosensors placed at different distances from the incident point. By processing this data you can then get an estimate of the scattering coefficients. This paper shows a sensor based on this idea being applied to human skin, and this one has more details about setting up a sophisticated (multi-layer) BSSRDF based on empirical data.


OpenGL set a vertex element stride of zero Problem X: I need to provide "compensation" streams for attributes that are not plugged by the mesh data, but the shader requires them.
(In d3d not matching the input layout is a hard error)

Solution Y: In D3D I've created a 16 bytes buffer initialized with some default color (when the missing semantic is color, let's take this as my typical use case) and specified a stride of zero so that one element is repeated whatever the index in the index buffer, and this way I don't need to clog VRAM with the biggest buffer that's necessary to cover all objects. This works in D3D.

But looking at glVertexAttribPointer I see that the API is eating away my stride and makes it automatic when I specify zero.

Any workaround ?

Disable the vertex attrib array and use glVertexAttrib to set the data that should be used.


In what order to apply lighting and translucency? I'm making a game that is supposed to like like a early 90s 3D game, so i'm rendering in software on an 8-bit indexed bitmap, using lookup tables for translucency and lighting.

Now I have to draw a translucent, lit sprite in front of an already-lit background (the light level of said background is lost at that point). I'm unsure whether to first darken all the pixels and then average them with the background or the other way round.

When blending multiple layers, physically the "right" thing to do is calculating lighting on each layer separately, then composite the lit layers together. This way, for instance, you can have a translucent sprite standing in a spotlight, in front of a dark background, and the sprite will be well-lit while the background visible through it stays dark. Or conversely, you can have a dark sprite silhouetted against a brightly lit background.

If you blended the sprite with the background before lighting, it wouldn't be clear which lighting environment you should use if the sprite and background have different ones.


Alternatives to Ambient Occlusion Recently I wanted to improve the lighting in my game engine, naturally I looked towards Ambient Occlusion. I was considering SSAO but are there any better techniques? Maybe some that are Physically Based? Are there any other techniques that replace SSAO or HBAO+ entirely?

I am aware of VXAO but is that basically it?

Ambient occlusion cannot be physically based no matter what algorithm you use to calculate it. It's a simplification of global illumination that assumes all occluders only block light and are themselves completely black, which isn't true and generally makes corners darker than they should be.

Ideally, you would be using some kind of global illumination method, but they generally don't have enough resolution to capture the effect ambient occlusion simulates and screen space global illumination tends to be too expensive.

That means ambient occlusion is still a reasonable choice, even if it's not entirely physically correct. Screen space ambient occlusion methods are the fastest, but suffer from the limited information they have access to, which can lead to artifacts like halos around objects in the foreground and the edges of the screen. Having more information about the scene to calculate the AO from, like VXAO does, eliminates those artifacts, but it again comes at a performance cost.

In the end, it depends on your perf budget. The more time you can spend calculating global illumination and ambient occlusion, the better the results are going to look. This seems to be a solid resource about screen space ambient occlusion algorithms in particular. I've personally had good results with Alchemy AO accelerated for large radii with a depth mip-chain similar to the Scalable Ambient Obscurance paper.

For an alternative to traditional AO, there's Bent Normals and Cones in Screen-Space that I've heard a lot about. As far as I understand it, the key difference is that instead of just calculating how much a point in space is occluded, it also captures information about what directions in particular are occluded. This information is then later used for lighting, where only light coming from the unoccluded directions is considered. It's still not what you'd call physically based, but in my opinion it comes closer than the usual ambient occlusion.


Reliable test for intersection of two Bezier curves How to reliably find out whether two planar Bezier curves intersect? By "reliably" I mean the test will answer "yes" only when the curves intersect, and "no" only when they don't intersect. I don't need to know what parameters the intersection was found at. I also would like to use floating-point numbers in the implementation.

I found several answers on StackOverflow which use the curves' bounding-boxes for the test: this is not what I'm after as such test may report intersection even if the curves don't intersect.

The closest thing I found so far is the "bounding wedge" by Sederberg and Meyers but it "only" distinguishes between at-most-one and two-or-more intersection, whereas I want to know if there is at-most-zero and one-or-more intersections.

An alternative way to formulate the problem is to define a function that gives the distance between points on the two curves, as a function of the curves' parameters. Then attempt to find the global minimum of this function. If the curves intersect, the minimum will be zero; otherwise the minimum will be some positive distance.

To be explicit, given a pair of 2D curves defined by $c_1, c_2 : [0, 1] \to \mathbb{R}^2$, define the distance-squared as

$$ f(u,v) : [0,1]^2 \to \mathbb{R}_{\geq 0} \equiv \bigl|c_2(v) - c_1(u)\bigr|^2 $$

For cubic curves, the function $f$ is then a sixth-degree polynomial in two variables. You can then apply numerical optimization techniques such as the simplex method or conjugate gradient descent. Unfortunately the function can have several local minima (it's not convex), so optimization isn't easy. There may well be more specialized optimization methods available for polynomials, but this isn't an area of expertise for me.


reducing image size increases file size? When I download this image:

(source: ucn.edu.ni)

then its total size is 341,0 kB.

The image has a width of 2880 pixels and height of 1100 pixel and 1 X/Y resolution pixel/in.

I opened the image with gimp and pressed scale image and reduced the width to 1920px and auto adjusted height of 733px, and keeped the resolution as it is and used the interpolation cubic. If I now export this as a jpg with 90% quality, then the image size is 546 kB!

How is that possible and how can I avoid that?

JPEG is a lossy format and depends on both the 2d frequency components of the image and the user specified quality level.

It is possible that down-scaling an image can increase the higher frequency components and potentially result in an increase in file size, typically higher frequency components a encoded using a higher number of bits compared to lower frequency components. An example would be to imagine shrinking a checkerboard texture, the smaller it gets the more it shifts into higher frequency.

It may also be possible that your original image was encoded with a low quality factor and your new image has a high quality factor so it can also gain extra bytes from that. On a more technical level this may be because of a mismatch in quantization bands leading to higher number of DCT coefficients being needed compared to the original JPEG. The quality parameter is not standardized and different applications have different ideas of what '90% quality' means.

Another problem is that because you are using JPEG as a source image it will introduce ringing artifacts into the image, which then need more bits to encode when trying to encode it.

Unfortunately there is no easy fix for this apart from reducing quality.


Fresnel reflectance of gold: red channel greater than 1? I've been trying to understand some of the physical principles behind light and material interaction lately. In his talk Physics and Math of Shading, Naty Hoffman explains Fresnel reflectance and defines the characteristic specular color F0 of a material as the Fresnel reflectance at 0° incident light angle.

On slide 65, F0 of gold is given as 1.022, 0.782, 0.344 (linear). Hoffman adds:

  its red channel value is greater than 1 (it’s outside sRGB gamut)

All of this doesn't make too much sense to me. A value greater than 1 would mean that in the wavelengths contributing to the red channel, more energy is reflected than is received. Does this really happen, and if so, how and why?

In addition, here is a reflectance curve from Wikipedia for some materials including gold (Au). The curve is certainly high for red wavelengths around 600nm but does not seem to go over 100%.

RGB color is a bit more complicated a subject than readily seems apparent. The reflectance wavelength diagram shows the reason quite well actually.

RGB color model has several central problems:

What the colors represent: They represent 3 spikes in a continuous spectrum. The R, G and B aren't energetically equivalent let alone evenly spaced.
What their range is: The colors do not actually mean anything without information of what space they span. In the assumed sRGB color space the space does not span the entire sensable range. So energetically equal but more vivid colors exist.
Human sensory apparatus isn't actually reading 3 color spikes but the sensors nonlinearly overlap.

As a result one can not draw the conclusion that a reflectance color channel of greater than 1 automatically means that energy is inserted into the system. That is simply one of the possible interpretations.

Another interpretation is that the color is more intense than your color space allows. As a result your color vector component could be over 1.

Human eyes may also bleed color from one sensor to another due to overlap of the sensors. Such things happen with the sky which seems light blue, but is actually far darker blue but so intense that we see it as light blue. But in 50% reflections it would look wrong if we wouldn't account for this.

In the end, it can also mean energy is inserted into the system. Either the energy comes from elsewhere or is generated by the surface.

Rendering is often not a scientific measure of things. No energy principle needs to be broken to achieve this.

Summa summarum (tl;dr)

Color is often a compound attribute at the same time as it measures energy levels it also measures something other. Namely the location in the color space. Thus you can not differentiate the two signals (energy and color intensity) easily.

In this case it is a more intense color because the source says so: Outside of sRGB gamut = more intense color than the color space can make.


What remedies are there for percieved color artifacts due to pixel geometry? I have a white line on black background. Because of the pixel geometry the line does not look really white; one edge looks red-isch, the other blue-isch. Other fore- and background yield other sensations. Pixel geometry is striped RGB, DPI is ~90, viewing distance about an arms length.

Is there any software based solution or maybe visual design that makes this less appearant, assuming we know the screen pixel geometry in advance?

In computer typography, the technique for adjusting strokes so that they align with the pixel grid is called hinting.

The color artifacts you are seeing are due to another technique called subpixel rendering.


Is it possible to alpha blend multiple render targets using a specified alpha? Say I have render targets 0 through N and RT 0 happens to have in its fourth component an alpha channel specified by a material or mask or something.

Is it possible to have the hardware compositor blend render targets 1 through N using the alpha of the first render target?

As far as I'm aware there's no way in either DX or GL to re-use RT 0's alpha for all the blending operations.  Unfortunately, it doesn't seem to be something that's supported by hardware.

You can configure different blend modes for each render target, or enable blending for some and disable for others; however, if blending is enabled for a render target, it always uses its own alpha.

There's also a mode called "dual-source blending" (see DX11 doc and OGL doc), which allows you to specify the alpha for blending with an entirely separate output from the pixel shader, not the render target's alpha channel.  However, this mode only works with one render target on current hardware.

So as far as I can tell, the only options for blending several render targets with the same alpha are:

Output the same alpha on all render targets (sacrificing the ability to store other values to the alpha channel as you would in deferred shading, for instance).
Repeat the rendering in a separate pass for each render target, using dual-source blending.
Use UAVs / image load-store to execute the blending in the pixel shader (only works if geometry is non-self-overlapping in screen space, because there's no protection against race conditions; also probably kinda slow).
On hardware that supports it, DX11.3/DX12 Rasterizer-Order Views, NV_fragment_shader_interlock, or INTEL_fragment_shader_ordering (the latter also exposed by AMD GPUs).  These are three names for the same thing: basically a "critical section" in the pixel shader that lets you read-modify-write a texture atomically with respect to other pixel shader invocations. It essentially enables arbitrary programmable blending, but it's probably fairly slow, and only available on recent hardware.


Best way to remesh a quad 3D mesh? I have a quad-based 3D mesh. I want to:

remesh it while keeping the shape the same
the end output should be in quads
ideally it should have a similar number of verts / polys as the input
ideally the remeshing would be random, and output mesh would be distinct from the input

Right now I'm remeshing with Meshlab, which outputs tris. I then use the Tris to Quads function from Blender. The problem is that that function is not perfect, and leaves many tris.

So ideally there'd be a quad-based random remesh tool, but if anyone knows of a better tri-to-quad tool that would also be helpful?

Thank you.

I'd recommend using OpenVDB. It's quite battle-tested.

Use meshToVolume to voxelize the mesh. Then use volumeToMesh to produce a high-quality quad mesh.

meshToVolume voxelizes the mesh into a SDF. volumeToMesh seems to use dual contouring / SurfaceNets.


Calculate the average color of an image I want to calculate the average color+opacity of an image. First I thought about just averaging all ARGB values component-wise, but this got really dark on images with a lot of transparency. I also don't think mixing RGB colors is a good idea in general since they are nonlinear due to Gamma or something. I also had a look at HSV/HSB but this seems a pretty stupid idea.

So, what is a reasonably physically correct way to average the color of an ARGB image? I prefer solutions in Java and using 8 bit per color channel. The solution should also take care that the image may have an arbitrary amount of transparent or semi-transparent pixels.

Averaging RGB should be correct assuming you do it in linear RGB. If your images are in sRGB, you can remove the gamma correction by doing something like the following for each of the R, G, and B components:

float sRGBToLinear(UInt8 component)
{
    float tempComponent = (float)component / 255.0;
    if (tempComponent <= 0.04045)
    {
        tempComponent = tempComponent / 12.92;
    }
    else
    {
        tempComponent = pow((tempComponent + 0.055) / (1.055), 2.4);
    }
    return tempComponent;
}

You can then average together all of the red values in the image, all of the green values in the image, and all of the blue values in the image after they've been run through the above conversion. You can then do the opposite conversion to get back to sRGB:

UInt8 linearRGBTosRGB(float component)
{
    float tempComponent =  0.0;
    if (component <= 0.00318308)
    {
        tempComponent = 12.92 * component;
    }
    else
    {
        tempComponent = 1.055 * pow(component, 1.0 / 2.4) - 0.055;
    }
    return (UInt8)(tempComponent * 255.0);
}

(My apologies if I've messed up the syntax. I'm a C/C++ programmer and don't know the intricacies of Java.)

Note that alpha makes things slightly more complicated. If you're using premultiplied alpha, you can simply apply the above calculations for the average. If you're using straight alpha, you'll want to multiply each of the R, G, and B components by the alpha before doing the averaging.


Determining the Bounding Box values of a glTF file Is there a way when looking at the source of a glTF file that you can determine what the boundingBox values of the object are?

There isn't a bounding box for the whole glTF file, but individual primitives have POSITION accessors that indicate min and max values.  This is specified in the Meshes section, with the following text:

  POSITION accessor must have min and max properties defined.

For a concrete example of this, take a look at the Box sample model, which assigns accessor 2 as POSITION

"meshes": [
    {
        "primitives": [
            {
                "attributes": {
                    "NORMAL": 1,
                    "POSITION": 2
                },
                "indices": 0,
                "mode": 4,
                "material": 0
            }
        ],
        "name": "Mesh"
    }
],

The accessor at index 2 (the third one in the list) looks like this:

"accessors": [
    { ... },
    { ... },
    {
        "bufferView": 1,
        "byteOffset": 288,
        "componentType": 5126,
        "count": 24,
        "max": [
            0.5,
            0.5,
            0.5
        ],
        "min": [
            -0.5,
            -0.5,
            -0.5
        ],
        "type": "VEC3"
    }

In this accessor, min and max are arrays of x, y, z raw position data of the primitive.  This particular sample model is a unit cube, with values ranging from -0.5 to 0.5 along each axis.

To understand how this affects the whole model though, one must walk the tree of scene nodes, and look at how these mesh primitives are being transformed by the nodes.  Also, if there are any animations or morph targets taking effect, they can influence the final positions of these meshes as well.


Difference between relativeOffset and offset in VAO with Direct Access Ext Using The DSA Ext with opengl , API calls to guide opengl to the buffer format are :

glVertexArrayAttribFormat(  GLuint vaobj,
GLuint attribindex,
GLint size,
GLenum type,
GLboolean normalized,
GLuint relativeoffset);

and another one is :

glVertexArrayVertexBuffer(  GLuint vaobj,
GLuint bindingindex,
GLuint buffer,
GLintptr offset,
GLsizei stride);

my question is what's the difference between relativeOffset and the offset in both function calls ? when i set one the results are ok but when you set both your layout will be incorrected and thus the results are wrong.

The offset in glVertexArrayVertexBuffer is the common offset for all attributes bound to that buffer. It's where the vbo data starts.

The offset in glVertexArrayAttribFormat is added to that common offset for that attribute.

This allows you to interleave attributes using a single vbo binding point. and change the vbo binding for those attributes in a single call.

To make an example:

struct Vertex {
    vec3 pos;
    vec3 normal;
}

glVertexAttribFormat(posAttrLoc, 3, GL_FLOAT, false, offsetof(Vertex, pos));
glVertexAttribBinding(posAttrLoc, vertexBindingPoint);
glEnableVertexAttribArray(posAttrLoc);

glVertexAttribFormat(normalAttrLoc, 3, GL_FLOAT, false, offsetof(Vertex, normal));
glVertexAttribBinding(normalAttrLoc, vertexBindingPoint);
glEnableVertexAttribArray(normalAttrLoc);

then to bind the buffer you pass the offset to where you put in where the data was uploaded to, this is 0 in the basic case.

glBindVertexBuffer(vertexBindingPoint, mesh.vbo, mesh.vboOffset, sizeof(Vertex));


What formula or algorithm can I use to draw a 3D Sphere without using OpenGL? I know that there are 3 techniques to draw 3D objects:
(1) Wireframe Modeling and rendering
(2) Additive Modeling
(3) Subtractive Modeling

Am I correct?

What formula or algorithm can I use to draw a 3D Sphere?

I am using a low-level library named WinBGIm from colorado university.

For example, how to draw this:

And, this:

Scratchapixel has a nice tutorial on writing a basic rasterizer here. Also, you could use the projection algorithm here to get the position of the vertices in screen space, then use Bresenham's algorithm or DDA to draw lines in between. If you want to fill them too you can use scanline (you can find it on Wikipedia).

For ellipsoids, you can either just turn them into a triangle mesh, or the approach here might work, although I haven't tried it.

The Z-Buffer algorithm is very straightforward, just calculate the distance from the point to the camera and store that somewhere for each pixel, then make sure that that's less than whatever was already there.


How do I extrude a 2D Bezier curve representation into a 3D solid using Python? Where P is the control point, w is weight.
I plan to extrude this 2d representation into 3d. I have a discrete 2D representation (array which has [n_points, 2, 1] for each sample) of an airfoil that has to be extruded to form a wing along the z axis which has parameters like Span and variation on chord length. In theory, this seems easy, however, in code, I can't find any basis on how to implement it.
I have not found anything regarding this which doesn't use high utility libraries. Any leads or explanations will be highly appreciated.

You can't create a solid by extruding a single curve or set of curves. Even if they are closed. Think about a circle on on the XY plane. If you extrude the circle along the Z axis you have an uncapped cylinder.
Let's walk it back a bit though. If you have a single Bezier curve and extrude it you create a Bezier surface. You can quickly get your head around a Bezier surface if you understand a Bezier curve.
https://en.wikipedia.org/wiki/B%C3%A9zier_surface
You can think of the Bezier surface as a blend through a set of curves. To extrude the curve you simply duplicate your original curve, and set the control points Z value to your extrusion distance. The original curve and new extruded curve create the control points for your surface.
I actually have a Bezier surface example I wrote in Python. The only dependencies are Numpy and Matplotlib. It also uses the Bernstein Basis function imported from another file in the repository. You can play with the control points to see how modifying them effects the resulting surface. There's no error checking so you'll have to know what you're doing.
https://github.com/jrz371/IntroToNurbsPython/blob/master/BezierSurface.py
As for parameterizing the span and chord length you'll have to derive those yourself. I'm unfamiliar with your requirements. Understanding the Bezier curves and surfaces thoroughly should help you resolve that.
Some of the comment suggest evaluating the curve and creating triangles. This is a bad approach. Tessellating a Bezier surface causes you to lose a lot of information. The Bezier surface is smooth whereas a set of triangles is not. For drawing a Bezier surface they are ultimately tessellated for the display, but they are kept in the original Bezier representation for modification.
Bezier curves are also not the best representation for curves and surfaces. BSplines and NURBS are able to represent more complex surfaces and curves. Understanding Beziers is fundamental prerequisite before tackling those.
Good luck!


glClear's result can't be seen when using bindless texture In my program, I use a additional frame buffer to render something into a texture(only a glClear), which I will reference it's handle in next draw command. And draw the texture back to the default frame buffer.
My program almost run like this:

frame buffer create and attached by a GL_TEXTURE_2D texture and other setup
glGetTextureHandle
glMakeHandleResident

bind this frame buffer
glclear(1.0f, 0.0f, 0.0f, 1.0f)

glBindFramebuffer to 0(back to default frame buffer)
ShaderUse//nothing but return texture's color
glUniformHandleui64(location, textureHandle)//witch is the texture above
//if I use normal bind texture, the result will be correct
glDraw*//just draw a quad

Here I got a strange issue, The 2nd glDraw cmd gives an empty result(Back Buffer is an black image). And I also checked the first frame buffer's render texture witch is a pure red image, and it's a correct result. And my GPU is AMD R7 200.
But when I change bindless texture to normal bind texture style, and everything is ok finally.Also I checked in NV GPU-GTX650 with same program, and the result is also ok.

I am sure there is no opengl error occurred in my program, and I also checked GL_ARB_bindless_texture extension is supported by my devices.

So I wonder, am I missing some memory sync or coherent?

I also looked up to the document and I found this:

  (4) Can texture or sampler objects be modified while they are used by
  a
          texture or image handle?

  RESOLVED:  No.  If such changes were permitted, each modification to the
  texture object would require potential synchronization with one or more
  texture/image handles extracted from the texture.  This seems like a lot
  of bookkeeping with little benefit.  One exception to this rule is that
  individual texels of such texture objects may still be updated with APIs
  such as TexSubImage* and by rendering to a framebuffer object to which
  the texture is attached.

One exception to this rule is that
      individual texels of such texture objects may still be updated with APIs
      such as TexSubImage* and by rendering to a framebuffer object to which     the texture is attached.

Does glClear belong to this exception?

This is clearly a bug in AMD's driver, though it's a very understandable one.

What is probably happening is that the driver does not correctly detect that there is a dependency between the framebuffer operation and the use of the texture. It's easy for an API to tell that you've bound a texture that was the subject of an outstanding render target command, since the API sees exactly what texture you're going to use. It can simply keep track of the texture objects recently used in an FBO, and if it sees you render with one of them bound, it issues whatever dependencies are needed to ensure that the operation is complete before the rendering operation.

It's much harder when you're not binding the texture; you're just passing an arbitrary number to the shader. The API has no idea if that number represents a render target.

To verify that this is the problem, insert a glFinish() between the clear command and the draw command that uses the texture. Odds are good that it will go away (or at least be partially mitigated). A glTextureBarrier() call might also fix it. If you're getting the texture handle after attaching the texture to the FBO, you might also want to get the handle first. Just in case that shakes something loose.

Note that you shouldn't have to do any of these; the specification requires that things work exactly as you expect.

What AMDs driver needs to do is to realize that you're using bindless texturing and emit the appropriate dependency operations when you change framebuffer render targets. That is, it has to assume that any glBindFramebuffer or any calls that modify the FBO that's currently bound will result in any subsequent rendering command reading from previously written textures. Therefore, it must issue a synchronization at the first rendering command that involves a bindless texture operation (if the texture used by the FBO is made resident).

Either that, or it somehow has to figure out which arbitrary integers represent which textures. This is very difficult, since you can pass those handles in a variety of ways. UBOs, for example; you don't want implementations combing through images.


Join two bezier curves so that the result is two-times continuously differentiable I have a task to join two bezier curves, so that the resulting curve is two-times continuously differentiable.
I have the cubic bezier Curve C with control points:
c0 = (1,1)
c1 = (3,4)
c2 = (7,5)
c3 = (8,2)
I shall continue this curve C with Curve D from control point c3 to a control point d3 = (12,1) so that this curve is two-times continuously differentiable.
First Task: Determine control points d0, d1, d1 for the new curve.
Second Task: Specify a pecewise defined formula for the new curve G(v) with v out of [0,1] that passes through c0,c3 and d3. Thus connect curves C and D in v = 1/2.
Third Task: Prove by calculation that the transition between C and D is two-times continuously differentiable.
Regarding first task: I don't know how to determine the points. Can someone help to do this? The rest of the tasks is then maybe something easier to do for me.

For 2 bezier curves to connect and be continuous at all, the last control point of the first curve (C in your case) must be the same as the first control point of the next curve (D). So c3 has to equal d0. That gives you C0 continuity or positional continuity.

The next step is to make sure that the tangents where the 2 curves meet are the same. That will ensure c1 continuity. (That is continuity of the first derivatives of the 2 curves.) To do this, the slope must be equal at both points. In order to do that, you need to ensure that the line from c2 to c3/d0 is continuous with the line from c3/d0 to d1.

I don't know of a good way to describe C2 continuity, but this Stack Overflow answer describes it better than I could. That answer should also explain the calculation you need for your proof.


OpenGL Render front and back faces with different shading for 3d modeller I'm working on a simple 3d modeller and I would like to render the back facing triangles a bit darker, to easier differentiate between front and back faces. Like you can see in most modelling applications. How does one achieve is?

Should I enable back face culling and render the primitives and then enable front face culling, rotate the normals by 180 degrees and render the primitives again? Is this a practical approach?

A fragment shader receives the gl_FrontFacing system-defined input value. It will be false if the fragment was generated from a back-facing triangle, and true otherwise. So you can use that in your shader to determine how to generate a color value.


Different result with DSA (vertexArrayVertexBuffer, vertexArrayAttribBinding) than with glBufferData I am trying to draw a triangle.  Here is my code:
    const char* vertexShaderSource = "#version 330 core\n"
        "layout (location = 0) in vec3 aPos;\n"
        "void main()\n"
        "{\n"
        "   gl_Position = vec4(aPos.x, aPos.y, aPos.z, 1.0);\n"
        "}\0";

const char* fragmentShaderSource = "#version 330 core\n"
    "out vec4 FragColor;\n"
    "void main()\n"
    "{\n"
    "   FragColor = vec4(1.0f, 0.5f, 0.2f, 1.0f);\n"
    "}\n\0";

GLfloat vertices[] = {
    -0.5f, -0.5f, 0.0f,
    0.5f, -0.5f, 0.0f,
    0.0f,  0.5f, 0.0f
};

// Create the VAO
GLuint vao;
GLuint buffer;

glGenBuffers(1, &buffer);
glBindBuffer(GL_ARRAY_BUFFER, buffer);
glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), vertices, GL_STATIC_DRAW);

glGenVertexArrays(1, &vao);
glBindVertexArray(vao);
glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 3 * sizeof(float), (void*)0);
glEnableVertexAttribArray(0);
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindVertexArray(0);

This results in the following image:

However, trying to use newer, DSA type methods does not draw the correct triangle.  If I switch my code to:

glCreateBuffers(1, &buffer);
glNamedBufferStorage(buffer, sizeof(vertices), vertices, 0);

glCreateVertexArrays(1, &vao);
glVertexArrayVertexBuffer(vao, 0, buffer, 0, sizeof(GLfloat) * 3);
glVertexArrayAttribBinding(vao, 0, 0);
glEnableVertexArrayAttrib(vao, 0);

I get this image:

The triangle is squished.  In fact, changing any of the coordinates of the last row of vertices (the point that is squished) do not change the image result.  Why is this happening?

You forgot to call glVertexArrayAttribFormat:

glCreateBuffers(1, &buffer);
glNamedBufferStorage(buffer, sizeof(vertices), vertices, 0);

glCreateVertexArrays(1, &vao);
glEnableVertexArrayAttrib(vao, 0);
glVertexArrayAttribFormat(vao, 0, 3, GL_FLOAT, GL_FALSE, 0);

glVertexArrayVertexBuffer(vao, 0, buffer, 0, sizeof(GLfloat) * 3);
glVertexArrayAttribBinding(vao, 0, 0);


Shadow Mapping artefacts I've been playing around with shadow mapping in OpenGL using depth textures. The depth texture is fine and I can map it onto the scene but I have some strange artefacts on the back of the object:

My question is what is causing this and how can I fix it?

The fragment shader I'm using is fairly straightforward (I stripped out the colour for simplicity in case you're wondering why there's no blue here):

in vec4 vShadowCoord;

uniform sampler2DShadow shadowMap;

out vec4 fragColor;

void main()
{
    float bias = 0.005;
    float visibility = 1.0;
    if (texture(shadowMap, vec3(vShadowCoord.xy, vShadowCoord.z / vShadowCoord.w)) < (vShadowCoord.z - bias) / vShadowCoord.w)
        visibility = 0.25;

    fragColor = vec4(visibility);
}

Edit: As requested, minimum working example screenshot that uses only the above code (no colour).

This issue looks like standard shadow map acne artifacts. Additionally your's lighting equation is incomplete or wrong. Light shouldn't influence faces with normals facing away from it. This also means that with a proper equation the "dark" side of the sphere shouldn't have any acne artifacts.

There are three sources of acne artifacts:

First acne source is shadow map precision. Make sure that near and far planes of shadow casting light are as tight as possible. All objects before the near plane can be pancaked, as their exact depth isn't important.
Second acne source is shadow map resolution. For a directional light you should be doing cascaded shadow maps with at least 3 1024x1012 cascades for ~100-200m of shadow distance. It's hard to cover similar shadow distance with one shadow map with uniform projection.
Third acne source is wide shadow map filter like PCF, as using a single depth depth comparison value across a wide kernel is insufficient. There are many methods to fix it, but none of them is robust.

To sum up, a tight frustum with a few cascades and some bias tweaking is enough to get the general case working (directional light). Start tweaking by disabling shadow filtering and tweak for filtering only when basic shadow maps are robust enough.

Additionally apart from the constant depth bias (which you currently use) you should also add slope depth bias and max slope depth bias. Both can be be implemented either as render state or as shader code during shadowmap rendering. Slope depth bias is simply a magic bias value scaled by dot( normal, lightDir ).

There are some additional interesting methods and most of them are implemented in the excellent demo: Matt Pettineo - "A sampling of shadow techniques".

Flip culling during shadow map rendering (trades acne for Peter panning).
Normal offset shadow mapping does wonders for bias issues, but requires to have vertex normals around during shading.
Variance based methods (ESM, VSM, EVSM) completely remove bias issues, but have other drawbacks (light leaking and/or performance issues).


Correct post-displacement normal calculation (Y-component and epsilon) I often use a 2D Perlin noise texture to displace a grid in the vertex shader (e.g. terrain, ocean).  However, every time I google the method to calculate post-displacement normals there is a slight difference.  For example, the value to place in the Y-component varies depending on the source.  In this example, they place a '1' in the Y-component prior to normalization:

Real-Time Parametric Shallow Wave Simulation | Intel® Software

Whereas in these codes, the value '2' is chosen:

Terrains, Normals, and a Heightmap | dian-xiang.com/blog

What gives?  Is there a definitive source for calculating normals from a height map?

Might this be related to unprojecting mouse clicks?  It always bothered me that you could throw a '0.5' in the Z-component, as mrdoob mentioned here:

Three.js Projector and Ray objects | Stack Overflow

Or use a '1' in the Z-component as Anton does here:

Mouse Picking with Ray Casting | Anton's OpenGL 4 Notes

Is there a connection between calculating normals from height maps and unprojecting mouse clicks?

The second part of my question concerns choosing an epsilon to sample the height map.  Before I learned about height maps, I read about getting the normal from an implicit surface as in this link:

Fractals, computer graphics, mathematics, demoscene and more | Inigo Quilez Blog

It seems like the epsilon number is just made up though.  In my projects, I typically just fudge the value until it looks good.  Is there a definitive source, or at least a good rule of thumb?

The Epsilon is chosen to hit the appropriated Nyquist-Frequency avoiding aliasing. It refers to the highest frequency in your heightfield function. You'll need well knowledge of the definition of your function to find it. For example, the figure shows an iterated Brownian motion (IBM) which approximate Perlin noise. The frequency of each added noise function is given by . So the Epsilon would be the last added frequency: . That means the Epsilon is predefined for each Noise.

IBM by Hugo Elias approximating Perlin Noise.

  Might this be related to unprojecting mouse clicks? It always bothered me that you could throw a '0.5' in the Z-component, as mrdoob mentioned here:

Your first question has nothing to do with mouse picking in my opinion. Your mouse is defined in normalized-device-coordinates (NDC) meaning a click in a window is ranging from -1 to 1 in each direction. The Z component is not given since our screen is just 2-dimensional, meaning we have to select a value from -1 to 1. -1 refers to the near plane and 1 to the far plane.

For further information on that topic:
https://www.youtube.com/watch?v=DLKN0jExRIM&t=371s


How to pass only the needed portion of the data calculated in a fragment shader to another fragment shader? I have an image and I want to draw an arbitrary number of objects on that image. The center coordinates of those objects are calculated inside a fragment shader after series of image processing operations. However, because fragment shader gets executed per-fragment, I cannot draw objects centered on those points in the same shader in a reasonably fast way. Somehow, I need to pass those center coordinates that I calculated in my fragment shader to another fragment shader in a second render pass, preferably as uniforms. Then I can use those coordinates in the second fragment shader to draw my objects in an efficient way.
I thought maybe I can render those points as white spots on a black texture, and after the first pass, I can search the raw data of the texture on the CPU side, then add those white points' coordinates into an array, and send them as uniforms. However, I guess it would be very inefficient as it is a very sparse data (a lot of black pixels and very few white pixels), and I would be doing thousands of iterations (even for a simple 720p image) on that raw data. So, I'm not sure if this should be the way to go.
How can I do this in an efficient way, in OpenGL ES 2.0?
P.S. I'm new to this area, please enlighten me if I made no sense.

I'm assuming that you don't know how many objects there will be or have any way to index them until the image processing operations are completed, i.e. you have some final pass that determines for each pixel "do I want to put an object here, or no".
OpenGL ES 2.0 is pretty limiting. There may be no better way to do this than what you described: output from the fragment shader to a texture, pull it back to the CPU to extract the points, then put those into a uniform buffer for the second pass.
One thing you could attempt, though I'm unsure if it could be fully made to work in ES 2.0, is to do a parallel prefix sum over the output texture from the first pass. This would involve recursive fragment shader reduction passes (a little bit like generating mipmaps) that would first count up the number of objects within each region of the image, starting with 2×2 regions and doubling at each step until you've covered the full image, then go back down the mip chain and calculate the prefix sums, which are the number of objects "before" each region (using some defined ordering). At the end of that, you would have the exact index where to store each object into an output buffer.
FYI, in GL ES 3.1 and other more fully-featured APIs, there are shader atomic counters, which allow implementing an "append buffer"; your first fragment shader pass could output directly to this buffer, then turn around and use it for an input buffer, or for indirect draw parameters, on the next pass.


Is a constant condition more costly than switching shaders? In general, branching in shaders is not a good idea. But now I have a shader with a condition that is constant with respect to the entire draw call. So the branch that is executed is always the same for one draw call.

Is such kind of branching still more costly than having multiple shaders without these branches and switch between them?

On modern hardware if all invocations in a group follow the same path then the unused path doesn't get evaluated.

in pseudo code:

if(cond){
   res = ...
}else{
   res = ...
}

becomes

if(anyInvocationARB(cond)){
    res1 = ...
}
if(anyInvocationARB(!cond)){
    res2 = ...
}
res = cond?res1:res2;

Where anyInvocationARB will be true if any invocation of the shader will have true as cond (from the opengl extension ARB_shader_group_vote).

If cond is derivable from uniforms alone then the driver may optimize and evaluate the condition before starting the render and replace the if with a goto to the correct branch. OpenGL has a feature called uniform subroutines that makes it explicit.


Can you batch vertex data and still sort by depth? In order to reduce the amount of draw calls per frame I've decided to use a vertex array object to store vertex data from several different meshes. More precisely, I use the vao to the define the vertex format (attributes) and the buffers which contain the vertex and index data.

Now I could call glDrawElements on the batch's vao once and draw all the stored vertex data directly. But I also see a lot of discussions about sorting your draw data front-to-back for opaque objects (in order to reduce overdraw) and back-to-front for translucent objects (in order to blend 'correctly').

I was wondering how you would go about sorting the contents of several vertex batches (vaos) even though a batch most likely contains data for objects at different distances from the camera.
Say batch A contains objects with depth { 1, 3, 5 } and batch B contains objects with depth { 2, 4 }, how could this be sorted properly so that it can be drawn in the order { 1, 2, 3, 4, 5 } without specifying 5 separate draw calls (with each a different buffer offset).

Were it just the overdraw optimization I would probably just say screw it as it would allow me to draw an entire buffer's contents in one go (instead of having a draw call with a different offset for each object), but then there's still the sorting of translucent objects.

Thanks in advance!

Drawing back-to-front is not optional with blending (unless you use an order-independent blending technique), so getting the visual effect you actually want has to trump performance.

But drawing front-to-back is purely a performance thing. Specifically, it makes it possible for early-Z to save you fragment processing time, as earlier objects are in front of later ones. So it's a GPU performance improvement.

Batching is a performance improvement for the CPU, in that you're spending less CPU time doing state change stuff.

So you need to decide which is more important: CPU performance or GPU performance. With deferred rendering, saving FS computation time also means saving memory bandwidth.

But the general construction of your scene, what you're actually drawing, can help inform which is more important.

Oftentimes what happens is that you render static terrain first, perhaps using your visibility system to order what you render coarsely front-to-back. Then you render your dynamic objects. Because dynamic objects usually require a lot of state changes, you won't necessarily draw all of them in a single batch (though there are techniques you can use to hide even texture changes).

In any case, the point is that there is a tradeoff. And the one you pick depends on exactly what you're rendering. A top-down game like StarCraft or DotA would never bother with front-to-back sorting; you just render the terrain after the characters. A tight-quarters FPS by contrast would probably really need to employ such techniques, particularly for terrain.


Normal Map Storage (Not Unit Length) Looking at various assets (models / textures / scenes) available online, I have noticed a trend that conflicts with my assumptions. Many of the normal maps contain texels whose vector length deviates from unit length by a significant amount.

For example, this image from Wikimedia Commons (https://commons.wikimedia.org/wiki/File:Normal_map_example_-_Map.png) contains many vectors (hundreds) that are around 15%+ shorter than unit length, and it is hardly an outlier (in fact, it would seem to have been chosen as an exemplary basic example). In other cases, I have found normal maps where the z component is uniformly set to 1.0.

On the specification side of things, I note that GLTF 2.0 merely states (https://github.com/KhronosGroup/glTF/issues/993) that a shader implementation should renormalize after sampling, but that it is not required for the maps themselves to have vectors of unit length.

It seems to me that this poses an accuracy issue. Certainly if normals are sampled using bilinear filtering, the result will generally be shorter than the texels used to generate it, and thus renormalization will be necessary. However, if this is the argument as to why it is not necessary to store as unit length vectors, then it seems short-sighted to me.

When sampling with e.g. bilinear filtering, if the texels are not all of the same length, the resulting normal will be biased in direction towards the samples with greater length. Therefore, even if you renormalize after sampling, you will not be able to recover the accuracy lost by not starting with unit normals in the first place.

So I would like to know why such low priority is seemingly placed on generating normal map textures that contain only unit-length normals. Is it just because the loss in accuracy isn't deemed to be significant enough subjectively/perceptually, or is there something else completely here that I am missing?

Maps with z = 1.0 everywhere are likely partial derivative maps. This is a variant on normal maps that's become fairly widespread. There are a few reasons to prefer it, but probably the most important is it lets you store the map as just two channels (red and green), which allows using better quality BC5 compression, while being cheap to decode in the shader. The normals in such a map are of necessity not all unit length, and as you pointed out that will affect interpolation and blending slightly, but this is generally deemed an acceptable trade-off.

However, if it's purporting to be a traditional old-school normal map and it has pixels that are significantly off from unit length, that sounds odd and probably unintended. I'm not sure why that would have happened in the particular Wikimedia .png example you linked. However, non-unit-length normals could generally be introduced by doing filtering/blending/compositing operations on the image without renormalizing after (for example, if a normal map was rendered at a higher resolution and then downsampled using a standard image resampling algorithm). They could also be introduced by lossy compression.


Will sRGB color space get replaced by wider gamut in future, will sRGB which is a common standard monitor color space nowadays be taken the place by the wider gamut color spaces?

since wide gamut monitor keep publishing in the market and may be affordable for anyone one day, will the majority change to use the wider gamut in like coming 20 to 30 years?

or stay with sRGB and not worth to change.

Displays are already moving towards wider gamuts. I believe that newer iPhones, for example, are all P3 displays. So there is no doubt that sRGB usage will be replaced by wider gamuts in the future.

There are really 2 important questions here:

Which gamuts will people end up using?
When can/should we move our apps to using wider gamuts?

The answer to the first question will depend on the market. People have been pretty happy with sRGB for a few decades. P3 is better, but most consumers probably can't tell the difference. There are much wider gamuts like ACEScg, Rec. 2020, etc. Will it be affordable to move to those? If it is, will consumers care?

The answer to the second question depends entirely on what your target audience is using. If you can guarantee they're using devices with P3 displays (say, if you require the iPhone X's IR sensor), then you should move to P3. If your users are high-end users creating output for professional use, it's probably wise to move to P3. If your users need to create content that looks reasonably good on end-user devices like web pages rendered on aging machines, then it is probably a waste of effort to move away from sRGB.

Note that there is a cost to moving to a wider gamut. You will need to do significantly more work to ensure that all of your processing is done in the right color space because input images to your app will likely come in a variety of gamuts and you now have to convert them all to your working space, process them, and  then convert to the output space. That's not cheap to do and not easy to get right.


Is there ongoing research on Markov Chain Monte Carlo algorithms (other than Metropolis-Hastings) for use in ray tracing? Metropolis Light Transport is an application of the Metropolis-Hastings algorithm. Its variants differ only in the used "mutation strategy".

Is there ongoing research on other Markov Chain Monte Carlo algorithms (with faster convergence rates) for use in ray tracing?

If so, I'm interested in any paper.

Overview

Here is a short overview of the most used space representations, MLT variants and mutation strategies for these MLT variants. As you can see, there are quite some papers dating back to 2017 (e.g., three papers explore combining the Path Space and the Primary Sample Space by jumping back and forth between the two).

Path Space (PS) representation, Metropolis Light Transport (MLT)

Veach E., Guibas L. J.: Metropolis Light Transport. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (New York, NY, USA, 1997), SIGGRAPH ’97, ACM Press/Addison-Wesley Publishing Co.
Veach E.: Robust monte carlo methods for light transport simulation. Stanford University PhD thesis, 1997.
Van de Woestijne J., Frederickx R., Billen N., Dutré P.: Temporal Coherence for Metropolis Light Transport. Eurographics Symposium on Rendering - Experimental Ideas & Implementations 2017.

[Example implementation available in Mitsuba]

Primary Sample Space (PSS) representation, PSSMLT

Kelemen C., Szirmay-Kalos L., Antal G., Csonka F.: A Simple and Robust Mutation Strategy for the Metropolis Light Transport Algorithm.

Extended Primary Sample Space (EPSS) representation, Multiplexed MLT (MMLT)

Hachisuka T., Kaplanyan A. S., Dachsbacher C.: Multiplexed Metropolis Light Transport. ACM Trans. Graph. 33, 4 (July 2014).

Note that MMLT improves upon PSSMLT.

[Example implementation available in pbrt-v3]

Manifold Exploration MLT (MEMLT)

Jakob W., Marschner S.: Manifold Exploration: A Markov Chain Monte Carlo Technique for Rendering Scenes with Difficult Specular Transport. ACM Trans. Graph. 31, 4 (July 2012).
Jakob W.: Light Transport On Path-Space Manifolds. Cornell University PhD thesis, 2013.

[Example implementation available in Mitsuba]

Half Vector Space representation, Half Vector Space Light Transport (HSLT)

Kaplanyan A. S., Hanika J., Dachsbacher C.: The Natural-constraint Representation of the Path Space for Efficient Light Transport Simulation. ACM Trans. Graph. 33, 4 (July 2014).
Hanika J., Kaplanyan A., Dachsbacher C.: Improved half vector space light transport. In Computer Graphics Forum (2015), vol. 34, Wiley Online Library.

Note that HSLT is more general than MEMLT: MEMLT is an optional add-on mutation for MLT, whereas HSLT replaces the complete Path Space representation.

Gradient Domain representation, GDMLT

Lehtinen J., Karras T., Laine S., Aittala M., Durand F., Aila T.: Gradient-domain Metropolis Light Transport. ACM Trans. Graph. 32, 4 (July 2013).
Manzi M., Rousselle F., Kettunen M., Lehtinen J., Zwicker M.: Improved Sampling for Gradient-domain Metropolis Light Transport. ACM Trans. Graph. 33, 6 (Nov. 2014).

Note that there are variants for path tracing and bidirectional path tracing exploiting the Gradient Domain representation as well.

Path Space <-> Primary Sample Space representations

Reversible Jump MLT (RJMLT)

Bitterli B.: Informed Choices In Primary Sample Space. Master’s thesis, Swiss Federal Institute of Technology Zurich, 2015.
Bitterli B., Jakob W., Novák J., Jarosz W.: Reversible Jump Metropolis Light Transport Using Inverse Mappings. ACM Transactions on Graphics 37, 1 (Oct. 2017).

Charted MLT (CMLT)

Pantaleoni J.: Charted Metropolis Light Transport. ACM Trans. Graph. 36, 4 (July 2017).

Fusing State Spaces

Otsu H., Kaplanyan A. S., Hanika J., Dachsbacher C., Hachisuka T.: Fusing State Spaces for Markov Chain Monte Carlo Rendering. ACM Transactions on Graphics (Proc. of SIGGRAPH) 36, 4 (2017).

Genetic operators in Path Space and Extended Primary Sample Space, Genetic MLT (GMLT)

Delabie M.: Genetic Operators in Metropolis Light Transport. Master’s thesis, KU Leuven, Belgium, 2018.

Note that the Path Space is represented via 3D path vertex positions instead of directions to facilitate crossovers (compared to MLT).

Post Scriptum

  Its variants differ only in the used "mutation strategy".

Different mutation strategies and space representations are used. Furthermore, GMLT replaces and extends Markov states representing single paths to complete path populations (Evolutionary Monte Carlo, Evolutionary Markov Chain Monte Carlo <> Markov Chain Monte Carlo).

Some references to Evolutionary Monte Carlo:

Liang F., Wong W. H.: Evolutionary Monte Carlo: Applications to Cp Model Sampling and Change Point Problem. STATISTICA SINICA 10 (2000).
Liang F., Wong W. H.: Real-Parameter Evolutionary Monte Carlo With Applications to Bayesian Mixture Models. Journal of the American Statistical Association 96, 454 (2001).


Problem on understanding gamma correction I am struggling understanding gamma encoding/decoding.

So the colors I see and the values I get from a color picker application(like MS Paint) are already in sRGB space(correct?).

Assume the color picker gives color C1=(186,70,2). If I create an image programmatically with all pixels in color C1 and Do Not apply gamma encoding C1^(1/gamma), and save it as ppm file, I see exactly same color C1 on opening the file on a viewer.

But this is not the case in my experiment. I save values of color C1 into the file and see color C1. And this is what I don't understand: if the display raise the input color C1 by gamma why do I still see C1 on the screen? Should I not expect to get C1^gamma?

UPDATE:

Basically here is the ppm file with all pixel RGB values (128,128,128).
No gamma-encoding has been applied. That is simply writing linear values of 128.

P3
400 400
255
128 128 128
128 128 128
128 128 128
128 128 128
128 128 128
...

According to Wikipedia Gamma correction page:

  For example, when using 24-bit RGB color (8 bits per channel), writing
  a value of 128 (rounded midpoint of the 0–255 byte range) in video
  memory it outputs the proportional ≈ 0.5 voltage to the display, which
  it is shown darker due to the monitor behavior.

Wikipedia clearly says write a value 128 and you will see a darker than you expect. Those values in my ppm file are not gamma-encoded and so I expect seeing
a darker color(0.5^gamma) on the screen. Yet this is not the case and if you open this file on a viewer you will see pixels with brightness half as white.
That is what I don't get.

Update 2 (After Nathan good answer..)

So with input of 186 the screen outputs 50% of white intensity?
And the eye perceives it as 50% as bright as white? is this right?
But if the screen output is 50% in intensity, and eye has its own curve(roughly power of 1/gamma) then the screen output of 50% should be seen brighter?

Another question I have is that,
So is it the programmer responsibility to write gamma-corrected values into the file or the whatever API we use to save image file formats does it internally?
For example i use .Net Bitmap class. Do I need to save bitmap with gamma-encoded values or Bitmap.Save method does it?
I ask this since according Wikipedia, and you mentioned this too, almost all image formats are encoded

  binary data in still image files (such as JPEG) are explicitly
  encoded (that is, they carry gamma-encoded values, not linear
  intensities)

Almost all image formats store colors gamma-encoded, so if you write out those values to a file then that's what you'll see on opening it. The pixel pickers etc operate on the same values stored in the file. It's much less common to store linear-light color values in a file (one example is digital camera RAW files).

When you "see color C1" on the screen, you are seeing C1^gamma. Or rather, you are seeing however that particular monitor renders the RGB value C1, which is at least approximately described by C1^gamma for some gamma. The color you see does not differ between the paint program and the programmatically generated file because they're both sending C1 to the monitor, and the monitor is doing the same thing with it.

Re: your update, you wrote:

  if you open this file on a viewer you will see pixels with brightness half as white

Think about this carefully. How are you judging that the pixels are "half as white"? Are you measuring them with a photometer and finding that they emit half the luminous intensity of a full white pixel? Or are you looking at them and judging that they appear roughly halfway between black and white?

Human perception of brightness is very non-linear. A value that looks perceptually halfway between black and white is physically closer to 20% of the brightness of white, not 50%.

A good way to find a true 50% brightness is to create an image with a checkerboard of black and white pixels, then defocus your eyes slightly, or stand farther away from the monitor, so that the pattern blurs into a uniform grey. That grey is 50% brightness (because it's an average over 50% black pixels and 50% white pixels). You will find that this grey looks considerably brighter than an RGB (128, 128, 128) grey.

For example, open this image in a new tab to be sure it's appearing at 1:1 resolution (browsers resizing it do not necessarily respect gamma):

You will see that the checkerboard in the center looks a lot brighter than the 128 grey on the left, and should be pretty close to the 186 grey on the right. The 186 is encoding a 50% brightness at 2.2 gamma.

Re: update 2, you wrote:

  So with input of 186 the screen outputs 50% of white intensity? And the eye perceives it as 50% as bright as white?

Not quite—the eye perceives a 50% physical intensity as more than halfway between black and white. Maybe around 70% or so. Conversely, the grey that the eye perceives as halfway between black and white is more like a 20% physical intensity.

Actually, gamma-encoded values are a fairly close match to how we perceive brightness, which is convenient (although it's sort of a coincidence; the original reason for gamma was due to how CRT phosphors worked, not anything to do with perception). For a true perceptually uniform color space, you can look at CIELAB, which has a more complicated nonlinear transformation that's designed to mimic average human perception.

  is it the programmer responsibility to write gamma-corrected values into the file or the whatever API we use to save image file formats does it internally?

It's the programmer's responsibility. Almost all image file formats, libraries, and APIs are just passing around gamma-encoded values and do not do any transformations on them.


Why use image load / stores instead of FBOs One example of Image Load / Stores that I have seen could be done using framebuffers instead of Image Load / Stores.  Framebuffers have been available longer in OpenGL and so my question is:

Why would you use Images instead of Framebuffers?

What advantages do they offer?

The referenced example says:

  Note that this demo can also easily be imlemented using Framebuffer Objects. But with Framebuffer Objects you cannot write to arbitrary locations in a texture, though.

I am not exactly sure what is meant here. My understanding is that you can write to whatever pixel coordinates you want in an FBO.  Am I mistaken here?

Why would you use Images instead of Framebuffers?

Image Load/Store is not intended to replace framebuffers. It's a feature that allows (relatively) arbitrary reading and writing to memory. So you would use images when you need to arbitrarily read and write to memory. You would use framebuffers when you're rasterizing primitives for rendering purposes.

The fact that you can build scenarios where you would use the latter in place of the former does not mean that this is the purpose of the feature.

  What advantages do they offer?

Again, it's not meant to replace framebuffers. But the comparison is pretty simple.

A particular FS invocation cannot:

Choose what framebuffer texel(s) to write to. An FS invocation will only ever write outputs to the framebuffer texel(s) that map to its current fragment location. gl_FragCoord is an input value; it is non-modifiable. An FS's outputs will either be discard or written to the framebuffer texel(s) corresponding to gl_FragCoord.xy.

Yes, through point rasterization, you can select which fragment is being generated. But it isn't the fragment shader itself that's deciding this; it's whatever process wrote the vertex to the buffer that it's being read from. That is not under the direct control of the FS invocation itself.
Choose to write to multiple distinct framebuffer texel(s) locations. A fragment cannot cover multiple texels.
Choose which output attachments in the framebuffer to write to. The decision of which fragment shader outputs will be written is made by the write mask state, which the FS has zero control over. An FS can choose to be discarded, but that discards all of its outputs.
Choose to read from the current framebuffer texel(s) it is writing to. Blending is done by a fixed-function stage; the FS has zero control over this. The closest you can get is to employ some texture barrier gymnastics, but that's not something the FS can directly control.
Choose to read from framebuffer texel(s) other than the one it's writing to. Even with texture barrier gymnastics, you cannot read from other people's texels.

Now granted, image load/store doesn't exactly make read/modify/write passes easy, due to incoherent memory access rules.

Of course, image load/store cannot handle (easily):

Depth testing. Or at least, not in any way that is nearly as efficient as the rasterizer. The read/modify/write nature of the depth test makes this exceedingly tricky with image load/store. With a regular depth buffer however, this is trivial.
Simple blending. Again, you can do it, but it's really hard, and far less efficient than the normal rendering process.


How to find silhouette for concaver Given a point light source, a eye position, we can find the silhouette (outline) of the given model. We can first find the silhouette edge, project the model into a 2-D plane,  then traverse all silhouette edges. But when it comes to concave model, some part of the silhouette edges should not be included in the model outline. How to find those parts? Also, can we do this without projecting the model into a 2-D plane?

I'll explain the "geometric way" of solving the problem since that's the solution you seem to be after, instead of using rasterization that Dan suggested in the comments.

You can get the list of potential silhouette edges without 2D projection by first iterating through all edges and taking dot product of the normal of the attached two triangles against the eye->edge vector (any point on the edge will do, e.g. one of the end-points). If the dot products for the two triangles on the edge have different signs, then it's a potential silhouette edge. If the object is convex, then you are done and got the list of silhouette edges.

For concave objects it gets more complicated. For concave objects it could be that an edge is only partially a silhouette edge and can be potentially split to N fragments that all belong to the silhouette. So you need to process the list of potential edges and clip them with the geometry. The way you can do the clipping is to extrude every triangle in the geometry away from eye and use this volume to clip the list of potential silhouette edges. The remaining edge fragments that are not clipped away are your silhouette.


How does the OS + hardware support multiple screen resolutions and multi-display? Modern operating systems support a wide range of resolutions. Mine ranges from 800x600 to 1920x1080. They also support multi-displays. I've never tried it but I have seen other users with 2 or 3 monitors.
I'd like to know some details about how this is all supported. The trivial way is to size the graphics card buffer for the largest possible display, and then crop off any unused parts before sending it out the video port. But a little math, so 1920x1080, 24 bit color, is about 6.2 MB, which is not that big at all for a piece of VRAM. However, then we get to multi-display. What is a reasonable max number for displays? 4? 8? And then don't forget you want to support high frame rates of at least 60 fps, maybe 120 or even 240. Then remember you should support double- or even triple-buffering.
Not to mention, I have seen bigger rez monitors online that advertise themselves as compatible with Windows 7 and 10. So it seems to me that just providing the max possible buffer size is not feasible. The "max" changes too rapidly these days.
TLDR: I can understand if every monitor and GPU comes with its own driver. What I'm trying to understand is how one set of hardware can support a very wide range of screen resolutions without being trivially wasteful. Is there some clever way of doing this?
Disclaimer: I'm a CS guy and don't keep up with modern hardware. I used to read a little about various hardware back in late 90s / early 00s, but never worked on it career-wise or hobby-wise. I can build my own comp and understand basic specs of most parts, but the graphics capabilities are the biggest mystery and "dark arts" to me (other than RAM timings).

Any hardware in the graphics card is fixed.

No, the storage of a graphics card is fixed. How that storage gets used, particularly on any semi-modern graphics card, is flexible.
Essentially, what a graphics display device has is a mechanism for being told, "the bytes of video memory starting at address X is in format Y and size Z; send it at the next vblank period to display device W". Really old graphics cards had specific memory structures for holding framebuffers, which imposed strict limits on how things could work. But that hasn't been the case on PCs for a good decade at least.
There may be limitations on how many display devices a particular piece of hardware could hypothetically output to, or the sizes and formats which the hardware can use. Some hardware devices may only be able to use non-virtualized GPU addresses (which I'd guess is why NVIDIA has special memory types for allocating render targets) or somesuch, but even that is a matter of the user interface, not what the internal hardware is doing. As far as the display device's output is concerned, it could use more or less any GPU memory address.
So, what happens when you plug in a new display device? The OS allocates a framebuffer's (or two) worth of video memory for that display device's video output. If you change the resolution of a display device, the OS allocates a differently-sized slice of memory and frees the prior framebuffer allocation. And so forth.


I need to build my own map renderer I have OSM world data (essentially I processed the XML data into JSON containing arrays of objects (streets, buildings etc.) that each are represented by an array of lat/lon coordinates). Some of these are lines (roads etc.), others are areas that have to be filled (1st and last point close the area).

Anyhow, I would like to achieve an image similiar to google maps or mapbox but much simpler as I do not require city name labels or anything of the sort. For now all I really need is roads.

I need this for a game I am making so the method itself must not be the most elegant one as I probably wont be doing this on the fly but rather prerendering the world to then send the images prerendered to the players.

My question is merely, where the heck do I start? I have no idea how the rendering of bitmaps is done (or whatever other format is best.)

I know C# and NodeJS and am not afraid of learning new frameworks or technologies.

If you have the prerendered images, you can "simply" render them onto a grid of quads. This is done with texturing, i.e. you upload the image to your GPU as a texture and apply texturecoordinates to quads, such that when you render the quads, the textures are applied correctly to their surfaces.

You will however need to learn OpenGL (preferrably with GLSL), Direct3D with HLSL or even Vulkan, all of which will take some time. Depneding on where your goals lie, you may be better of going with a preexisting rendering engine and just learning like Unity or Unreal.

If you do want to learn one of the mentioned Graphics APIs, there are numerous tutorials on the web. For OpenGL I can recommend the youtube tutorials by ThinMatrix, for the other two APIs you'd need to ask someone else. Furthermore, there are splendid books online as well as offline. If you have some money to burn, the basics of rendering are covered in Real-Time Rendering and Mathematics for 3d Game Programming and Computer Graphics. In regards to OpenGL, there are the Online Tutorials of Learn OpenGL, as well as offline books in pdf and epub version.


Why does rotation about an arbitrary axis have degree of freedom 6 I have been trying to figure out why does rotation about an arbitrary axis have degree of freedom 6. I know that it takes two points, each take 3 values to determine in a 3-D space. But what about the rotation angle? Wouldn't that make the degree of freedom increase by 1 to get 7?

The length of the axis is irrelevant and so you can remove one degree of freedom.


Why do games only allow certain resolutions? Only a few games have resizable windows when running in full screen mode. All others have a finite fixed set of allowed resolutions. But why? I see some reasons, but none of them seems to be that important to me that would justify such a user-unfriendly feature.

Example

Let's use Dark Souls II, because it's the first I can think of. The allowed resolutions are:

800x450
1024x576
1152x648
1280x720
1440x810
1600x900
1680x945
1920x1080

What is the reason to not allow the resolutions 800x451, 800x452, ..., 801x450, 801x451, ...? What it I think can't be:

The UI. If we keep minimum and maximum window sizes as limitations, the only issue I can see here is with settings with a weird aspect ratio (1920x450, 800x1080). Otherwise, there shouldn't be any problems with free scaling between the resolutions.
Camera, Viewport. Same as above.

Note that in your Dark Souls example, all the resolutions listed are the exact same aspect ratio.  So the question is, why doesn't it support arbitrary aspect ratios?

The answer is that it costs time and effort to support arbitrary aspect ratios, and if that money spent doesn't translate to increased profit, then it doesn't make sense to do.

This answer covers the 2d layout issues, which includes extra time and effort to create a versatile layout that can be resized so that elements do not overlap, stretch, or pixelate in an undesired manner, and that the overal UI maintains the best user experience.
The developer would have to ensure that the window doesn't get too large or too small.  Does it make the game any better if the user can drag the window size to be smaller than 800 x 450?  probably not.
you have to define the behavior of the camera when the aspect ratio changes.

For example, take the following scene:

If we allow the user to change the horizontal size, we can decide to "zoom out" horizontally so that the same area still shows up horzontally but now we can see more vertically.

Or, instead, we can keep the same zoom level but just clip the viewing area more:

Note, that if you don't specifically handle the changing aspect ratio, the default graphics pipeline behavior will be to stretch the image:

These decisions take time and effort to design, code and test, but management has to decide if it actually makes the user experience better enough to be worth the time.  Apparently, most games have decided its not worth the effort.  I'd even go further and suggest that allowing users to screw up the best layout of the game is a bad feature.


Is BVH faster than the octree/kd-tree for raytracing the objects on a GPU? Apparently most people use BVHs in their raytracers. The ray-AABB intersection code is very fast and and the BVHs also compresses empty space efficiently, working even with infinitely large objects, while octrees (a more uniform subtype of BVH) apparently have a problems with compressing large open worlds - no way you can insert into a finite grid that huge mountain in the distance, without making the grid cells overly large. So what are the best practices?

First, it is important to observe that there are basically two different "classes" of methods for building accelerations structures for ray tracing: 1) space subdivision methods (e.g. BSP, kD-Tree, Octree, etc.); and 2) object subdivision methods (e.g. BVH). While space subdivison methods work by recursively subdividing the space with planes, without sticking to the geometry embedded into that space, object subdivison methods work by subdividing the geometry recursively into smaller and smaller parts, wrapping each part with a, usually tight, volume. More on the basics of space and object subdivision that can be found in the PBRT v3 online book.
Each method (space or object sudivision) has its own pros and cons. In the case of space subdivision methods, since the subspaces do not overlap, we can usually traverse the structure in front-to-back, or back-to-front, order more easily. When a ray is traversing such structure, as soon as it hits a surface, we can stop the traversal. This usually leads to faster traversal schemes. Several software renderers take advantage of the traversal efficiency given by the space subdivision schemes. On the other hand, space subdivision schemes may be more intricate to implement (usually you have to tweak some epsilons) and may lead to deeper trees. Also, they do not like very much dynamic geometry. If the geometry encoded into a space subdivision acceleration structure changes, we usually have to rebuild the acceleration structure from scratch.
Object subdivision methods have quite different characteristics. Since the object is subdivided with volumes, and these volumes may overlap, the traversal is traditionally slower. We cannot, for instance, stop traversing a BVH as soon as a ray finds an intersection with a surface. Since the volumes overlap, we may need to check for potential intersections with nearby primitives before quitting the traversal. On the other hand, it may be easier to implement a BVH because we do not have to split the object parts with planes. Also, BVHs usually generate shallower structures (which may eventually compensate the slower traversal). However, one of the most interesting aspects of the BVHs is that they are dynamic geometry-friendly. If geometry changes (but not much, actually), we can simply locally adjust the size and position of the corresponding bounding volume (i.e. refitting). These adjustments may cause the need to adjust the parent volumes, a procedure that may culminate in a chain reaction that may reach the root node of the BVH. All in all, assuming that we have a reasonably balanced BVH, these operations will be ~O(log n), which is really fast and cool.
The above paragraph describes the traditional characteristics of the BVHs. There is currently lots of research going on on the topic, and several of its ineficiencies have been significantly reduced over time. It should be noted, for instance, that still in 2007, Wald, Boulos and Shirley have designed a very efficient BVH-based acceleration structure, for interactive ray tracing on GPUs, that rivals with space subdivision schemes such as kD-Trees. This work was very influential.
Actually there are several reasons that make the BVHs the preferred acceleration data structure for interactive ray tracing o GPUs. According to Wodniok and Goesele, the main factors that have turned the BVH into a popular acceleration structure in the context of GPU ray tracing are its "memory footprint (...) efficient empty space cut-off, fast construction, and simple update procedure (...) while (...) offering similar (...) performance as kD-Trees". Regarding fast BVH reconstruction, it may be interesting to take a look at the LBVH, by Lauterbach et. al., the HLBVH, by Pantaleoni
and Luebke, and the one by Karras and Aila, that are capable to rebuild BVHs iteractively. Overlapping impacts the traversal procedure efficiency because, when a ray hit the overlap, it has to be tested against the two child nodes. SBVHs, by Stich et. al. is a hybrid structure that employs both spatial and object subdivisions such that overlap is reduced. The cons, according to the paper, are the fact that SBVH does not handle dynamic scenes anymore, and that it may increase memory consumption up to 30%. The SBVH building procedure, also, is not as fast as for the regular BVHs.
The current NVidia RTX GPUs support the traditional BVH, the SBVH and Karras/Aila acceleration structure. As pointed out and detailed by @DirectX_Programmer, in his answer below, these structures are actually combined, by the RTX drivers, into a more flexible arrangement composed by two levels.


Benefits of Clustered Shading vs Tiled Shading? The Unity Engine manual page Best Practice: Setting up the Lighting Pipeline has a section "Render pipelines" with an illustration of Tile Rendering and Cluster Rendering:

The section then mentions:

  Opaque objects will most likely be shaded using the tile system,
  whereas transparent ones will rely on the cluster system.

What is the benefit of using Clustered Shading with transparent objects? Is it related to performance, or are transparent objects not supported by Tiled Shading?

Why opaque objects will "not likely" be rendered with Clustered Shading?

Both Unity's "Deferred shading rendering path" and "Legacy Deferred Lighting Rendering Path" work only for opaque surfaces. They both rely on a very similar set of passes:

Render the opaque objects' lighting parameters to a number of render targets. This is referred to as the "G-Buffer pass" or "base pass".
Lighting is then computed in screen-space using the buffers of the first pass to compute lighting. The main benefit is that the cost of lighting is independent of scene complexity. This allows for much more complicated lighting equations with a greater number of lights.
Full-screen pass to resolve all these buffers to a final color.

The reason this doesn't work for transparent objects is that they must be rendered back to front completely shaded (lighting and all). If transparent object A is in front of transparent object B, B must be completely shaded when A blends its color with B. We need that final shaded color because the lighting parameters accumulated in G-Buffers cannot be blended.

So, if you can use it (there are downsides), deferred shading has some very nice properties that produce very good results with good performance. That is why opaque objects will "likely" use deferred shading. Quoting from Unity doc:

  When using deferred shading, there is no limit on the number of lights that can affect a GameObject. All lights are evaluated per-pixel, which means that they all interact correctly with normal maps, etc. Additionally, all lights can have cookies and shadows.

  Deferred shading has the advantage that the processing overhead of lighting is proportional to the number of pixels the light shines on. This is determined by the size of the light volume in the Scene regardless of how many GameObjects it illuminates. Therefore, performance can be improved by keeping lights small. Deferred shading also has highly consistent and predictable behaviour. The effect of each light is computed per-pixel, so there are no lighting computations that break down on large triangles.

I wasn't able to find much information about Unity's clustered rendering. From what I gather it's an optimization technique for forward rendering, but it's still fundamentally the same process. Here's a very nice write-up of how it works in Doom (go to the "Clustered-Forward-Rendering of Opaque Objects" section).


In a physically based BRDF, what vector should be used to compute the Fresnel coefficient? The well known Schlick approximation of the Fresnel coefficient gives the equation:

$F=F_0+(1 - F_0)(1 - cos(\theta))^5$

And $cos(\theta)$ is equal to the dot product of the surface normal vector and the view vector.

It is still unclear to me though if we should use the actual surface normal $N$ or the half vector $H$. Which should be used in a physically based BRDF and why?

Moreover, as far as I understand the Fresnel coefficient gives the probability of a given ray to either get reflected or refracted. So I have trouble seeing why we can still use that formula in a BRDF, which is supposed to approximate the integral over all the hemisphere.

This observation would tend to make me think this is where $H$ would come, but it is not obvious to me that the Fresnel of a representative normal is equivalent to integrating the Fresnel of all the actual normals.

In Schlick's 1994 paper, "An Inexpensive Model for Physically-Based Rendering", where they derive the approximation, the formula is:
$$F_{\lambda}(u) = f_{\lambda} + (1 - f_{\lambda})(1 - u)^{5}$$
Where

So, to answer your first question, $\theta$ refers to the angle between the view vector and the half vector. Consider for a minute that the surface is a perfect mirror. So:
$$V \equiv reflect(V')$$
In this case:
$$N \equiv H$$
For microfacet-base BRDFs, the $D(h_{r})$ term refers to the statistical percentage of microfacet normals that are oriented towards $H$. Aka, what percentage of the incoming light will bounce in the outgoing direction.
As for why we use Fresnel in a BRDF, it has to do with the fact that a BRDF by itself is only a portion of the full BSDF. A BRDF attenuates the reflected portion of light and a BTDF attenuates the refracted. We use the Fresnel to calculate the amount of reflected vs. refracted light, so we can properly attenuate it with the BRDF and BTDF.
$$BSDF = BRDF + BTDF\\
$$
$$\begin{align*}
L_{\text{o}}(p, \omega_{\text{o}}) &=  L_{e}(p, \omega_{\text{o}}) \ + \  \int_{\Omega} BSDF * L_{\text{i}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}} \\
 &= L_{e}(p, \omega_{\text{o}}) \ + \  \int_{\Omega} BRDF * L_{\text{i, reflected}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}}  \ + \  \int_{\Omega} BTDF * L_{\text{i, refracted}}(p, \omega_{\text{i}}) * \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}}
\end{align*}$$
So, in summary, we use $D$ to get the percentage of light that will bounce in the outgoing direction, and $F$, to find out what percentage of the remaining light will reflect/refract. Both these use $H$, because that is the surface orientation that allows a mirror reflection between $V$ and $V'$