Strange VSync issues on a GTX 1060 I'm out of my wits. I've been working an a voxel game for the last couple of years, written in C# and OpenTK (=OpenGL backend)

I've been struggling to achieve a stable frame rate and one of the major issues I can see after countless hours of frame profiling, is that the windows 10 nvidia graphics driver seems to act in some really strange ways.

It almost seems as if my card is in some sort of low power mode by default in which it renders at arbitrarily changing speeds, but once its put under load it renders reliably again.

I can reliably reproduce this by launching my game with non-demanding graphics setting, then up the GPU load with dynamic shadows, god rays and such, then lowering the demands again.

This is all with VSync enabled (and in the nvidia control panel the vsync option is set to "Let the application decide")

Here are some frame rate graphs:

Low gpu load, draw calls randomly take a few milliseconds longer, then it almost seems to overcompensate by drawing the next frame too fast.

Then I increase the gpu load for a few seconds and lower it again. The frame rate gets buttery smooth:

As mentioned I carefully profiled this issue and it always seems that the time is spend on various draw calls (usually GLMultiDrawElements), but highly inconsistent.

Anyone have an idea what the heck is going on here and how I can fix this?

Is there any call I could make to NVAPI to force it into this hypothetical high power mode?

Ok so setting the "power management mode" to "prefer maximum performance" in the nvidia control panel seems to fix it.

Since I already create an optimus profile automatically through code all I had to do was to set this value as well.


Performance of vector graphics versus bitmap or raster graphics Sometimes I use vector graphics, simply because they look just slightly nicer in some cases, and other times, I use bitmap/raster graphics.

I was wondering, are there any significant performance differences between these two options?

As TheBuzzSaw said, it does depend on lots of things, including implementations of the rasterized graphics vs the vector graphics.

Here are some high performance vector graphics methods that are rendered using traditionally rasterization methods.

Loop and Blinn show how to render a vector graphics quadratic bezier curve by rendering a single triangle, and using the texture coordinates in a pixel shader to say whether a pixel is above or below the curve:
http://www.msr-waypoint.net/en-us/um/people/cloop/LoopBlinn05.pdf

The basic idea is that you set your triangle corner positions to be the 3 control point positions, and you set the texture coordinates at each corner to be (0,0), (0.5,0) and (1,1) respectively.  In your shader, if the interpolated texture coordinate (x*x-y) is < 0, the pixel is underneath the curve, else it's above the curve.

You can see a faux implementation of it on shadertoy here:
https://www.shadertoy.com/view/4tj3Dy

As for the second method, here's a method from Valve, where distances to a shape are stored in a texture, instead of pixel data, allowing vector graphics to be drawn by using texture sampling.  The decoding is so simple, it could be implemented even on fixed function hardware using only an alpha test!
http://www.valvesoftware.com/publications/2007/SIGGRAPH2007_AlphaTestedMagnification.pdf

To give you an idea of how well the second technique works, this 1024x768 mustache image was generated from a 64x32 source image that had a single color channel! (aka 2KB uncompressed)

I also wrote some stuff about it on my blog:
http://blog.demofox.org/2014/06/30/distance-field-textures/

Here is some sample OpenCL code to show just how simple it is:

float alpha = read_imagef(tex3dIn, g_textureSampler, textureCoords).w;
float3 color = (alpha < 0.5f) ? (float3)(1.0f) : (float3)(0.0f);

Both of these techniques are super fast, and blur the line between vector and rasterized graphics a bit.  They are rendered using rasterization techniques, but have zooming / scaling properties like vector graphics techniques.


PBR and Specular Aliasing I have been following LearnOpenGL.com's tutorials on PBR. Everything makes sense and I wrote up a shader for my physically based renderer.

I noticed that the results look great, however all of my metal objects have such a strong specular on the edges thanks to my Fresnel calculation, and it is producing specular aliasing.

See image:

I can't seem to figure out why, or how to fix this. I also uploaded a simplified version of my PBR fragment shader. If someone could take a look, I would really appreciate it.

Simplified PBR fragment shader: https://textuploader.com/dv3h1

Here is the tutorial I am following: https://learnopengl.com/PBR/Lighting
It doesn't seem to have any of these specular aliasing issues.

Thanks!

I'll point out that even if you did implement it correctly, at 1 sample per pixel with low roughness materials you'll still see heavy specular aliasing. It's not uncommon to see the firefly pattern on edges on low roughness materials.
For reference see the Infiltrator demo without TAA

http://advances.realtimerendering.com/s2014/epic/TemporalAA_Compare.mov
http://advances.realtimerendering.com/s2014/index.html#_HIGH-QUALITY_TEMPORAL_SUPERSAMPLING


How to sample 3D points to visualize a B-spline surface? Given that I have implemented a function called calc_bspline_surf() in C, and that function has the following declaration:

calc_bspline_surf(ctrlnet, p, q, U, V, u, v)
/*
  ctrlnet --> control net of B-spline surface
  p, q --> degree of B-spline surface
  U, V --> the knots vector in two directions
  u, v --> parameter pair
*/

So for a pair of parameters {u, v}, corresponding B-spline surface coordinate could be generated via calling function calc_bspline_surf(u, v).

Here is a solution that sampling the 3D points uniformly in two direction

for u = {0, 0.05, ..., 1}
    for v= {0, 0.05, ..., 1}
       calc_bspline_surf(u, v)

However, how many 3D-points I need to sample? Is there classical sample strategy?

Based on your linked question, the problem appears to be that you don't get enough samples in highly curved areas of the surface. You could increase the sample density everywhere, but then you oversample the flatter areas and it becomes too slow.

So, you need to increase the sample density locally in areas where it's needed. A conceptually simple algorithm for this is adaptive refinement. You would start out with a uniform sample grid in UV space. Then, detect grid squares in which the curvature is too high for the grid spacing (above some threshold), and subdivide them, adding additional samples in between the existing ones. This process can be repeated recursively on the new, smaller grid squares, subdividing as finely as the surface requires. You can tune the curvature threshold to balance fidelity versus performance.

Since B-splines (NURBS) are based on rational functions, I would guess that the maximum curvature in a region can be found (or at least bounded) analytically. This would be ideal, if possible, since it would avoid the problem of missing a small high-curvature area if none of the initial grid points happen to land in it. However, in a pinch the curvature can be estimated by looking at the sample points you've evaluated. For instance, if $p_1, p_2, p_3$ are three sample points collinear in UV space, they should be close to collinear as well, if the surface is adequately sampled. So, you could for instance evaluate
$$\frac{\|p_2 - p_1\| + \|p_3 - p_2\|}{\|p_3 - p_1\|}$$
where the double bars mean vector length. If this is greater than some threshold value such as 1.1 or 1.5 (note that 1.0 would be perfectly flat), then you would subdivide the area between these points.


Send Texture to Shader I've been following the LearnOpenGL tutorials and I've come across a piece of code I don't understand.

glUniform1i(glGetUniformLocation(ShaderID, "textureVar"), 0);

In the above code ShaderID is a GLuint which references a shader.

textureVar is a sampler2D in the shader.

I don't understand the last parameter in that line of code. Does the 0 represent GL_TEXTURE0 or what exactly does the last parameter mean?

Yes, the 0 in there represents the texture unit that the sampler references. So in order for a texture to be referenced by textureVar in the shader, it should be bound to texture unit 0 (e.g. by glActiveTexture(GL_TEXTURE0); glBindTexture(GL_TEXTURE_2D, ...) or any other means you can bind a texture to a texture unit.

However, note that it doesn't actually matter which texture unit is active when setting that uniform, nor what texture is bound to that unit. Effectively to the outside world the sampler uniform is just an integer value specifying the texture unit to take its texture and sampler information from. It can be set to that texture unit whenever you want and the texture can be bound to that unit whenever you want (in fact in GLSL 4.20+ you can actually initialize the uniform right in the shader with e.g. layout(binding=0) uniform sampler2D textureVar). All that matters is that at the point the shader gets invoked (i.e. when actually glDraw...ing something) the sampler is set to the proper unit and the texture is bound to that unit.

For more information on the matter, take a look at the corresponding section in the OpenGL Wiki.


Explicit vs Automatic Attribute Binding I'm a beginner in OpenGL and I've been doing some research on the topic of shaders and attributes. Most places I go say that explicit attribute binding (whether it be in the shader itself with layout(location=x) or with glBindAttribLocation) is better than letting OpenGL generate some attribute number. However, I didn't completely understand WHY explicit attribute binding is better.

So I guess my question is, what are the pros/cons to explicit/automatic attribute binding, and which one is more conducive to better program design overall?

It basically gives you control of the semantics of the individual attribute indices. This mostly pays off in bigger projects, where you have many different shaders which, however, often use rather common attributes, like e.g. positions or texture coordinates. With explicit attribute locations you can establish a semantic for those attributes and enforce that on all your shaders, e.g. positions are always in attribute 0, texture coordinates always in attribute 1, ...

First, this has the advantage of not having to query the attribute index for every shader. Of course you could query it once and sotre it in e.g. a map from attribute names/semantics to indices, but establishing this beforehand is even better. And it brings us to a much bigger advantage, without explicit attribute binding you would have to bind vertex arrays of a model to the individual attribute channels not only for each model, but also for each shader, since your position array might go to attribute 0 in one shader and attribute 1 in another one.

This is not only a huge hassle, but also basically destroys the advantage of Vertey Array Objects (VAO), which are specfically there to capture all the vertex attribute state, including the enabled attribute indices and their buffer sources so that you can draw your object by just binding the VAO and calling a draw function. Without explicit attribute binding you would still have to reenable/bind the individual arrays at least for each different shader you use. With explicit attribute binding you know that your model's positions always go into attribute index 0.


Z buffer working in OpenGL After Z-Buffer Testing does the fragment shader run on the discarded fragments?
Does the Z-Test happen after the fragment shader runs?

Please, tell me the step by step events that occur in the Z-Test.

Also what is early Z-Test and how to achieve it?

When depth testing is enabled, the testing occurs after the fragment shader executes. The procedure is relatively simple (and spelled out pretty well here):

Get depth of fragment
Test against depth buffer
Discard if it fails

Early-z discard is when a rasterizer discards a fragment before a fragment shader runs because it's clear early on that it won't pass the depth test. However, the rasterizer can't safely do this if certain things happen (such as calling discard() or modifying the fragment depth in a fragment shader). This is a hardware optimization and isn't required to occur unless you force it with OpenGL 4.2 or with an extension.

There are things that can prevent early-z discard from occurring. To achieve it:

Don't use discard() in a fragment shader
Don't modify the depth of a fragment in a fragment shader
Don't use alpha testing (deprecated)
Make sure you have depth testing turned on
Make sure you have a depth buffer


Primary ray directions (MC Path tracer) Do primary rays conform to the frustum? If they're randomly directed, is the randomness fitted to some distribution (as it would be for a diffuse BRDF? I'm building my existing raymarcher into a monte carlo path tracer and I'm really unsure about how to orient my rays before tracing actually starts.

Yes, in the simple case, primary rays conform to the frustum. If you're doing depth-of-field optically, then the rays don't quite conform to the frustum, because you need to vary the ray origins slightly as well as the directions. How exactly the variation works depends on how closely you're simulating the lens system and aperture. You can picture it as sampling from several frustums, one for each pinhole camera you're averaging.

I've never done full Monte Carlo for ray sampling: generating the rays according to a uniform distribution is bad at distributing them usefully around the image. Popular options include:

You can generate the ray directions according to a space-filling curve on the image plane, and then adaptively supersample once you have enough samples to estimate contrast.
Start with a regular grid that goes through the centre of each pixel, but randomly jitter the directions within each pixel. This was popular for many years but (done naively) it's still slightly biased because you're too likely to sample the corners of pixels.
Start with a uniform random sampling but use Poisson discs to avoid putting rays too close together. This is closer to a pure random sampling, and it has the advantage you can keep using the technique as you refine/supersample the image by decreasing the disc radius over time.


Convert RGB image to YUV420 using a hlsl shader I am new to shaders and I need to convert efficiently an RGB image to YUV420.

I can convert the RGB image to YUV420 with a simple c++ program that uses the

long length = m_width * m_height;

long yuv420FrameLength = (float)length * 1.5;
unsigned char *yuvdest = (unsigned char *)malloc(yuv420FrameLength * sizeof(unsigned char));

int r, g, b, y, u, v, ypos, upos, vpos;

for (int j = 0; j < m_height; ++j) {
    for (int i = 0; i < m_width; ++i) {
        r = (int)m_pRGBImage[(j * m_width + i) * 3 + 0];
        g = (int)m_pRGBImage[(j * m_width + i) * 3 + 1];
        b = (int)m_pRGBImage[(j * m_width + i) * 3 + 2];

        y = ((66 * r + 129 * g + 25 * b + 128) >> 8) + 16;
        u = ((-38 * r - 74 * g + 112 * b + 128) >> 8) + 128;
        v = ((112 * r - 94 * g - 18 * b + 128) >> 8) + 128;
        ypos = j * m_width + i;
        upos = (j / 2) * (m_width / 2) + (i / 2) + length;
        vpos = (j / 2) * (m_width / 2) + (i / 2) + length + (length / 4);

        yuvdest[ypos] = y;
        yuvdest[upos] = u;
        yuvdest[vpos] = v;
    }
}
std::ofstream myfile;
myfile.open(pPath, std::ios_base::trunc);
myfile << yuvdest;
myfile.close();

I can transform the pixels and create the YUV with the following shader code (I did not test it, if I'm wrong, please tell me).

sampler2D sampler0            : register(s0);

struct VS_INPUT
{
  float4 Position   : POSITION;
  float2 Tex0       : TEXCOORD0;
};

struct VS_OUTPUT
{
  float4 Position   : POSITION;
  float2 Tex0       : TEXCOORD0;
};

struct PS_OUTPUT
{
  float4 Color   : COLOR0;
};

PS_OUTPUT psmain(in VS_OUTPUT In)
{
  PS_OUTPUT Out;
  Out.Color = tex2D(sampler0, In.Tex0);
  int r = Out.Color.x;
  int g = Out.Color.y;
  int b = Out.Color.z;
  float y = ((66 * r + 129 * g + 25 * b + 128) >> 8) + 16;
  float u = ((-38 * r - 74 * g + 112 * b + 128) >> 8) + 128;
  float v = ((112 * r - 94 * g - 18 * b + 128) >> 8) + 128;;
  return Out;
};

VS_OUTPUT vsmain(in VS_INPUT In)
{
  VS_OUTPUT Out;
  Out.Position = In.Position;
  Out.Tex0 = In.Tex0;
  return Out;
};

The thing that is missing to me is how to arrange the bytes formation correctly as the C++ code:

ypos = j * m_width + i;
upos = (j / 2) * (m_width / 2) + (i / 2) + length;
vpos = (j / 2) * (m_width / 2) + (i / 2) + length + (length / 4);

The problem here is that you're not storing the YUV values of a pixel at the same place in the result image. What you seem to be doing is first storing the full size Y image and after that the quarter-sized U image and then the quarter-sized V image after that (also with a strange gap of half the image size between the Y and the UV data).

This doesn't work when trying to map it to a standard pixel shader approach, since a pixel shader outputs the value of exactly 1 pixel in the output image, but you actually would need to write 3 different scattered pixels in the output image, at least as your image data is currently laid out in memory.

So there's a few ways you could get around that:

You could output the YUV values directly into a full-sized YUV image (with the Y, u, and v values of a single pixel stored together in the RGB channels of the output) and then further compress this into the disparate memory layout you need, either on the CPU or in another pixel shader that just decides what channels to read from the Yuv image based on the pixel position inside of the output image.
You could combine the above two shaders into one, doing either the Y, the U or the V computation, based on which quarter of the output image you're in.
You can switch to a different layout for your YUV data, e.g. storing the Yuv values of a 2x2 pixel block of the input image in one compressed pixel of the output image. Then it might actually be useful to write into an unsigned integer image and do the converstion to bytes and the compression yourself in the shader.
You can use your C++ algorithm and write the data in a scattered fashion, but not using a pixel shader rather than a compute shader and writing into a generalized buffer/image instead (or combining the two and writing into a generalized buffer/image rather than the frambuffer directly from the pixel shader).

However, I left the above explanations decidedly vague and general for further studying, since there are a few more direct practical mistakes in your actual shader code (e.g. Out.Color should contain your output values, you're also computing with float, so they're likely in the [0,1] range and you shouldn't do all these integer operations on them), which indicate that you might first want to delve a little deeper into how shaders and HLSL work.


Vulkan: efficient way to update VBO mesh vertices? I would like to change/add/remove some mesh vertices in existing VBO's (while deforming and remeshing 3D objects) . With OpenGL I would have to reload whole vertex VBO's. I suppose with Vulkan there could be a way to directly access VBO's to only change/add/remove the changed vertex data. Is there some sample code out there demonstrating how this can be achieved?
Many thanks for any help.

With OpenGL I would have to reload whole vertex VBO's.

No you wouldn't.  You can use glBufferSubData

  I suppose with Vulkan there could be a way to directly access VBO's to only change/add/remove the changed vertex data

If you know the memory layout of the vertex buffer, you can find the offset of the contents you want to change, populate a transfer buffer with the new data and then uss vkCmdCopyBuffer to copy the data.  The VkBufferCopy parameter lets you specify the size as well as the source and destination offsets.  In other words, every call to vkCmdCopyBuffer is basically the equivalent of N calls to glBufferSubData

To clarify, updating a vertex buffer on Vulkan is almost the same as populating it.  If you're on a desktop architecture where device local memory is typically not mappable, then you have to create a transfer buffer that is host visible.  You can populate that buffer with a memcpy (or whatever), and then use vkCmdCopyBuffer to transfer it to the device local memory backed buffer.  If you're on a mobile architecture with unified memory, then device local memory is typically also host visible and so you can just map it directly and update it with your memcpy (or whatever)


Seamlessly transitioning between nearby environment maps I'm wondering what is the best way to achieve an effect like here: https://matterport.com/try/ when the camera is traveling between points in space.

Apart from simple color crossfade there seems to be some sort of perspective reprojection giving the illusion of movement.

Any idea how to do this? All data I have is environment cube maps with color + linear depth at each pixel (so I can reconstruct view space position).

It looks to me like they have a coarse polygon model of the room, and texture the model by projecting the cubemaps onto it. The model seems to include the rough shapes of the walls and furniture, but not fine details such as table legs, door handles or items on shelves. When the camera moves, the geometry gives you more-or-less-correct parallax, and the crossfade between cubemaps updates the textures.

You can also see the model clearly when the walkthrough first loads. Note that the camera starts outside of the house, letting you see the whole thing (much as you'd see a game level if you noclip the camera outside it), then it moves the camera into the living room. You can also get back to this view by selecting "Dollhouse" in their toolbar.

Their model may have been constructed by an artist, but in your case if you have depth information in the cubemaps, it's probably possible to programmatically extract some geometry from them and stitch the cubemaps together to form a coherent model. I don't know much about that subject, though.


Image rotation using FFT I'm not sure how practical this might be but is it possible to use Fast Fourier Transform to rotate a raster image?

To be honest, I never really understood FFT, but I saw it being used for JPEG, for example. What I try to say is let's pretend I can grab some FFT library, what do I do next?

Also, I read that there is this Gibbs phenomenon which causes ringing artifacts. Would it be a problem here as well?

Fourier transforms wouldn't help you with a rotation. You'd just end up having to rotate the matrix of Fourier coefficients, instead of rotating the original image.

Consider for example an image made of a perfect sine wave along the x-axis with wave-vector $(k, 0)$.  (The wave-vector is the spacial frequencies along the $x$ and $y$ axes).  The Fourier transform of this would be all black, with a single white pixel at position $(k, 0)$.

When you rotate the image by an angle $\theta$, you'll get a perfect sine wave along an oblique axis.  If you draw a diagram and do a little trigonometry, you can see that the new wave-vector will be $(k \cos \theta, k \sin \theta)$.  (It helps for this part to know that the frequency $k$ is one over the period.)

Therefore, the Fourier transform of the rotated result would be all black, with a single white pixel at position $(k \cos \theta, k \sin \theta)$.  But that's just what you'd get if you rotated the Fourier transform of the original wave by $\theta$.

As for the Gibbs phenomenon, it's an issue when downsampling an image, or compressing it by frequency quantization (as in JPEG). But simply taking the Fourier transform of an image doesn't introduce any Gibbs ringing. The Fourier transform is lossless and perfectly reversible if done properly: it represents all the information in the original image.


Moving each point of a surface in direction of corresponding normal So let assume that I have some convex smooth and unclosed surface.
I'm moving each point of it in a normal direction by some constant factor (This factor is same for all points on surface).

Can I replace this operation by Uniform or Non-uniform Scaling + Translate?

Will resulting surfaces be mathematically identical in these cases?

For example, I want to transform this surface (side view):

No this cannot be modelled by (non-uniform) scaling. It's fairly easy to construct a counterexample:

The issue is that the amount a section of the curve/surface grows depends on its curvature, not its orientation in space. Notice here that the circular arc grows uniformly in all directions (by a factor of $3/2$) whereas the length of the horizontal segments remains unchanged at a length of $2$.

Of course, if your surface is not only convex but also has constant curvature, then it's just a circular arc, and for circles your transformation is equivalent to uniform scaling. You can probably also construct curves of varying curvature where your transformation happens to correspond to non-uniform scaling, but for general convex surfaces, that's not the case.


UV partial derivatives of a cylinder shape primitive I am new to rendering and I would like to calcualte UV partial derivatives of a cylinder shape, which is parameterized by radius $R$ and length $L$.
Using cylindrical coordinate mapping, for a surface point $\mathbf{p}=(x,y,z)$, the UV map (normalized to $[0,1]$) is:
$$
(u,v) = \Big(\frac{\phi}{2\pi}, \frac{z}{L}\Big)\qquad \text{where} \quad \phi = \arctan \frac{y}{x}
$$
Then I compute the derivatives with respect to $\mathbf{p}$, that is:
$$
\begin{aligned}
\frac{\partial u}{\partial \mathbf{p}} & = \frac{1}{2\pi R^2} \Big( -y, x, 0\Big) = \frac{1}{2\pi R} \Big( -\sin\phi, \cos\phi, 0\Big)\\
\frac{\partial v}{\partial \mathbf{p}} &= \Big( 0, 0, \frac{1}{L}\Big)
\end{aligned}
$$
Or equivalently:
$$
\begin{aligned}
\frac{\partial \mathbf{p}}{\partial u} & = 2\pi R \Big( -\frac{1}{\sin\phi}, \frac{1}{\cos\phi}, 0\Big)\\
\frac{\partial \mathbf{p}}{\partial v} &= \Big( 0, 0, L\Big)
\end{aligned}
$$
Everything looks good until now.
Question
In pbrt-3, the relevant part of cylinder.cpp is: (i) UV same as above; (ii) $\partial \mathbf{p}/\partial u$ however is different:
Float u = phi / phiMax;
Float v = (pHit.z - zMin) / (zMax - zMin);

// Compute cylinder $\dpdu$ and $\dpdv$
Vector3f dpdu(-phiMax * pHit.y, phiMax * pHit.x, 0);
Vector3f dpdv(0, 0, zMax - zMin);

What am I missing here?

I think it's easiest to get the tangent frame by writing the forward mapping from $(u,v)$ to $\mathbf{p}$:
$$
    \mathbf{p}(u,v) = \begin{bmatrix} R \, \cos (2\pi u) \\ R \, \sin (2\pi u) \\ vL \end{bmatrix}
$$
Then you can see that the derivatives are:
$$
    \frac{\partial\mathbf{p}}{\partial u} = \begin{bmatrix} -2\pi R \, \sin(2\pi u) \\ 2\pi R \, \cos(2\pi u) \\ 0\end{bmatrix} = \begin{bmatrix} -2\pi y \\ 2\pi x \\ 0\end{bmatrix}, \qquad
    \frac{\partial\mathbf{p}}{\partial v} = \begin{bmatrix} 0 \\ 0 \\ L \end{bmatrix}
$$
which matches the pbrt code.
I think your derivation went wrong by assuming that $\partial\mathbf{p}/\partial u$ was the component-wise reciprocal of $\partial u / \partial{\mathbf{p}}$. It's not so simple, because the latter object is really the gradient of $u$, where $u$ is now considered as a scalar field defined over all space (not just on the cylinder). To invert this properly we really need another coordinate running perpendicular to $u$ in the $xy$ plane, so that we have a locally invertible mapping; then we can take its Jacobian matrix and invert that. We can use the radius $R$ as such a coordinate. Then we have the inverse mapping,
$$
\begin{aligned}
    u(x,y) &= \frac{1}{2\pi} \arctan(y,x) \\
    R(x,y) &= \sqrt{x^2 + y^2}
\end{aligned}
$$
whose Jacobian is:
$$
    \frac{\partial(u,R)}{\partial(x,y)} =
    \begin{bmatrix}
        \tfrac{\partial u}{\partial x} & \tfrac{\partial u}{\partial y} \\
        \tfrac{\partial R}{\partial x} & \tfrac{\partial R}{\partial y}
    \end{bmatrix} =
    \begin{bmatrix}
        -y/(2\pi R^2) & x/(2\pi R^2) \\
        x/R & y/R
    \end{bmatrix}
$$
and the inverse of that is:
$$
    \frac{\partial(x,y)}{\partial(u,R)} =
    -2\pi R \begin{bmatrix}
        y/R & -x/(2\pi R^2) \\
        -x/R & -y/(2\pi R^2)
    \end{bmatrix} =
    \begin{bmatrix}
        -2\pi y & x/R \\
        2\pi x & y/R
    \end{bmatrix}
$$
from which you can then read off $\partial{(x,y)}/\partial u$ as the first column, which matches the result obtained for $\partial\mathbf{p}/\partial u$ above.


How to compute normal of surface from implicit equation for ray-marching? I want to represent ovaloids with ray-marching. I have ovaloids defined with an implicit equation.

For instance, we can consider the equation of an ellipsoid given as: $$ \frac{x^2}{a^2} + \frac{y^2}{b^2} + \frac{z^2}{c^2} -1 = 0, $$ where $a, b, c$ are constant.

Given this equation I need to caculate the normal for the shading of the suface. In an standard course of Curves and Surfaces I would take $f(x,y,z) = \frac{x^2}{a^2} + \frac{y^2}{b^2} + \frac{z^2}{c^2} -1 $ and compute $N(p) = \frac{\nabla f(p)}{|\nabla f(p)|}$ as the normal of the surface.

Here I have to check that $0$ is a regular value of $f$ and that $f  $ is differentiable. However, the second check is not automate.

How is it done in real-world systems?
Can you give an explanation of the methods used and ideally point me some implementations that exist?

Please, answer in full generality and not only considering the example given above.

The usual way of doing this with raymarching is to define your surface as a signed-distance field and use finite differences to get the gradient of the distance function at the point you’re sampling. In other words, if you have a function map(p) that returns the signed distance value at a point p, the normal at p is given by:

float epsilon = 0.001; // arbitrary — should be smaller than any surface detail in your distance function, but not so small as to get lost in float precision
float centerDistance = map(p);
float xDistance = map(p + float3(epsilon, 0, 0));
float yDistance = map(p + float3(0, epsilon, 0));
float zDistance = map(p + float3(0, 0, epsilon));
float3 normal = (float3(xDistance, yDistance, zDistance) - centerDistance) / epsilon;

Note that you can improve the accuracy by taking an extra 3 samples on the other “side” of the center position and subtracting those from the first three, then dividing by 2× the epsilon value, but if your distance function is complicated then that can be a lot of additional work for not much visible benefit.

The sample function you provide isn’t a signed-distance function as-is, and I don’t know of a straightforward way to convert it into one, but there’s an article by Íñigo Quílez (kind of a legend in this field) here with various functions you can make use of, including one for an ellipsoid. There’s also a lot of reference material available on Shadertoy—search for “raymarching” and you’ll find plenty of examples.


How can I improve the performance of my custom Global Illumination approach? I want to experiment with some new stuff about modern rendering techniques and in that way I'm trying to implement a custom GI (somewhere between pathtracing and instant radiosity).
My scene is a bit less than 1,000,000 tris and my algorithm requires to render the same scene a lot of times into different tiny textures (let's call them "bounces").

I've began my implementation in a naive way so I rapidly ran into some performance problems:
With my 1TFlops laptop GPU I can render my 1M poly scene I can have 200 "bounces" before dropping bellow 30FPS which is far from enough.

My "naive" algorithm looks like this in pseudo code:

for (unsigned int i = 0; i < res ; ++i) {
  UpdateCameraUniform(i);
  glBindFramebuffer(GL_FRAMEBUFFER, BounceFbo[i]);
  glDrawBuffers(1, renderBuffer[i]);
  for (Mesh& m : Scene) {
    m.render();
  }
}

with

Mesh::render() {
UpdateTransformUniform();
glBindVertexArray(_vao);
glBindBuffer(GL_ARRAY_BUFFER, _vbo);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, _ebo);
glDrawElements(GL_TRIANGLES, size, GL_UNSIGNED_INT, 0);
}

The full code is here (the depot is really messy for now...).

I need to find a new strategy to render my scene so I searched what I could do.
I think I have 2 strategies:

Trying to reduce the number (or the performance) of my drawcalls. It seems that there is some magic function like glMultiDraw* (I've found it in a post on "OpenGL superbible" here). But I don't really understand what this does:

Does this render multiple times the same mesh?
Different mesh but with one shader?
Different mesh with different shader but just with low CPU overhead?

Trying to reduce the render time while rendering multiple times the same mesh in "Bounces" using the same technique as this but I have absolutely no idea how it works or even if it can apply to me...

Obviously I am open to any other suggestions.

Edit: I can reach 200 "bounces" not 35

You should always profile to determine where your slowdown occurs. From your question, it's unclear whether your work is CPU bound or GPU bound.

  But I don't really understand what is does: Does this render a multiple time the same mesh ? different mesh but with one shader ? different mesh with different shader but just with low CPU overhead ?

The glMultiDraw* draw commands are for allowing you to render multiple objects in one draw call. In terms of OpenGL 3.0 and later, this means that you could specify a VBO holding a bunch of objects and use glMultiDraw* to render them. You would use this with the same shader. If you want to render the same object multiple times, then using something such as instancing would be a good solution.

This all concerns draw calls though. If you have one million triangles for only a few meshes, you won't see a massive speedup with rendering. You didn't specify your framebuffer size, but if it's small enough, you can cause severe overdraw from many small triangles. If the images are really that small, maybe you should think about adding an LOD system or use lower polygon meshes and tessellate.

  trying to reduce the render time while rendering multiple time the same mesh in "Bounces" using the same technique than this

Stereo instanced rendering is specifically a technique for VR. It relies on the assumption that the scene is pretty much identical in each eye, and so it uses instancing to render to two sides of the screen at once. However, if your bounces render similar geometry, then you may be able to render multiple bounces onto a larger framebuffer provided that the bounces are independent. There is potential for speedup from avoiding framebuffer switching (which is a rather expensive process).

In newer OpenGL versions (4.0 and above), there is added support for indirect rendering, which allows you to determine parameters through GPU computing, but it's unclear if that will help you in this case.


How can I transform an ellipse into a circle? I have an ellipse which has -45 degree axis.

How can I transform the ellipse into a circle? Can I use shear transformation along the x/y axis?

You can use a shear, or equivalently a rotation by 45, scale along one of the axes, rotate back (which is equivalent to the shear which you are looking for).

EDIT: On second thought, you don't need to even rotate back unless your circle is marked, since a circle is rotation invariant (if the rotation is around its center).


Let GPU write to itself As someone who is trying to dive into WebGL for parallelizing simulations, or Machine Learning, or typical IIR filters I've come to realize that for everything that I want to do requires the GPU to update itself and send "gossip" signals to the screen.
Sending data to the screen is fairly straight forward, however having the GPU write to itself in order to step into the next time step for a simulation and then send that information to the screen is something I have no clue about doing. I don't know what it's called or if it's even possible.
I hate asking a "is it possible to do X?" question, but sadly this is one of those because my knowledge regarding WebGL is so bad, and studying WebGL for a few months just to realize that it's not possible is not time I can afford right now. I have no one else to ask.
However, to give this question some more meat to the bones I'll say that I assume that it is possible and it is just a matter of setting up two buffers and linking them together in some special way.
Here's the actual question: I'm familiar with the fact that you set up pipelines in WebGL, does WebGL support pipelines to itself that never goes to the screen so it can modify its internal buffers?

What you are talking about is transform feedback. With transform feedback you can capture primitives (vertices, triangles, lines) in a buffer such they can be used in subsequent draw calls. You don't have to grab the data from the GPU to the CPU and back again as you can directly use the GPU buffer you have just rendered to.
I'm not exactly sure what your use-case exactly is, but you mention simulation and timesteps. Imagine a particle system where each particle has a position and a velocity at each time step of the simulation. We have two buffers A and B. A will be rendered from on even numbered frames and will be written to on odd-numbered frames and B that we write to on even numbered frames and render from on odd-numbered frames. We also have a simple vertex shader that updates the positions of vertices using the aforementioned position and velocity attributes.
Then using some initial position and velocity for each particle we can start rendering from buffer A into buffer B. After having finished this process all particles will have an update position and we can render from buffer B into buffer A.
With this you also have the option to not render to the at all screen by using the GL_RASTERIZER_DISCARD functionality.
Take a look at the examples listed on this page under 'Transform Feedback'. Or at this example.


chess-like rectangles on the mesh I am new to concepts in CG. What are the red and white rectangles trying to represent in the following screenshot?

The image is from the paper "MoSh: Motion and Shape Capture from Sparse Markers".

I don't think it's trying to represent anything in particular.

In CG there is a concept of textures, which is a bit like painting the surface of an object a specific color/look. For example, on a car you may want to paint scratches and dust on the surface.

Here they have created a human model from their capture data, and when visualising it they have decided to texture/paint a red/white checker pattern on top.

They have most likely added it for the viewer's benifit, as it helps emphasise the curves and overall shape of the model, esspesially as it animates. They also probably chose contrasting colors so the model was easier to view on a white background.


Why is the scan line filling algorithm so seemingly over complicated? It makes use of tables, buckets, and sorting, which all seem unnecessary.

I don't understand why I can't just fill between pairs of scan line intersections, ignoring vertices and edges with gradient 0.

My lecturer didn't provide the best notes on this algorithm, and all I can find online are even more complicated versions of it.

An ideal answer would be an explanation of the algorithm, and a reason why simply filling between pairs of intersections won't work.

The scan-line algorithm (as described on Wikipedia for instance) is concerned with generating the pixels in order, left-to-right and top-to-bottom, with each pixel needing to be touched only once. It was developed in the late 1960s, for devices with no framebuffer memory—so it has to generate each pixel just-in-time as it scans out to the display.

The constraint to generate the pixels exactly in order is a strong one. Now, it's possible to imagine a very simple and naive algorithm that doesn't need tables, buckets, or sorting to accomplish this. For example, for each pixel, you might just iterate over all the triangles, test whether the current pixel falls inside it, and keep track of the closest triangle that passes that test. (That's effectively ray tracing without any acceleration structure.)

It's pretty clear that this is inefficient as soon as you have more than a handful of triangles. The complicated data structures and stuff that show up in the classic scan-line algorithm are there to optimize this process. For instance, pre-bucketing the edges by Y-coordinate and maintaining an "active edge table" lets you quickly and incrementally identify the edges that affect each scanline as you move down the image. Keeping the edge intersections sorted by X-coordinate allows you to quickly generate the pixels left-to-right within each scanline, and keeping a Z-sorted list of active triangles as you scan left-to-right enables you to do hidden surface removal at the same time.

Incidentally, this algorithm is of mainly historical interest. It is very far from how modern GPU rasterizers or even software rasterizers work. Nowadays we pretty much always have a framebuffer and use double buffering, so we don't need to generate the pixels precisely in order anymore; consequently, we usually iterate over the triangles first, then find which pixels each triangle covers, rather than the other way around.

Moreover, the scanline algorithm is pretty inherently serial and doesn't make good use of today's highly parallel hardware. Modern approaches rely on multicore processing and SIMD to test many pixels against many triangles at once, rather than incrementally updating sorted lists of edges and suchlike.

For more on modernized software rasterization, see Fabian Giesen's articles Triangle rasterization in practice and Optimizing the basic rasterizer.


How to take consistent screenshots across multiple machines? I have some OpenGL based software and I would like to set up some automated testing using image comparison, but one problem we always have is that the screenshots we take always have some small differences across machines, usually due to graphics driver difference or something else (?).

What strategies are there for getting consistent screenshots across machines with varying hardware and graphics drivers?

Nicol Bolas explains why you can't expect pixel-accurate reproductions, but that doesn't mean you can't test with screenshots.

Don't compare the screenshots byte-for-byte. Instead, use a perceptual diff to quantify how different the renders are. You can then set a threshold in your test and have the test fail if the renders are too different. A harsh threshold will let you filter out the rounding differences between platforms, and the slight differences around object silhouettes, but still detect objects in the wrong place or inconsistent texturing. A more forgiving threshold will allow bigger differences in texturing (or even with different shader versions or extensions, if you have different rendering paths for different hardware), and only complain if you have gross differences such as missing lights or a completely different scene being rendered.

The pdiff tool is commonly used for this task, and I've set up test systems based on it before.


What is shader warming, and why does Direct3D seem to avoid it? I am involved in a project that had been modified to run on Apple and Linux computers, when it was originally Windows only. Specifically, Metal and Vulkan renderers have been written alongside the Direct3D one.

Something I have noticed is that there is a static startup image for 3-4 mins with the text "warming shaders", which is not there in the Windows version. This screen is not present after the first launch.

Searching online did not give me any results that shed light on this. The one SO question asked about "background prewarming" which I presume could be used to avoid a long load screen? In any case, it doesn't explain what it IS.

Simply put, what is shader warming, and why does Metal and Vulkan seem to require it? I have experience with OpenGL from many years ago where shaders were compiled from source at runtime. Is this a related concept? As I understand it, both Metal and Vulkan use binary/bytecode shaders, so compiling shouldn't be necessary.

I hadn't heard the phrase 'shader warming' before, but from some googling around, it seems that some games/engines use this term for a method of forcing the shaders to be fully compiled during level load, as explained in this answer, to avoid hitches during gameplay.

In Metal/Vulkan/D3D12, it's true that shaders are precompiled to a bytecode form, but this is hardware-independent bytecode; the shaders still need to be further compiled to the GPU-specific machine code, which (in those APIs) is done by the graphics driver when a pipeline state object is created. If there are a lot of shaders, that can potentially take awhile. The fully compiled shaders can also be cached, so this only needs to be done the first time you run the app (at least until you update the driver, which could invalidate the cached results).

So my guess about what's going on in your app is that the Metal/Vulkan renderers are calling it 'shader warming' when they go through and create all the pipeline state objects they're going to need—which entails compiling all the shaders. Meanwhile, in the Windows renderer, assuming it's D3D11 or earlier (not D3D12), perhaps the app is relying on the driver to compile shaders in the background, which lets it appear to load faster—at the cost of potentially having hitches a bit later if it tries to use a shader that's not finished compiling yet.


How can I draw a filled circle on a grid with the less rectangle as possible? I have a grid with a variable width/height spacing. Let's say 100 horizontal lines separated by 80 pixels and 100 hundred vertical lines separated by 50 pixels.

Each cell of the grid in my logic have the same function as a pixel. So each of my pixels will be 50 px width and 100 px height.

I need to draw a circle filled with a color (let's say black) and without stroke.

I also need my circle to (as it is made out of filled cells) be with the less rectangles as possible, so certain rectangles are bigger than other ones.

for example the center of the circle is made out of one big rectangle then as we go closer to the edge the rectangles becomes smaller.

for example here I have circle with a grid of same width/height spacing made out of 9 rectangles (sorry for the quality of the image)

Any algorithm to do that?

To get a proper circle, you can use a transformation matrix to transform your cell coordinates into equally spaced cartesian coordinates. In the case of your 50 x 100 pixel cells, your transformation matrix would probably either double the width or halve the height. So something like this:

| 0.5 0.0 |
| 0.0 1.0 |

If you want to draw a circle of radius r at a center point (cx, cy), you could iterate over the cells from (cx - r, cy - r) to (cx + r, cy + r) and test if each one is in the circle, and if so, fill it in, otherwise, don't. Something like this pseudocode:

double rSquared = r * r;
for (int row = cy - r; row < cy + r; row++)
{
    for (int col = cx - r; col < cx + r; col++)
    {
        double x = col * transform[0][0] + row * transform[0][1];
        double y = col * transform[1][0] + row * transform[1][1];
        double distSquared = x * x + y * y;
        if (distSquared <= rSquared)
        {
            // inside the circle, fill in this cell (col, row)
        }
        else
        {
            // outside the circle, don't fill in this cell (col, row)
        }
    }
}

As for drawing rectangles, it's not clear what you're trying to accomplish there. There's no need to use rectangles when drawing a circle.


How does DirectX 12 SLI VRAM stacking work? Mainly talking about dual-SLI here for consistency. With past DirectX (and OpenGL) APIs, VRAM was mirrored across graphics cards. With dual-SLI, this was possible by rendering one frame with one graphics card and another frame with another one. There was also a rendering option in which one graphics card would render part of the screen. Unfortunately, there doesn't seem to be much technical information on how VRAM stacking would work. What possible techniques could graphics cards or DirectX 12 be using to allow for this?

In DX12, there is no implicit driver-implemented SLI as there was in DX11. Instead, multiple GPUs are exposed to the application as separate "nodes" within a single DX12 device, and each VRAM resource lives on a single node, specified at creation time. There is no implicit mirroring of resources to both GPUs as in DX11 SLI.

So, the game engine or application has full control and responsibility over how data and work are distributed between the GPUs. It's up to the app developers to implement patterns like alternating frames between GPUs, or splitting the frame across GPUs, if they wish. For example, to implement alternating frames, the app would have to allocate all buffers, textures, render targets etc. on both GPUs and populate both of them. Then, when rendering a frame, it would generate a command list for the current GPU using its local copies of all resources. Any inter-GPU transfers needed (for temporal effects, for instance) would also be the app's responsibility.

More information on this topic can be found on this MSDN article on DX12 multi-GPU support.


Creating shared vertex normals on GPU I've mostly succeeded at porting an implementation of Marching Cubes from CPU over to OpenGL compute shaders, but I haven't tackled normals yet and wondering the best way to go about it.

My implementation deals specifically with binary valued fields (I'm trying to model 3D fractal functions that don't have a distance estimator yet), so gradient and forward differences methods won't work. I have shared vertices working, and my CPU implementation uses Quilez's method described here to accumulate face normals onto each neighbouring vertex.

I could just port this implementation over to another shader, but the problem I see with this is the massive number of atomics needed. Since we can only use atomics on scalar integer types, and I cant think of a way to pack 3 signed ints into 1 in a summable way, that means 3 axes * 3 vertices = 9 atomic adds per shader invocation. They'll be spread throughout memory of course so it's not like hitting a single atomic counter 9 times, but it still seems like a hell of a lot.

The other alternative is to run a shader invocation per-polygon and build the face normal list (i could probably pack to x10y10z10 this way), then a shader per-vertex to accumulate all the normals of neighbouring faces. This would be an enormous memory hog though, the storage space of the face indices would need 12 int per vertex to deal with the worst case. There's also the problem of how to write into this storage without again resorting to atomics to work out how many faces have already been written to a particular vertex.

Anyone have any better ideas on how to do this?

For an nVidia only solution you can use floating point atomic add intrinsics (like NvInterlockedAddFp32) Unlocking GPU Intrinsics in HLSL | NVIDIA Developer

I tried this on 80.000 vertex mesh and it's quite fast (something like 1 or 2 ms on a GTX980M, if I remember correctly)

Just beware of compiling your shaders in release for the intrinsics to work (due to nvidia bug/limitation)

Also beware of vertex splits (due to UV discontinuities for example), you'll have to handle them or else you'll have unwanted hard edges at UV seams.


Meaning of the parameter in a line-plane intersection in the space The scenery is the following:
I have a polygon in the space, defined (the polygon) from your vertices. By 3 vertices of the polygon (non-aligned) revenue equation of the plane containing the polygon.
Later I have a ray of light, represented by the parametric equation of the line in the plane (the parameter is $\textstyle t $).

I must determine the intersection between the line and the plan and see if the point of intersection (if they line and plane are not parallel) is internal or external to the polygon.

Written the equation of the plan then,
it's simple find the point of intersection: I replace the parameter values of $\textstyle x $ $\textstyle y $ and $\textstyle z $ of the line in the equation of the plane and from here I determine the value of the parameter t.
Then I replace t in the parametric equation of the line, and I have the coordinates of the intersection point.

The text that I am following (3D COmputer Graphics) tells me that if I get from the calculation that $\textstyle t < 0 $ the point will stay in a part of plane that does not contain the polygon.

I did not understand why. Than intuitive-geometric meaning has $\textstyle t $?

I point that t <0 is sufficient but not necessary condition to say that the point does not belong to the polygon. If it isn't  t <0 I'll have to explicitly test whether the point is inside the polygon or not but the book says that if t <0 I can avoid all this job  because the point is certainly outside the polygon (in a part of the plan that the book calls half plane ). I would like to know why, what mean the parameter t.

The $t$ parameter represents the distance from the ray's origin point to the intersection point, as a multiple of the ray's direction vector. It is positive for intersections "in front" of the origin, negative for those "behind" the origin as judged by the direction vector.

In raytracing we always discard any intersections with $t < 0$, because we don't want to render objects behind the camera, and so forth. A ray is only half a line, and only the points $t \geq 0$ are part of the ray.

It is not correct to say "if $t<0$ the point will stay in a part of plane that does not contain the polygon". If the polygon is behind the ray origin, then the intersection point may well be inside the polygon. However, we can immediately discard this intersection due to its being located behind the ray origin, so we would not need to perform any further tests of whether the point on the plane is in fact inside the polygon.


Why is my fragment shader not showing any colors? I am basically following this tutorial about diffuse lighting in OpenGL.
I did some modifications but the idea is the same, at least that's what I think. However, my fragment shader is not showing any colors. I checked and rechecked my uniform variables and light properties and data is ok on CPU side (don't know how to check on GPU side, though).
Here are my shaders where I cannot spot any errors, maybe someone can help me out

VERTEX SHADER

#version 410

in vec4 vposition;
in vec3 vnormal;

uniform mat4 M;
uniform mat4 VP;

out vec4 fcolor;
out vec3 fnormal;

void main()
{
    gl_Position = vposition * M * VP;
    fcolor = vposition + vec4(0.5);
    fnormal = (M * vec4(vnormal, 0.0)).xyz;
}

FRAGMENT SHADER

#version 410

in vec4 fcolor;
in vec3 fnormal;

out vec4 color;

uniform vec3 lightColor;
uniform vec3 lightDirection;
uniform float lightIntensity;

void main()
{
    vec3 N = normalize(fnormal);
    vec3 L = normalize(lightDirection);
    float visibility = clamp(dot(N, L), 0.0, 1.0);
    color = fcolor * vec4(lightColor, 1.0) * lightIntensity * visibility;
    //color = vec4(1.0); Not even this shows model in complete white
}

UPDATE

The buffer for my mesh are created like follows, maybe it helps out detecting the problem

void Mesh::initBuffers()
{
    glGenVertexArrays(1, &m_verticesVao);
    glBindVertexArray(m_verticesVao);

    glGenBuffers(1, &m_positionsVbo);
    glBindBuffer(GL_ARRAY_BUFFER, m_positionsVbo);
    glBufferData(GL_ARRAY_BUFFER, m_numVertices * 3 * sizeof(float), &m_positions[0], GL_STATIC_DRAW);
    glEnableVertexAttribArray(0);
    glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 3 * sizeof(float), 0);

    glGenBuffers(1, &m_normalsVbo);
    glBindBuffer(GL_ARRAY_BUFFER, m_normalsVbo);
    glBufferData(GL_ARRAY_BUFFER, m_numVertices * 3 * sizeof(float), &m_normals[0], GL_STATIC_DRAW);
    glEnableVertexAttribArray(1);
    glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, 3 * sizeof(float), 0);

    glGenBuffers(1, &m_positionsEbo);
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, m_positionsEbo);
    glBufferData(GL_ELEMENT_ARRAY_BUFFER, m_numIndices * 3 * sizeof(unsigned int), &m_indices[0], GL_STATIC_DRAW);

    glBindVertexArray(0);

    m_initBuffers = true;
}

Commenting all the code related to the normal in shaders results in a correct rendered mesh with colors (or white)

If it helps, you can checkout my code

After checking and rechecking my code over and over again. I finally made it work.
The problem was that I was creating the buffers and assigning locations to the position and the normals buffers

// Positions
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 3 * sizeof(float), 0);
// Normals
glEnableVertexAttribArray(1);
glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, 3 * sizeof(float), 0);

However in the vertex shader I was not setting the locations

#version 410

in vec4 vposition;
in vec3 vnormal;
...

So changing the previous code to

#version 410

layout(location = 0) in vec4 vposition;
layout(location = 1) in vec3 vnormal;

fixed the application.
Now, I am not really sure why this works, but I remember reading somewhere that OpenGL assigns locations as it sees best, so maybe in my code the location for the normal was not the same as the enabled vertex attrib array. I let someone more experienced clarify this.


How to set equivalent PDFs for cosine-weighted and uniform-sampled hemispheres? I'm trying to add BRDFs to a very basic path tracer. Starting out, I'd like to convert just the Lambertian material, with two different sampling methods, to ensure that everything is working right. Theoretically, a uniform sample and a cosine-weighted hemispherical sample should both render similar images with different amounts of noise.

However, I'm finding that the uniform sample yields a significantly darker image than the cosine-weighted sample. This makes sense as the uniform set will always be scaled by 1 whereas the weighted set will usually be scaled by more than one - but I don't know exactly how I should fix that. Maybe someone can enlighten me as to how these PDFs should work?

Here is the relevant material code (edited based on the most recent):

func (l *Lambert) Scatter(_, n geom.Unit, uv, p geom.Vec, rnd *rand.Rand) (in geom.Unit, bsdf Color, pdf float64) {
    in, pdf = sampleUniform(n, rnd) // should be able to swap this out, right?
    bsdf = l.texture.Map(uv, p).Scaled(in.Dot(n))
    return
}

func sampleUniform(n geom.Unit, rnd *rand.Rand) (sample geom.Unit, pdf float64) {
    sample = n.RandInHemisphere(rnd)
    pdf = 0.5
    return
}

func sampleCosWeighted(n geom.Unit, rnd *rand.Rand) (sample geom.Unit, pdf float64) {
    sample = n.RandInHemisphereCos(rnd)
    pdf = n.Dot(sample)
    return
}

func sampleReflected(out, n geom.Unit, rnd *rand.Rand) (sample geom.Unit, pdf float64) {
    sample = reflect(out, n)
    pdf = 1
    return
}

And the relevant render loop code:

func color(r Ray, s Surface, depth int, rnd *rand.Rand) Color {
    if depth >= 50 {
        return black
    }
    hit := s.Hit(r, bias, math.MaxFloat64, rnd)
    if hit == nil {
        return black
    }
    out := r.Dir.Inv()
    emit := hit.Mat.Emit(hit.UV, hit.Pt)
    in, bsdf, pdf := hit.Mat.Scatter(out, hit.Norm, hit.UV, hit.Pt, rnd)
    if pdf <= 0 {
        return emit
    }
    indirect := color(NewRay(hit.Pt, in, r.T), s, depth+1, rnd).Times(bsdf).Scaled(1 / pdf)
    return emit.Plus(indirect)
}

So, for Uniform sampling the PDF is $1/2π$, For Cos-weighted its $cos(θ)/π$. The Lambertian BRDF has a $\pi$ term as well in the denominator for energy conservation.

When not optimizing things you should be dividing by $\pi$ during the BRDF calculation, then dividing by the proper PDFs mentioned above.

Considering all the factors into account, for uniform sampling you'd have

$L_i*brdf*\cos(\theta) * \displaystyle{\frac{1}{pdf}}$

$L_i * \frac{color}{\pi} * \cos(\theta) \displaystyle{\frac{1}{\frac{1}{2\pi}}}$

$L_i * {color} * \cos(\theta) * 2$

Similarly for Cosine-weighted hemispherical sampling you'd have

$L_i*brdf*\cos(\theta) * \displaystyle{\frac{1}{pdf}}$

$L_i * \frac{color}{\pi} * \cos(\theta) \displaystyle{\frac{1}{\frac{\cos(\theta)}{pi}}}$

$L_i * {color} $

Since you weren't dividing by $\pi$ in the BRDF, the division by the PDF should have been $1/2$ and $\cos(\theta)$ as you suggested but couldn't understand since the actual pdfs have a factor of $\pi$ in them.


Resulting Probabilty Density in Path Tracer for paths using Next Event Estimation I am trying to implement my own Gradient Domain Path Tracer by following the code of this guy who already implemented it:

https://gist.github.com/BachiLi/4f5c6e5a4fef5773dab1

I already managed to go through various steps but I wanted to do something more. I expanded the code of in the reference by implementing next event estimation and here are some results.

Normal Path Tracer image:

Gradient Domain resulting image:

Results are already good.. but as said before, I wanted something more. So I implemented Next Event Estimation and here is the result of the basic Path Tracer:

Here is my code:

private Vector3 SampleWithNEE( Ray ray )
{
  // prepare
  Vector3 T = (1,1,1), E = (0,0,0), NL = (0,-1,0);
  int depth = 0;
  // random walk
  while (depth++ < MAXDEPTH)
  {
    // find nearest ray/scene intersection
    Scene.Intersect( ray );
    if (ray.objIdx == -1) break; //if there is no intersection
    Vector3 I = ray.O + ray.t * ray.D; //go to the Hit Point on the scene
    Material material = scene.GetMaterial( ray.objIdx, I );
    if (material.emissive) //case of a light
    {
        E += material.diffuse;
        break;
    }
    // next event estimation
    Vector3 BRDF = material.diffuse * 1 / PI;
    float f = RTTools.RandomFloat();
    Vector3 L = Scene.RandomPointOnLight() - I;
    float dist = L.Length();
    L = Vector3.Normalize( L );
    float NLdotL = Math.Abs( Vector3.Dot( NL, -L ) );
    float NdotL = Vector3.Dot( ray.N, L );
    if (NdotL > 0)
    {
        Ray r = new Ray( I + L * EPSILON, L, dist - 2 * EPSILON ); //make it a tiny bit shorter otherwise I risk to hit my starting and destination point
        Scene.Intersect( r );
        if (r.objIdx == -1) //no occlusion towards the light
        {
            float solidAngle= (nldotl * light.getArea()) / (dist * dist);
            E += T * (NdotL) * solidAngle * BRDF * light.emission;
        }
    }
    // sample random direction on hemisphere
    Vector3 R = DiffuseReflectionCosWeighted( ray.N );
    float hemi_PDF = Vector3.Dot( R, ray.N ) / PI;
    T *= (Vector3.Dot( R, ray.N ) / hemiPDF) * BRDF;
    ray = new Ray( I + R * EPSILON, R, 1e34f );
  }
  return E;
}

Things work really fine and the results are shown with the picture above. One more thing: I have only diffuse surfaces in my scene.

Now, the problem is that in this method I use 2 kind of PDF's:

one is given by randomly sampling the light in the Direct Lighting of the Next Event Estimation, indeed that SolidAngle is our PDF, or better 1/PDF.
while the second PDF is the one led by the use of DiffuseReflectionCosWeighted which brings a PDF equals to CosTheta/PI.

Everything fine so far and for any implementation detail you can just look at my code but problems come with my Gradient Domain Path Tracer. Indeed, there, as also in the reference link above implemented by Tzu-Mao Li, I need the final probability density of a whole path to compute the final gradient image. How did I calculate it in case without Next Event Estimation (NEE) ? In that case (since I have only Diffuse surfaces) this probability is the product of CosTheta / PI at each bounce in the scene. Everything is fine and the resulting gradient image is shown above.

Instead, in case I use NEE things don't work anymore because the Probability Density of my whole path changes and I don't manage to understand how it is. The resulting Gradient Domain Image with Next Event Estimation is:

I need to understand how to calculate the final density probability of a path. Can you help me doing it? Thanks in advance!

I don't have any experience with Gradient Domain Path Tracing, but here are my thoughts:

There seems to be a different problem

If you look carefully at the little spikes of distortion in the final image, you will see that they are all lit from the same direction - on their top left side at a consistent 45 degrees. The sphere also appears to be lit from this angle, rather than from above by the light source.

This is unlikely to be explained by an incorrect probability estimation for a path. I would expect there to be a different problem with the code, that these distortions are hinting at.

I will therefore address these two separate points:

You want to know how to calculate the Probability Density of a path when using Next Event Estimation.
There is evidence of some problem unrelated to this.

I'll also review the code for non-essential points - but I'll leave this until after the essentials.

Probability Density of a path when using Next Event Estimation

Looking at the paper on which the code you are following was based it seems that the novel shift mapping described in section 5.2 is defined in terms of the reflective properties of the surfaces found at the vertices of the path. I must emphasise that I don't have a full understanding of this, but it suggests that Next Event Estimation may not require a change in this approach, as the surfaces encountered will be the same. Hopefully once the other problems are cleared up it will be easier to judge whether the image looks correct.

Note that section 5.2 of the paper already mentions (just below Figure 10) that they take into account sampling the emitter "either using BSDF or area sampling".

The difference with Next Event Estimation is that the area sampling happens at every vertex of the path, but it isn't obvious to me that this should cause a problem.

The fact that your scene uses only diffuse surfaces means that the offset path should in most cases rejoin the base path at the second vertex, so you would only need to recalculate the area sampling for the first vertex of the offset path.

The cause of the incorrect lighting direction

In reading through the code to familiarise myself with how it works, I noticed that NLdotL is calculated, but then not used. A text search revealed that the only other occurrence has a different case: nldotl. Here are the two variables in context (first and ninth lines of this excerpt):

float NLdotL = Math.Abs( Vector3.Dot( NL, -L ) );
float NdotL = Vector3.Dot( ray.N, L );
if (NdotL > 0)
{
    Ray r = new Ray( I + L * EPSILON, L, dist - 2 * EPSILON ); //make it a tiny bit shorter otherwise I risk to hit my starting and destination point
    Scene.Intersect( r );
    if (r.objIdx == -1) //no occlusion towards the light
    {
        float solidAngle= (nldotl * light.getArea()) / (dist * dist);
        E += T * (NdotL) * solidAngle * BRDF * light.emission;
    }
}

Since nldotl is not defined, the result of the code is undefined behaviour. In practice, the program is likely to act as if nldotl is zero, or for some compilers perhaps a constant arbitrary value, or even a different arbitrary value at each iteration. For your particular compiler it appears to be a constant value, and I strongly suspect this is the cause of the distinctive alignment of lighting angle on all the speckles and the sphere. If there is also another contributing problem it will be easier to analyse it in a separate question once this initial problem has been fixed.

It may be worth considering using a compiler and/or flag setting that gives an error or at the very least a warning for undefined variables, as this type of mistake is both very easy to make, and very easy to overlook later.

Additional contribution of light source

There appears to be another problem that will cause results to be incorrect in a more subtle way, with no obvious distortion. Due to Next Event Estimation, the light source is contributing to each step along the path. This means it should not make a contribution if the path itself hits the light source directly. Otherwise, for that path the light source will be contributing twice. You can correct this by changing:

if (material.emissive) //case of a light
{
    E += material.diffuse;
    break;
}

to:

if (material.emissive) //case of a light
{
    break;
}

This will give a zero contribution from the intersection with the light.

Note that since I can only see this one function, I can't guess whether this will also make the light appear black in the image. You may or may not need to adjust for rays that start at the camera and hit the light directly.

Code review

Doubly finite rays

I'm used to defining a ray as a half infinite line segment - having a starting point but no ending point. I notice that this code gives a ray both a starting point and a length. The only place I can see a reason for this is when testing a shadow ray against the light source: the code checks that there are no intersections on the way to the light, so intersections behind the light (or on the light itself) must be excluded. In all other places, the ray is defined with a pseudo-infinite length (1e34f).

The following suggestion won't affect the correctness of your code, but it may be more readable and avoid you having to work around the need for infinity and having to account for epsilon twice.

If the ray is simply a starting point and a direction, then shadow rays can simply check that the first intersection is the light, rather than checking that there is no intersection. For example, by replacing:

if (r.objIdx == -1) //no occlusion towards the light

with:

if (r.objIdx == LIGHT) // There is no object intersected before the light

Here I use LIGHT as a placeholder for the id of the light source, since that part of the code is not included in the question.

This will always give false if the light is occluded by a nearer object.
This will always give true if the ray hits the light before any more distant objects.

This is therefore equivalent to the current code, but does not require the ray to store a length.

Lights that don't reflect

The code currently models lights and surfaces separately. This means that if an object is a light then it is only emissive, and reflects no light from other objects.

This causes a negligible difference in the example scene in the question, which has a single bright light. However, if used with a number of dimmer lights, it would be more noticeable that they do not light each other. In many cases the difference will not be noticeable, so this is not a problem if it works for the scenes you wish to render.


Why does the red color channel lose the most details during jpeg compression I have an e-commerce website dealing with flowers and cakes. We get our photoshoot done inhouse and also the post-shoot editing of images. While we upload them on the website all images go through the same image compression algorithm. But if we compare all images, then we find product images having red color in them loosing out most of its details only in the areas with red color.

I have attached an image below for your reference, here the left-hand side image is at 80% quality and the right side has no compression in place. If you compare all these images, then you will see that the red-colored areas of the images lose the most details, at least visually.

I have read about jpeg compression but it says that color doesn't make any difference, but it is not the case here.

Can we do anything during photoshoot or post-editing to fix these issues?

One issue is "chroma subsampling". The image data of a JPEG is not stored in packed RGB but YCbCr as 3 seperate planes. The Cr and Cb channels contain colour information and the Y channel is luminance. In JPEG it is common to use a lower resolution Cr/Cb channel, typically at half the resolution of the Y channel, this is know as 422 Chroma subsampling (another variant is 420 where only the horizontal resolution is halved).

What you can do is select 444 Chroma subsampling, if possible, which will mean no down-scaling of the Cr/Cb channels.

In Gimp you can select the subsampling type in "advanced options" when exporting JPEG files. This is independent of the quality setting slider.

There is a great image from wikipedia which compares the various subsampling modes side-by-side with a visualisation of one of the chroma channels, you can see the red decal in the centre is most affected by downscaling as that channel has high contrast edges

Wikipedia article for Chroma Subsampling


How would I create a virtual cabling system? I am trying to devise a GUI system that functions as a virtual patchbay, as seen in something like Logic Studio's environment editor, or seen in virtual synthesizers that use virtual patchcables.

I'm not much of a coder, so I've been experimenting with LiveCode.

The challenge seems to be part code, part graphical. I can't even fathom how a system like that works. What bitmaps should I produce (if any) and how should they be manipulated in code?

Any assistance would be greatly appreciated.

The shape you’re trying to draw is called a catenary: it’s the shape that a cable/cord of constant density takes when supported at each end. You’ll have to do some research to find a parametric equation for its shape—this page has a start, though it doesn’t let you substitute in the endpoints so you’ll need some additional work there.

Once you have an equation that gives you a point along the curve as a function f(t) = (x,y), where t is in the range [0,1], you can draw it the same way you would any other parametric curve: step along the function from 0 to 1 in small increments, drawing individual line segments from the value of f at the previous increment.

For nicer-looking cables, you can follow that process with a wide line in a dark color followed by a narrow line in a lighter one to give the appearance of shading, draw a shadow with the same curve offset vertically in partially-transparent black, etc.

As to the patch ports themselves… you can make grids pretty easily by using an image for a single port and drawing it at regular intervals across the space you want to cover. To figure out which port a given screen position corresponds to (e.g. if the user clicks on a particular one), get that position, divide it by the size of the interval you drew the ports at, and take the ceiling of that value. In other words, if you’re drawing a port every 20 pixels and the user clicks on (42, 27), dividing each coordinate value by 20 gives you (2.1, 1.35), which ceils to (3, 2)—i.e. the third column and second row.


How can I debug GLSL shaders? When writing non-trivial shaders (just as when writing any other piece of non-trivial code), people make mistakes.[citation needed] However, I can't just debug it like any other code - you can't just attach gdb or the Visual Studio debugger after all. You can't even do printf debugging, because there's no form of console output. What I usually do is render the data I want to look at as colour, but that is a very rudimentary and amateurish solution. I'm sure people have come up with better solutions.

So how can I actually debug a shader? Is there a way to step through a shader? Can I look at the execution of the shader on a specific vertex/primitive/fragment?

(This question is specifically about how to debug shader code akin to how one would debug "normal" code, not about debugging things like state changes.)

As far as I know there are no tools that allows you to steps through code in a shader (also, in that case you would have to be able to select just a pixel/vertex you want to "debug", the execution is likely to vary depending on that).
(NOTE: Renderdoc allows you to do so by selecting a specific thread/pixel)
What I personally do is a very hacky "colourful debugging". So I sprinkle a bunch of dynamic branches with #if DEBUG / #endif guards that basically say
#if DEBUG
if( condition )
    outDebugColour = aColorSignal;
#endif

.. rest of code ..

// Last line of the pixel shader
#if DEBUG
OutColor = outDebugColour;
#endif

So you can "observe" debug info this way. I usually do various tricks like lerping or blending between various "colour codes" to test various more complex events or non-binary stuff.
In this "framework" I also find useful to have a set of fixed conventions for common cases so that if I don't have to constantly go back and check what colour I associated with what.
The important thing is have a good support for hot-reloading of shader code, so you can almost interactively change your tracked data/event and switch easily on/off the debug visualization.
If need to debug something that you cannot display on screen easily, you can always do the same and use one frame analyser tool to inspect your results. I've listed a couple of them as answer of this other question.
Obv, it goes without saying that if I am not "debugging" a pixel shader or compute shader, I pass this "debugColor" info throughout the pipeline without interpolating it (in GLSL with  flat  keyword )
Again, this is very hacky and far from proper debugging, but is what I am stuck with not knowing any proper alternative.


What is a Gaussian Lobe concerning BRDF and NDF? I was reading a paper about voxel cone tracing and came across this term. So, what is a Gaussian Lobe or a Lobe in general?

I am asking in the context of Computer Graphics, BRDF and NDF.

In a BRDF or NDF this most likely refers to a spherical Gaussian distribution, also known as a von Mises–Fisher distribution.

It's similar to the usual Gaussian, but using the dot product between two unit vectors as a measure of distance to the mean:

$$ p(\hat v) = a \exp(\lambda(\hat \mu \cdot \hat v))$$

where $a$ is a normalization factor, $\lambda$ is a number controlling the width of the lobe (roughly corresponds to $1/2\sigma^2$ in the usual Gaussian), $\hat \mu$ is the unit direction vector for the mean of the lobe, and $\hat v$ is the unit direction vector at which we're evaluating the lobe.

(You'll also sometimes see it defined with $\lambda(\hat \mu \cdot \hat v - 1)$ in the exponent, but the $-1$ term can be pulled out and rolled into the normalization factor.)


Uniform Buffers: What is quicker? So I want to render a buttload of quads by using a single 4 point vertex array and change the position of the quad. What is quicker:

Using glBufferSubData to change the positional data or

Using glMapBufferRange to DMA the buffer and glUnmapBuffer to push the data back to the GPU?

When using the vertex buffer data, I need to also call glVertexAttribArray to reassociate the data with the vertexArrayObject.

When using uniform buffers and do it with shader magic I only need to change the uniform buffer.

What is quicker?

A key rule of thumb for high-performance code in general, and graphics in particular, is to batch large amounts of work together rather than doing it in small chunks. In your case, rather than sending one quad's worth of data at a time and drawing one quad at a time, it will be far more efficient to send many quads' data in a large batch, and draw many quads together in a single draw call.

A good way to accomplish both is instancing. You would have a 4-point vertex buffer for the corners of a quad, and a second vertex buffer for the values that are per-quad (what you're putting in your uniform buffer now). This second buffer would be large enough to hold the data for all the quads at once, not just one quad. To populate the vertex buffer, you can map the whole thing, write all the data, and unmap. Then do a single instanced draw call to render all the quads at once.

However, if instancing can't be used for some reason (for instance if each quad needs a different texture, and it's not possible or desirable to put them together in an atlas), you can still upload all the data in one batch. Create a uniform buffer large enough to hold an array of all the quads' data; map/write/unmap it all in one big batch as before; then before each draw call, use glBindBufferRange to select one quad's data from that large buffer.

Batching can also be used when you don't know the number of quads in advance. Just pick a "reasonably large" batch size (say, 1024 quads) and size your buffers for that number. Accumulate quads into the buffer until you reach the size limit, then draw them all, and start over (using the GL_MAP_INVALIDATE_BUFFER_BIT flag with glMapBufferRange to get a fresh copy of the buffer). You'll also need to do a partial draw at the end of this process to handle any left-over quads that didn't reach the size limit.


What's the difference between Clipping and Culling? In a tutorial about OpenGL both words are mentioned, unfortunately without a good explanation.
To find out, I have read this article three times, but I fear I'm a too common man... my understanding is this:
The video frame is rectangular, and the camera can't take the complete scene, so everything outside the camera's rectangular view is "cut" - or "clipped".

On the other hand, "culling" means that those polygons are dismissed in further calculation that can't be seen anyway, because of other polygons in the foreground.

I guess this is what the linked article tries to describe in summary items 1. and 2. Correct?

What I still don't get are these points:

  4.When compared to culling, clipping establishes the highest polygon. In video games, more of the clipping method is used than the culling
  technique.

So... I have a choice whether to use culling or clipping? Then my understanding above is wrong...?

  5.The clipping technique is used when the objects are partially visible. By utilizing the clipping technique, the video quality is
  increased and the frame rate is improved. Moreover, clipping also
  increases the presentation speed.

I understand that framerate increases when I don't have to waste time calculating polygons I don't need (which applies to both clipping and culling).
But how does this increase video quality?
And why should clipping do so, but culling not? Or does it?

When it comes to graphics, the term "culling" by itself doesn't really have a single definition or location within the pipeline. The term is sometimes applied to primitives and it is sometimes applied to fragments. It is sometimes applied to faces based on orientation relative to the camera, and it is sometimes applied to primitives based on various parameters. So it's not useful to talk about "culling" as though it is a single thing; there are a lot of things called "culling".

By contrast, "clipping" with regard to the graphics pipeline is almost always used to refer to what happens to the vertices of primitives after any programmable shading, but before rasterization. Clipping happens differently for different kinds of primitives, but the basic idea is to take a primitive and split it into one or more primitives, such that all of the new primitives are entirely within the clip-space volume.

If a primitive is partially outside of the clip-space boundary, the process of clipping creates one or more new triangles that are entirely within that boundary. The clip-space boundary, through the viewport transform, defines everything which is visible with the current rendering state, so if something is outside of the clip-space boundary, it is not visible. Well, in theory; in practice, there are ways to draw stuff outside of the viewport (point size, line width, multisampling, etc), but nevermind those now.

Now this describes the abstract model of how OpenGL and similar specifications talk about rendering. What actually happens in the hardware is often a different matter. Because clipping has the potential to generate many primitives from one (and thus make the pipeline choke a bit, since it has to run more rasterization cycles), hardware tends to avoid performing actual clipping unless absolutely necessary. So the notion that it is a performance optimization is not really true.

Note that a primitive which is entirely outside of the clip-space boundary gets clipped into nothingness. This particular case is often called "culling" the primitive, which is why the terms sometimes get used in similar places. Point primitives are either rasterized or culled; they don't ever undergo real clipping (since there is only one vertex). But overall, this process of reducing primitives to those within the clip-space boundary is referred to as "clipping", not "culling".

Clipping also allows users to define some user-defined clipping parameters, which clip primitives based on values provided by the previous vertex processing stages. You can use these to implement clipping planes, such that primitives are clipped at the intersection of an invisible plane. All parts of a primitive on one side of the plane are visible, but not on the other side.

Things which are referred to as "culling" formally within rendering APIs:

Face culling. Discarding triangle primitives based on their winding order relative to the viewport.
Vertex-based culling (a relatively new feature), whereby entire primitives can be culled based on vertex processing parameters.

The article you're talking about seems to mostly equate "culling" with visibility culling techniques. These are computations implemented by user code which chooses to render or not render an object based on whether it is visible. These can range from the very simple frustum culling test (ie: asking if the object is even close to the visible region) to complex BSP systems or portal visibility culling (which attempt to detect if objects are entirely behind static geometry). But these are not part of the GPU's rendering pipeline, even if GPU processes are sometimes used to do such testing.

Overall, that article gets more wrong than it does right, so it's best to just pretend you never read it.


LWJGL 3 render to texture offscreen without display At the moment, I render everything in my OpenGL application to a single framebuffer (color only, no depth buffer). I then go from the texture ID and read out some data to use it. My problem is, in the init, I first init the GLFW window, make it visible calling glfwShowWindow(window); and then create the frame buffer. This means I have an empty window showing up where nothing is rendered.

But when I comment this line to keep the window hidden, the frame buffer status after creation is GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT. How do I go about rendering offscreen without window, and is there a possibility to not use GLFW at all?

So the way I went by not showing the window seems the way to go, and the error about GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENTapparently wasn't related to this at all and is gone.


Difference between using + and * operator after tex2D to add albedo color? I have a two different codes with a little difference

Case 1. I used a + operator before _myColor.

void surf(Input IN, inout SurfaceOutput o) {
       o.Albedo = (tex2D(_myTexture, IN.uv_myTexture) + _myColor).rgb;
    }

Case 2. I used a * operator before _myColor.

void surf(Input IN, inout SurfaceOutput o) {
       o.Albedo = (tex2D(_myTexture, IN.uv_myTexture) * _myColor).rgb;
    }

In first case It gives bright color.

But in second case it gives very dark color.

My question is why and how this is happening?

In a shader, colors in the viewable range are assumed to be in the range 0 to 1. If you take a value that's between 0 and 1 and add a positive value to it, it will get brighter. However, if you take a value between 0 and 1 and multiply it by a value between 0 and 1, it will get darker. To understand, think about multiplying 0.5 by 0.5. The result is 0.25.

More generally, any value between 0 and 1 can be thought of as a percentage. 0.05 = 5%. 0.25  = 25%. So multiply any value between 0 and 1 by a percentage between 0 and 1, and it will be less than the original value. (50% of 50% is 25% in the earlier example.)

If you have a photo editing application like Photoshop or GIMP, you can see the difference between the 2 by creating 2 layers with image data in them, and setting the blend mode of the top layer to either Add or Multiply. It's particularly revealing if you use a left-to-right gradient of black to white over the same gradient rotated 90°. You'll see how all the values react to the current blend mode.


Need a fast ray-box intersection that handles if a ray is parallel and in line with a plane of the box I've seen lots of spins on the ray-box intersection test, and a lot of them seem to boil down to this code: (omitting some details here)

  invRay.x = 1.0 / ray.x;

  tx1 = (box.min.x - rayOrigin.x) * invRay.x;
  tx2 = (box.max.x - rayOrigin.x) * invRay.x;

  tmin = min(tx1, tx2);
  tmax = max(tx1, tx2);

  // Repeat for Y and Z
  // Then check to ensure tmin < tmax

(I got mine specifically from this discussion and this source. Also note that if you use std::min and std::max, it's far from branchless)

However, this raises a significant problem.

What if the ray's origin in one or more dimensions is equal to the min or max points?

What I found is that I get this situation on an example ray and box:

Ray origin: (3.225, 0.243, 2.221)
Ray direction: (0, 0, -1)
Box min: (3.223, 0.241, 2.119)
Box max: (3.225, 0.243, 2.121)

Is that when tx1 is calculated, it correctly gets -inf. However, when tx2 is calculated, it gets (3.225 - 3.225) * inf which is coming out to -nan. This means that later, when we do:

float tmax = std::max(tx1, tx2);

We get max(-inf, -nan) which comes out to -inf and from that point on in the tests, tmax will always be -inf, making the overall ray-box test fail.

This is confirmed by some comments on the page where I got the code:

  Something to consider here is that 0 * inf =nan which occurs when the ray starts exactly on the edge of a box

And

  Having read both this and part 2, I'm unable to work out what changes I need to make in order to reliably detect intersections when a ray is exactly on the edge of an AABB.

Does anyone know a fast method for ray-box intersection with edge and corner hits?

I should note this is not being used with a BVH. I need an exact intersection test, I cannot increase the size of the box at all.

Here is how I found this can be solved.

(The below also happens at the start, just fyi)

  float tmin = std::numeric_limits<float>::lowest();
  float tmax = std::numeric_limits<float>::max();

Add an additional check:

if(rayDir.x == 0.0f)
{
  if (rayOrigin.x() < box.min.x() || rayOrigin.x() > box.max.x())
  {
    return false;
  }
}
else
{
  tx1 = (box.min.x - rayOrigin.x) * invRay.x;
  tx2 = (box.max.x - rayOrigin.x) * invRay.x;

  tmin = std::max(tmin, std::min(tx1, tx2));
  tmax = std::min(tmax, std::max(tx1, tx2));
}

This way the tx1/tx2 values never get calculated if the ray's direction vector is zero in that direction, aka if it's parallel to that plane.

It's not particularly compact or pretty, but it does neatly solve the issue.


Irradiance Map Equation What is "dω"? Link to paper : Irradiance and Incoming Radiance | lawrence.edu

  Using a shader to compute En

  Computing En as described above is a very compute intensive operation,
  because we have to compute the integral over the cube map for each new
  direction vector n that we want to work with. At the same time, the
  details for different values of n are highly repetitive. This suggests
  that we should enlist the aid of shaders to compute this mapping for
  us. Here is the outline of a strategy that makes it possible to do
  this.

  Use OpenGL to render a two dimensional square centered at the origin
  with sides of length 2. This square represents one of the sides of our
  cubical irradiance map. Given the mapping above we can map any point
  (x , y) on the square to a direction vector n.
  We render the square to
  a framebuffer with dimensions size by size pixels, where size is the
  desired size of our irradiance cube map texture.
  In the fragment
  shader, we translate the interpolated fragment positions we are given
  into direction vectors n and construct loops that sum over the six
  faces of the environment map:

  When rendering is done, we convert the image in the frame buffer to a
  texture for use by our irradiance cube map.

I understand that Li is color at i. ωi is the direction of incoming light.

However I do not understand what dωi is.

Paper describes it as...

  Computing the solid angle dωi subtended by a particular texel is a
  little more involved. Texels near the center of the texture subtend a
  somewhat larger angle, while texels near the corner take up a smaller
  solid angle when we map the cube texture onto a sphere. Here is a
  reference that explains how to compute the solid angle correctly from
  the positions xi and yi.

However I am not following words to understand what it means.

$E_n$ is illuminance (or irradiance), $L_i$ is luminance (or radiance) in direction $\omega_i$ and $d\omega$ is differential solid angle.

TL;DR: $d\omega_i$ is the area of a pixel in the cubemap projected onto the unit sphere.

And then a longer explanation (:

Solid angle is essentially the projected area of an object on a unit sphere and has unit of steradians (sr). Entire unit sphere has surface area of $4\pi$ and hemisphere has the area of $2\pi$, which comes up a lot in lighting calculations due to integrals over spheres and hemispheres respectively.

If you would want to calculate solid angle for an object and point $p$, you would project the object on the unit sphere at $p$ and calculate the area of the projection. Further the object is from $p$ smaller the solid angle gets. As an example, we can calculate solid angle for Sun as seen from the Earth. We know that Sun has about 0.52 degrees subtended angle from Earth, so we can calculate that the solid angle is:
$$2\pi*(1-Cos(0.52/2))\simeq0.00006469sr$$

So the differential solid angle is just an infinitesimally small solid angle. If you would numerically integrate over a sphere ($4\pi$ steradians) by taking $N$ uniform samples on the sphere, your $d\omega$ would be $4\pi/N$. With integrals $N$ goes to infinity, while with numerical approximation of the integral $N$ is some "sufficiently large" number.

However, in this paper you refer to they perform integration by rendering to a cubemap at $p$ and then iterating through all the rendered pixels. But because these pixels are rendered onto planes (cubemap faces), their projection on the unit sphere isn't constant, so $d\omega$ needs to be weighted by the position in the rendered image, which they explain in your second quote:

  Computing the solid angle dωi subtended by a particular texel is a
  little more involved. Texels near the center of the texture subtend a
  somewhat larger angle, while texels near the corner take up a smaller
  solid angle when we map the cube texture onto a sphere.


What are Jittering and Dithering I've heard a lot about jittering and dithering but I would like to know more about those techniques, especially when used to avoid visible sampling in a fragment shader.

What are the differences between both?
When are they used the most?
What's the best way to implement them?
Can we combine them?

Jittering and dithering are both techniques of adding noise to reduce visible artefacts (such as banding) in an image. They solve different kinds of artefacts so they are used in different situations. Jittering moves sample positions in space to reduce artefacts caused by regular sampling. Dithering changes the way colours are rounded (when reducing precision), replacing banding with noise.

When is jittering used?

A use case for jittering is when generating eye rays in a ray-tracer. Without jittering, you might generate one ray in the centre of each pixel. But then if you trace those at a regular pattern, such as a grille of bars each one pixel wide, you might find that every ray hits a bar, making the grille solid; or that every ray hits a gap, and the grille isn't seen at all.

Instead, you can add a small stochastic displacement to each ray direction, so that each ray goes through a different part of its pixel. That way, some rays will hit the bars of the grille and some will hit the gaps, making a noisy image instead of a completely wrong image.

When is dithering used?

The normal use of dithering is when rounding colours (intensity values) from a high-precision format (such as floating-point numbers) to a low-precision format (such as 8-bit integers). Say you have a smooth gradient you want to represent in an 8-bit image. If you round each number separately, then starting from the black end of the gradient, you get a lot of black pixels until they get light enough to round to 1/256; then these all have the same value until you get to 2/256; and so on. You end up with regular bands where all the pixels have the same value.

Instead, when you round the first number down, you can let it "donate" the lost intensity to another nearby pixel. That way, then at the black end of the gradient, maybe every tenth pixel is 1/256 and the other nine are 0. The ratio of black to nearly-black pixels increases gradually, until eventually all the pixels are 1/256, and then every tenth is 2/256, and so on. Instead of visible bands, the overall intensity increases smoothly along the gradient - but the price of this is that pixels which should be the same are slightly different, adding noise.

Dithering is used a lot in printing, where the colour resolution is quite poor (i.e. the steps between colours are big) but the spatial resolution (how small you can make each dot of colour) is very fine. It used to be used in the days of 256-colour displays for making colours which couldn't otherwise be represented, and it's still used to avoid banding when displaying (for example) a 16-bit-per-channel image on an 8-bit display.

Is dithering the only way?

Because traditional dithering algorithms involve giving information from each pixel to its neighbours, they don't parallelise well, so they're not suited to modern hardware. Another popular technique is to deliberately add noise to the source image. The range of the noise is less than one step of the destination format (e.g. the noise ranges from -1/512 to 1/512), so it can't be seen directly, but it means that each pixel will be rounded differently. Like true dithering, it replaces any banding with subtle noise in the output image.

Can we combine them?

There's no reason you can't use both in the same image, but as they solve different problems, that's not really "combining" them. It's like asking if we can combine hoses and axes. A fire truck has both, and the firefighters might use both at the same incident, but one is for extinguishing fires and the other is for freeing trapped people.

An example of when you might use both is if you ray-trace an image into a floating-point framebuffer, using jittered ray positions to avoid sampling artefacts, and then reduce that to an 8-bit image using dithering to better preserve the colours.