Replacing Vertex Attributes (glBufferData vs glBufferSubData) I'm sending a large number of data to the vertex shader. I use glBufferData to generate my VBO. Later on I have to replace the data in my VBO so I do the following:

glBindVertexArray(VAO);
glBindBuffer(GL_ARRAY_BUFFER, VBO);
glBufferSubData(GL_ARRAY_BUFFER, sizeof(glm::mat4) * models().size(), &models[0], GL_DYNAMIC_DRAW);

What I noticed is when I used the above code (I'm assuming) that some of the previous data is left in the VBO because I not only draw the new objects I want to but also some of the old ones.

Therefore I had to change my code to the following:

glBindVertexArray(VAO);
glBindBuffer(GL_ARRAY_BUFFER, VBO);
glBufferData(GL_ARRAY_BUFFER, sizeof(glm::mat4) * models().size(), &models[0], GL_DYNAMIC_DRAW);

The reason I don't like the above aproach is because according to Khronos

  When replacing the entire data store, consider using glBufferSubData rather than completely recreating the data store with glBufferData. This avoids the cost of reallocating the data store.

How do I replace the whole VBO with my new data, but if my new data is smaller than the old data then how do I reallocated the VBO efficiently? Or is there something equivalent to a NULL terminating character where the graphics card can know not to read any further data?

The reason I ask this is because from my understanding glBufferStorage is like glBufferData except that the data store generated by glBufferStorage is immutable meaning that the size of the data store cannot be changed, as stated on Khronos and on Stackoverflow.

  glBufferStorage and glNamedBufferStorage create a new immutable data store.

But if glBufferData creates a data store which cannot shrink in size then what's the point of it? (Unless I'm misunderstanding its intended use)

There is a difference between "can" and "should".

You "should" not use glBufferData to change the size of an existing non-immutable buffer. You can still do so, but don't expect this to be advantageous.

You "can" not change the size of an immutable buffer with any function.

  How do I replace the whole VBO with my new data, but if my new data is smaller than the old data then how do I reallocated the VBO efficiently?

Why would you want to? If the size of your data got smaller, then it is entirely possible that a few frames from now it will be bigger again. Perhaps bigger than it was before. That extra storage isn't waste; it is extra storage you're keeping around because you intend to use it at some point in the future.

The cost of allocating memory is high; you should avoid it whenever possible. If you need to increase the size of a buffer, then the implementation must allocate new storage for it. You want to avoid that whenever possible.

What you ought to be doing is figuring out what a reasonable maximum size would be for this data, and then allocate a buffer of that size.

  Or is there something equivalent to a NULL terminating character where the graphics card can know not to read any further data?

That's up to you. You tell OpenGL exactly how many vertices to read from a vertex buffer. This is determined based on:

The vertex format for each attribute.
The offset within the buffer for each attribute.
The number of vertices you send in the glDraw* command.

OpenGL doesn't arbitrarily process data; it reads the data you tell it to read. If you want it to not read past a certain point, stop telling it to read past that point ;)


Why integrate over a hemisphere (and not a sphere) to solve the rendering equation? In most text books that I have seen, this is how the rendering equation is written:

$$L_0( \omega_0)= L_e(\omega_0)+\int_{\Omega}{f(\omega_i, \omega_0)L_i(\omega_i)\,\mathrm{d}\omega_i}$$

Where $\Omega$ is defined to be a hemisphere (and all those functions depend on more variables, omitted here for simplicity's sake).

Now suppose the surface being rendered is some kind of glass, or some transparent plastic. Why would it make sense to only integrate over a hemisphere? I would imagine that there can be incoming light from any direction, and thus the integration domain should be the entire sphere. How is the light coming from behind the glass accounted for?

The form of the rendering equation that uses only the BRDF ($f$ in your example, often called $f_r$) and integrates over one hemisphere does not account for transmission.

When adding in transmission, it's fairly common to add a second integral over the opposite hemisphere, using a different BTDF function (bidirectional transmission distribution function). This is equivalent to an integral over the full sphere of directions with a BSDF function, but since that function would usually have to be defined as a piecewise function, writing it as two integrals can be more straightforward.


Why the gradients generated in original Perlin Noise causing axis-aligned clumping? I am learning Perlin Noise, the original version and the improved version.

In the paper  which Ken Perlin wrote in 2002. He said "The second deficiency is that whereas the gradients in G are
distributed uniformly over a sphere, the cubic grid itself has
directional biases, being shortened along the axes and elongated
on the diagonals between opposite cube vertices. This directional
asymmetry tends to cause a sporadic clumping effect, where
nearby gradients that are almost axis-aligned, and therefore close
together, happen to align with each other, causing anomalously
high values in those regions". Here I can not understand what means "directional biases, being shortened along the axes and elongated
on the diagonals between opposite cube vertices." and why it causing "axis-aligned clumping".

Could anyone help me out? Thank you.

Very briefly, Perlin noise ideally is meant to have, on average, the same frequency characteristics no matter where in the texture you are or in what direction you are looking.

However, the way it's done is to define pseudo-random values (and derivatives) on a regular 3D grid which introduces some compromises.

To make this easier to understand, imagine instead using just a 2D grid, and on every grid location with an even-numbered X coordinate, you set the value to 0.0, and on every odd X, 1.0.  So at (0,0), (2,0), (4,99) etc you have 0.0, while at at (1,0) etc, you have 1.0.

If you followed a horizontal line, e.g (0,0) (1,0) (2,0) etc, it would go up and down, in a sine-wave-like pattern every 2 steps, but if you followed it in a diagonal direction,  (0,0), (1,1), (2,2,) then, per distance traveled, it would go up and down more slowly.

I believe that is what Perlin is referring to.


Is it possible to do color conversion on decoded video without getting it from GPUmemory to CPU memory? Decoding video will result in different pixel formats on different GPUs and drivers. I have an application which can render (using OpenGL), only the RGB8 pixel format, therefore I need to do color conversion from the decoded pixel format to RGB8. I can access the gl... functions in this program but I cannot edit the shader, so I'm stuck with RGB8.

This ffmpeg example demonstrates how to do hardware decoding: https://github.com/FFmpeg/FFmpeg/blob/release/4.2/doc/examples/hw_decode.c

At line 109 it does this:

/* retrieve data from GPU to CPU */
            if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0)) < 0) {

I want to avoid this because it takes time. Therefore, I need a way to reuse that decoded video, which is in GPU memory, to redo color conversion.

Is it possible to create a shader program that takes input from the GPU memory, not CPU memory? If I want to pass image from CPU to my shader, I simply use glTexImage2D. Is there a similar way to do it but from GPU memory?

Can I do even deeper and render the color decoded video on screen without copying from GPU memory too?

If what I wanna do is possible, are there shader programs available to color convert pixels for some major graphics cards/drivers? I don't want to support all of them, but I'd like to support at least the Jetson Nano board from NVIDIA.

So it sounds like the main issue you're having is how to render something when all resources are already on the GPU. I would do it like this:

Create the destination RGB texture that's the proper width and height
Create an FBO
Set the texture created in step 1 as the output (write) texture of the FBO
Bind the NV12 texture to the texture unit you'll use in your shader
Draw a textured quad that covers the entire output
Apply a shader that samples the NV12 texture to generate the RGB value at each fragment

For step 1 you use glGenTextures() and glTexImage2D().

For steps 2 and 3 you use glGenFramebuffers() and glBindFramebuffer() passing glBindFramebuffer() the texture created in step 1 to attach it to GL_COLOR_ATTACHMENT0. (See here for more details.)

For step 4 you use glBindTexture().

For step 5 you can simply draw a quad that goes from (-1, -1) to (1, 1). You don't need any model view matrix or anything.

For step 6 you'll need to calculate where the red, green, and blue samples that correspond to the current output pixel are in the input texture and then sample them using the texture() function. You'll need to do 3 different look-ups for each output pixel. This link has C code for figuring out where each sample is from.

Once you're done you can delete the frame buffer and use the texture created in step 1 in your next step (display it, process it, save it to disk, or whatever).


Why are reflections in a spherical surface sharp, regardless of depth of field? A question has come up recently in Blender SE, remarking on the absence of blur due to depth of field in reflections in a spherical surface. This is a real-life effect.. can anyone come up with a nice explanation of it?

There's no visible blur for the same reason you often don't see blur in very wide angle lenses: the circle of confusion is much smaller than a pixel. But it is still there. An extreme example of this is fisheye lenses.

If you had a camera with infinite resolution, such that you could zoom into the reflection on the ball, you would eventually see the blur. And it would be exactly the same blur as if you had aimed a flat mirror at the part of the scene you're looking at through the ball.

Now, we don't have such a camera, but we have something even better: software! So I built this simple setup with two perfectly sharp checkered planes, a sphere and a camera with some depth of field:

Rendering it with some depth of field at 512x512, you'd think the reflection in the sphere is perfectly sharp:

But here is a central crop of the exact same scene rendered at 8192x8192:

And we can see the exact same blur on the checker pattern. It's as simple as that: you don't see the blur because it's too small.


Issue with loading a texture on the GPU with OpenGL 4.1 Core profile I’ve been trying to render a texture on the whole screen in OpenGL 4.1 using the Core profile and followed a few tutorials on the matter, but all I have so far is a blank screen.

I tried using my texture coordinates as color components and got the result I expected so the vertice data must be in order, but using the dimensions of the texture as color components I determined the size of the texture in my shader is 1x1 when it should be 2x2, so I guess I am doing something wrong when uploading it to the GPU.

This is my pixel data:

const GLubyte pixels[] = {
    255, 255, 255, 255,     0, 0, 0, 255,
    0, 0, 0, 255,           255, 255, 255, 255,
};

This is how I load my texture:

glGenTextures(1, &texture);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 2, 2, 0, GL_RGBA, GL_UNSIGNED_BYTE, (GLvoid*)pixels);

I also tried using a buffer:

glGenTextures(1, &texture);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glGenBuffers(1, &texture_buffer);
glBindBuffer(GL_PIXEL_UNPACK_BUFFER, texture_buffer);
glBufferData(GL_PIXEL_UNPACK_BUFFER, sizeof(pixels), pixels, GL_STATIC_DRAW);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 2, 2, 0, GL_RGBA, GL_UNSIGNED_BYTE, (GLvoid*)0);

And this is how I render:

glUseProgram(shader_program);
// Binding my vba and vbo, uploading vertices.
glUniform1i(texture_location, 0);
// Enabling vertex attrib arrays.
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
// Drawing arrays, disabling vertex attrib arrays.

I have also put the whole source file on pastebin, it uses GLFW for windowing and does some basic error checking.

I’d be really grateful if someone could point out what I am missing here.

You seem to not change GL_TEXTURE_MIN_FILTER and not fill all the mipmap layers. This will usually result in the texture not being applied, since by default the filter uses mipmapping.


How does Primary Sample Space Metropolis Light Transport work? For simplicity, assume we're only dealing with surfaces which have either a Lambertian or perfectly specular material. Morever, assume that the only type of lights are area lights (i.e. surfaces with an emissive material) and we stop tracing at lights.

In path tracing, we start by tracing a ray from the eye through a random point on the area of the view plane corresponding to the $i$th pixel. Now, depending on the BRDF of the hitted surface, we choose either a uniformly distributed direction on the hemisphere over the normal at the hit point (in the case of a Lambertian surface) or the direction of perfect mirror reflection (in the case of a perfectly specular surface). This process stops, when either no surface or a surface with an emissive material (i.e. an area light) was hit.

Quesiton 1: Now, PSSMLT constructs a sequence $(X_n)_{n\in\mathbb N}\subseteq[0,1]$. Actually, it's even clear to me how this sequence is modified by small step and large step modifications. However, what are the components $X_n$ exactly? Are $(X_1,X_2)$ the (normialized) coordinates of the randomly chosen area of the view plane corresponding to a pixel? If so, which pixel? And what is $(X_3,X_4)$? Maybe randomly chosen points on a unit square which need to be mapped to the hemisphere of the corresponding hit point? If so, we would need to store the normal and the BRDF together with $(X_3,X_4)$, don't we? And what if the surface which was hit has a perfectly specular material? Are then $(X_3,X_4)$ simply ignored?

Question 2: I've read that initial sequence $(X_n)_{n\in\mathbb N}$ is chosen by approximating the "luminance of path contribution" which is said to be independent of the pixel. I don't get that at all. And I have no idea how it is approximated by using path tracing.

PSSMT operates directly on the space of random numbers that generate valid light paths. As such, mutations in the unit hypercube lose their physical interpretation since they do not have direct knowledge of the actual light path constructed. Recent research in rendering has shown that it is possible to bridge the gap between path space (that acts directly on the paths) and primary sample space (that acts on uniform variates) to some extent. See Bitterli 2018, Pantaleoni 2017 and Otsu 2017 if that interests you.
I am not entirely sure what you mean here, but typically an initial path of length $k$ will be formed using path tracing, and thus random numbers will be used along the way ($k+2$ if you are using BDPT). PSSMLT will mutate this path by acting only on the random numbers. Path throughput is indeed independent of the pixel but you only care about paths actually reaching the eye. At the end of the day, you are importance sampling paths according to their contribution to the final image, and luminance is what is commonly used.


What's the difference between irradiance and fluence/radiant exposure? A paper$^1$ I'm reading says fluence measures the incoming radiance from all directions and that fluence is similar to irradiance. It's defined by $\phi(x) = \int_{4 \pi} L(x, \vec{\omega'}) d\omega' $ (The tick mark seems to emphasize $\vec{\omega'}$ varies over the integration, rather than being a fixed direction.)

Intuitively, radiance is the amount of light given off in a direction but attenuated by how far off the surface is from being perpendicular to the light direction. $L=\frac{d^2\Phi}{d\omega dA \cos \theta}$

Irradiance is the area light density of a surface. $E=\frac{d\Phi}{dA}=\int_{2 \pi } L_i(\vec{\omega}) \cos \theta d\omega$

Intensity is the amount of light hitting a surface from a particular direction. $I(\vec{\omega})=\frac{d\Phi}{d\omega}$

It also looks like you can write the radiance as irradiance times intensity.
$L=E\cdot I(\vec{\omega})$

Looking at the formulas, it seems like irradiance sums radiance over the hemisphere over a surface point and attenuated by how much the surface orients toward the light source, while fluence sums up radiance over the whole sphere surrounding a point (to measure inscattering).

Wikipedia says, "radiant fluence is the radiant energy received by a surface per unit area, or equivalently the irradiance of a surface integrated over time of irradiation." But this definition doesn't seem to match the one given above since there's no time involved.

https://www.researchgate.net/post/What_is_the_definition_and_difference_between_light_dose_and_light_fluence_rate

http://omlc.org/education/ece532/class1/irradiance.html

Real-Time Rendering of Translucent Materials with Directional Subsurface Scattering, Alessandro Dal Corso, 2014

I can only cite what I have learned in my lecture on Global Illumination techniques which was unfortunately some time ago:

Radiant Power : The amount of energy emitted by a light source in unit time. denoted by $\Phi$ and is measured in Watt which equals to Joules per second. (This does not specify any area!)
Irradiance: The irradiance denotes the incident radiant energy on a surface. It is defined as $E=\frac{d\Phi}{dA}$ - This quantity denotes the incoming energy per area and is measure ind $\frac{W}{m^2}$ Note here: radiant energy is measured in Watt which is Joules per second!
Radiant exitance: This is the reverse direction of Irradiance and is the energy an area emits in unit time and has the same definiton and units as Irradiance only with reverse direction $M = B = \frac{d\Phi}{dA}$. This is also often called radiosity B.
Radiance: Is the power per projected area per solid angle. It is defined as $L=\frac{d²\Phi}{d \omega * dA * cos\theta}=[\frac{W}{sr*m^2}]$ where $\omega$ denotes the solid angle, $\theta$ denotes the angle between emitting surface normal and the direction the energy is traveling to. So this is not attenuated by the distance but by the angles.

In general as I learned the distance between emitting and receiving surface is only important for Radiance because the projected area will be smaller if they are farther apart.

The time you are missing in your definitions seem to come from the conversion of Watt to Joules Per Second.


What methods/technologies to reduce required performance for virtual reality are there? I'm interested in virtual reality, but according to some sources, less than 1% of computers in use today have the necessary performance to run modern VR games, granted many of them are not intended for gaming, that's still a huge barrier for the widespread adoption of VR headsets into the market. So I was wondering what could be done to reduce the performance requirements of a VR game that hasn't been implemented with such optimizations in mind.
I'm talking about technologies like foveated rendering, which (ab)uses the fact that we only ever perceive a few degrees in the center of our field of vision as sharp while everything else is blurry. This can be combined with eye tracking so that only the areas where the player is looking are rendered in full quality, while we can reduce the render quality and therefore the performance requirements everywhere else.
This seems like a great advantage and I honestly wonder why none of the major VR companies have implemented eye tracking into their headsets and foveated rendering into their SDKs into their headsets. The technology exists and I can't imagine that it would add much to the price tag.
Paper about foveated rendering
There's also instanced stereo rendering which basically renders things to both eyes at the same time rather than rendering it for each eye individually.

It's also possible to use just one frustum for both eyes rather than two as discussed here.
VR and frustum culling
But are there other approaches to the performane problem?

Avoid stereo when possible
There was some research published recently measuring when users can or cannot tell whether the specular contribution, which is view point dependent, is different between eyes.
Perception of Highlight Disparity at a Distance in Consumer Head-Mounted Displays
In the future, this could be use to share some computation between the two eye renders. Unfortunately this sounds hard to take advantage of in a typical rasterization pipeline.
Some games already avoid stereo when rendering far objects and background.

Reduce shading when possible
Lens distortion
Part of the NVidia VRWorks SDK is a feature, Multi-Res Shading, that allow to reduce the amount of shading, to take into account the fact that some pixels contribute less to the final image due to lens distortion.
In Alex Vlachos' GDC 2015 presentation, he also mentions the idea of rendering the image in two parts of different resolution, to achieve the same goal.
Foveated rendering
Finally, like your question mentions, foveated rendering aims at scaling the resolution to take into account the fact that eyes have more precision in the fovea and less in the periphery. There is some research on the topic, but this requires eye tracking like in the Fove HMD.


Conversion from cubic catmull-rom spline to cubic b-spline I have a bunch of points that are the control vertices of a cubic catmull-rom spline. I would like to convert these to the control vertices of a cubic bspline.

I believe I can do this using this following relation:
$\boldsymbol{t} * \boldsymbol{B}_{\text{catmull}} * \boldsymbol{V}_{\text{catmull}} = \boldsymbol{t} * \boldsymbol{B}_{\text{bspline}} * \boldsymbol{V}_{\text{bspline}}$

where $\boldsymbol{B}$ is the basis matrix and $\boldsymbol{V}$ is a vector of the control vertices. To get the control vertices of the bspline, the formula just becomes:

$\boldsymbol{V}_{\text{bspline}} = \boldsymbol{B}^{-1}_{\text{bspline}} * \boldsymbol{B}_{\text{catmull}} * \boldsymbol{V}_{\text{catmull}}$

My question is once I process say 4 control vertices of the catmull rom spline, to get 4 vertices of the bspline, how do I process other vertices (i.e the order). If a catmull-rom spline has n-vertices with cubic segments, then we have at the most $n-3$ segments. So if we have say 5 vertices, $v_0, v_1, v_2, v_3$ gives us 4 vertices for bspline. How would I go about computing the other curve segment in the bspline?

Quoting the comments above for context:

Just to confirm, are you asking, given a set of $N$ CatRom control points,
$$\{CR_0, CR_1, CR_2, CR_3 ... CR_{n-1}\}$$
forming a piecewise curve, what is the equivalent $N$ points, $$\{B_0,B_1,B_2...B_{n-1}\}$$ for a matching piecewise uniform cubic bspline?

Exactly. That is what I'm looking for. I think conversion to cubic bezier's are easy. But I would like to know if this is even possible/doable with bsplines

I think it's a bit complicated: Using the approach you described above, i.e. $$ V_{targetformat} = M_{targetformat}^{-1} . M_{sourceformat}.V_{sourceformat}$$
you can easily go from a sequence of $n$ control points of Catmull-Rom to  $(n-3)$ sets of 4 Uniform BSpline control points, i.e. a total of $4n-12$ control points , but I can't see that this will allow you to stitch them together to obtain just $n$ points...
Assuming I've got the correct matrices for the 'typical' Catmull-Rom,
$$M_{CR}=\frac{1}{2}\begin{bmatrix}
-1 & 3 & -3 & 1\\
2 & -5 & 4 & -1 \\
-1 & 0 & 1 & 0 \\
0 & 2 & 0 & 0\\
 \end{bmatrix}$$
and uniform B-Spline:
$$M_{UB}=\frac{1}{6}\begin{bmatrix}
-1 & 3 & -3 & 1\\
3 & -6 & 3 & 0 \\
-3 & 0 & 3 & 0 \\
1 & 4 & 1 & 0\\
 \end{bmatrix}$$
then $M^{-1}_{UB}.M_{CR}$ should be
[UPDATE: Fixed errors in coefficents]
$$\frac{1}{6}\begin{bmatrix}
7 & -4 & 5 & -2\\
-2 & 11 & -4 & 1 \\
1 & -4 & 11 & -2 \\
-4 & 13 & -26 & 11\\
 \end{bmatrix}$$
It seems to me that, in any batch of four points, every output vertex thus depends on every input vertex, so it seems the neighbouring points can't be shared.
Having said this, I think perhaps there might be a way of getting an approximate solution (but I've not tried rendering it myself to check it so take with a grain of salt)
We know that a Catmull-Rom spline defined by $(CR_A, CR_B, CR_C, CR_D)$, passes through $CR_B$ and $CR_C$ at t=0 and t=1 respectively.
Similarly, using $\begin{bmatrix}t^3 & t^2 & t & 1\end{bmatrix}.M_{UB}$, we should see that at t=0 and t=1, a B-spline passes through $\frac{B_A + 4 B_B + B_C}{6}$ and $\frac{B_B + 4 B_C + B_D}{6}$.
First consider trying to go from BSpline to CR. If we had a 'loop' formed by N control points one would map from BSpline to CR with...
$$\begin{bmatrix}CR_0\\CR_1\\ ...\\CR_{n-1} \end{bmatrix} = M_{NxBtoCR}.\begin{bmatrix}B_0\\B_1\\ ...\\B_{n-1} \end{bmatrix}$$
where
$$
M_{NxBtoCR}= \frac{1}{6}.\begin{bmatrix}
4 & 1 & 0 & 0 & ... & 0 & 1\\
1 & 4 & 1 & 0 & ... & 0 & 0\\
0 & 1 & 4 & 1 & ... & 0 & 0\\
: & : &   &   & ... & : & :\\
1 & 0 & 0 & 0 & ... & 1 & 4\\
\end{bmatrix}
$$
If we invert that matrix to get $M_{NxBtoCR}^{-1}$, one should then be able to map the other way.
Using a maths package such as Maple, one finds that each row of $M_{NxBtoCR}^{-1}$ consists of the same set of values, all non-zero, which are progressively offset in each row. If we ignore the smaller terms, it suggests that $$B_i \approx 1.732 CR_i -0.464 (CR_{i-1}+CR_{i+1}) + 0.1243 (CR_{i-2}+CR_{i+2}) - 0.033(CR_{i-3}+CR_{i+3}) $$
There's probably some nice algebraic expression for this but I hope this might "do the trick".
Update Those weights don't quite sum to 1.0 - you'll need to normalise them slightly.
Obviously, this has assumed the curve is a loop. It's probably possible to adjust the weights for the end points but I'll leave that as an exercise for the reader.


Converting cartesian pixels to polar pixels (I've largely revamped this entire question, though the motivation remains the same.)

revised question

I want to convert a raster of cartesian pixels into polar pixels.  Is there a sensible algorithm for doing this?  For example, how do I compute the value of the shaded (polar) pixel in the image below, given the value of the three (cartesian) pixels that it overlaps?

original question

Is there a reasonable way to compute the area of the intersection of a square and an annular section, as shown in the orange section below?

The motivation: I have a raster of square pixels, and I'm converting it to "polar pixels" -- I want to find out the contribution of each cartesian pixel to each polar pixel.

I have implemented the cartesian-to-polar-conversion and have used different interpolation methods:

1) nearest neighbor

2) a subsampling approach, which averages 81 subpixel locations

3) bilinear interpolation

The 2nd row in the image below shows detail magnifications of the output for the three approaches:

Here is the GLSL shader code for approach 1 and 3:

precision mediump float;
varying vec2 tc; // texture coordinate of current output pixel
uniform sampler2D inputImg; // input image
const float PI = 3.141592653589793238462643383;
void main() {
  float phi = tc.x; // phi is x-direction in output
  float r = tc.y;   // radius is y-direction in output
  float xx = r * cos(2.0 * PI * phi);  // unit circle in range [-1.0, 1.0]
  float yy = r * sin(2.0 * PI * phi);
  float x = xx * 0.5 + 0.5; // input texture coordinate in range [0.0, 1.0];
  float y = yy * 0.5 + 0.5;
  gl_FragColor = texture2D(inputImg, vec2(x,y));
}

This is the GLSL shader code for approach 2:

precision mediump float;
varying vec2 tc; // texture coordinate of current output pixel
uniform sampler2D inputImg; // input image
uniform int outputWidth; // width of output image
uniform int outputHeight; // height of output image
const float PI = 3.141592653589793238462643383;
const int samplesPerSide = 4;
void main() {
  int samplesPerDirection = samplesPerSide * 2 + 1;
  // compute size of subsample step in texture coordinates
  float sampleStepX = 1.0 / float(outputWidth * samplesPerDirection);
  float sampleStepY = 1.0 / float(outputHeight * samplesPerDirection);
  vec4 sampleSum = vec4(0); // init sum to zero
  for(int i = -samplesPerSide; i <= samplesPerSide; i++) {
    for(int j = -samplesPerSide; j <= samplesPerSide; j++) {
      float phi = tc.x + float(i) * sampleStepX; // phi is x-direction in output
      float r = tc.y + float(j) * sampleStepY;   // radius is y-direction in output
      float xx = r * cos(2.0 * PI * phi);  // unit circle in range [-1.0, 1.0]
      float yy = r * sin(2.0 * PI * phi);
      float x = xx * 0.5 + 0.5; // input texture coordinate in range [0.0, 1.0];
      float y = yy * 0.5 + 0.5;
      sampleSum += texture2D(inputImg, vec2(x,y));
    }
  }
  gl_FragColor += sampleSum / float(samplesPerDirection * samplesPerDirection);
}

Approach 2, which is approx. factor 81 slower than 1 and 3, is probably the one you are looking for but I also like the bilinear interpolation result.

With a WebGL capable browser you can try out these implementations here:

https://www.gsn-lib.org/index.html#projectName=CartesianToPolar&graphName=CartesianToPolar


What is the difference between various space-filling curves? Space-filling curves are important in many graphics applications because they help expose spatial locality. We often hear about different algorithms using Z-curves, Morton codes, Hilbert curves, etc. What are the differences between some of these different curves and how do they apply to various applications?

The difference is how well a mapping preserves locality and how easy it is to encode/decode the keys. The paper "Linear Clustering of Objects with Multiple Attributes" by H V Jagadish says: "Through algebraic analysis, and through computer simulation, we showed that under most circumstances, the Hilbert mapping performed as well as or better than the best of alternative mappings suggested in the literature". On the other hand, z-order is a bit simpler to use, for example compare the various methods listed in Bit Twiddling Hacks for z-order and Wikipedia for Hilbert-order.

As for the applications, I think the main advantage in using space filling curves is that they map points from higher dimensional space to space of lower dimension. For example, they make it possible to window query for points using traditional B-tree database index. Again, on the other hand, the disadvantage is that one needs to know the bounds of the input in advance as it is difficult to "resize" the mapping later.

PS: "Z-curve" is the same as "Morton code".

PPS: Additional mappings include Peano curve and for applications see also Geohash.


Sharing calculation result between shader programs in OpenGL Question

Is there a way to compute something in a first shader program (eg. screen-space coordinates from word-space ones) and access those results in an another shader program?

Possible solutions

Give up multiple-shader-programs architecture and create one combined shader program
Save somehow results to a buffer objects (I couldn't find out if this is possible except computation shaders)
Render to texture and read the texture

Is there a nicer solution?

Yes you can, this is how deferred rendering for example would work. For that you would render to a texture in one pass, in openGL this would be via an FBO, then feed that texture as the input to the next pass/shader.


What is the relation between Aliasing and Flickering? So in the class I've learned that Aliasing refers to the jagged edges resultant from the discrete nature of computer graphics way of representation.

Also, I know that Anti-aliasing refers to a technic (mainly of blurring) to remove (our camouiflage) aliasing.

But I was presented a question about "A picture flickering in a game room when the user moves" and the answer was given as being an aliasing problem.

I did not get the relation between flickering and aliasing. Can someone clarify it to me?

Flickering can be a form of temporal aliasing. It's a similar phenomenon to spatial aliasing such as jaggies, but it occurs in time instead of space.

For instance, a common cause of image flickering in graphics is when the camera or geometry is in motion, and geometric features fluctuate in pixel size as they move. For example, imagine a railing with thin vertical bars. Depending where a bar appears relative to the pixel grid, it might get rendered as 2 pixels wide, only 1 pixel wide, or it might not appear at all. And in motion, it may rapidly fluctuate between these states, creating a visually objectionable flicker.

Another common cause of image flickering is specular surfaces with a bumpy normal map and a high specular power. The specular highlights can flicker in motion, due to their alignment with pixels changing from frame to frame.

Antialiasing strategies that address only spatial aliasing will often produce an image that looks good in a static screenshot, but turns into a flickery mess as soon as things start moving. This is one reason why temporal antialiasing has become popular in games in recent years.


What is the equation for flat shading? Is there an equation for flat shading?

To my understanding, it is this:

$$I = I_a \times K_a + F_{att} \times  I_l \times  (K_d \times  (N \times  L))$$

Final intensity = ambient reflection x ambient reflection coefficient + (attenuation x incident light intensity x (diffusive reflection coefficient x (surface normal x Light intensity))

However, I have only found one source for this equation (my lecture slides).

Are there any other sources to confirm this is the equation for flat shading? I can't find one.

I don't think having a equation for flat shading is meaningful as such. Mainly because the formula for flat shading is same as the formula for the model the flat shading is emulating. The difference is that flat shading is operating per primitive rather than per point or per vertex.

So there can in fact exist multiple formulas that work differently but are still all flat shading.


Display that supports full CIE xyY color space Right now, all current monitors use three (or four) primary colors to create their color, which could never allow them to produce all colors that can be seen by the human eye; in fact, their colors will always be inside a triangle on the xyY diagram of all possible colors.

But we could decide to replace these color filters with something such as Lyot filters, which can be tuned, similarly to LCDs, to allow only light whose wavelength is higher/lower than a certain value to pass through.

This would mean that we could basically create color filters that can be adjusted to produce any color, by replacing each pixel with two stacked filters, which allow a range $[a, b]$ to pass, and a third filter next to it which allows the range $[c,\infty)$ to pass through.

Using Mathematica, I have tested that a mapping between these three values is enough to create any possible $x$, $y$ and $Y$ in the CIE xyY color space:

So, in other words, the range of
$$\text{XYZ}(a,b,c)=\int_{[a,b]\cup[c,\infty)}\text{I}(\lambda)\cdot(\overline{x}(\lambda),\overline{y}(\lambda),\overline{z}(\lambda))$$

, where $\text{I}$ is the illuminant, is enough to produce the color of any object that can be illuminated by that illuminant.

So, my questions are:

Are there already any such displays that can produce any visible color, and if not, then why not?
Could this actually be done in practice?

No, most likely these kinds of displays do not exist, at the time of writing this post. Although the negative is hard to prove. And in this case i don't have deep enough understanding of the lore.

Reason for me saying this is that usable displays need some serious miniaturization. The miniaturization process is quite expensive and requires some serious investment. There should be quite massive patent trail if this was indeed true. So we should look in patent databases for this trail.

Second, having such tech probably will not yield full color space, at least initially. Because the filter can only filter out colors that the back light can generate. Most probably therefore this is some kind of projector technology at first. Also filters probably have a practical limit to how much they can be tuned.

Third, if/when such tech is available it must recuperate the development cost by selling enough units to overcome the capital investment. There is a likelihood that this is one of those things that can not be produced without the miniaturization step like digital cameras. So keep a lookout for a killer application for this tech.

Some major downsides of this kind of tech is the fact that you filter out light. This means it eats up part of the energy input. This leads to a heat dissipation problem, which for perfect narrow band filters is actually pretty huge part of the emitted light. Lot of energy goes wasted, and needs to be engineered around.


GLFW | What is a frame? Here I was reading about buffers and found such text:

  When the entire frame has been rendered, the buffers need to be swapped with one another, so the back buffer becomes the front buffer and vice versa.

And also this

  The swap interval indicates how many frames to wait until swapping the buffers, commonly known as vsync.

The question is: what is a frame? And how does it differ from buffer?

Conceptually, a frame is the image you're trying to create, presumably for the purpose of display. It is the product of your entire rendering operation for a specific slice of time.

A "buffer" is the memory for the image you either are trying to create or have already created. Hence the common term: framebuffer. Indeed, the choice of the word "buffer" is because you are typically buffering multiple images. That is, one buffer will currently be in the process of being displayed, while another buffer is being rendered.


face normal consistencies I am going through code snippet that reads in a triangular model, calculates & store the normal with each Triangle object and eventually check the consistencies of winding order. Let us define a triangle with the vertices v1, v2 and v3. The face normal of the triangle is calculated as follows:

void calculateFaceNormal()
{
   //vector from v3 to v1
   double dx13, dy13, dz13;

   //vector from v3 to v2
   double dx23, dy23, dz23;

    dx13 = v1.x_ - v3.x_;
    dy13 = v1.y_ - v3.y_;
    dz13 = v1.z_ - v3.z_;

    dx23 = v2.x_ - v3.x_;
    dy23 = v2.y_ - v3.y_;
    dz23 = v2.z_ - v3.z_;

    // Cross product gives normal.
    x_ = dy13*dz23 - dy23*dz13;
    y_ = dz13*dx23 - dz23*dx13;
    z_ = dx13*dy23 - dx23*dy13;
}

Each Triangle object store the references of its neighboring triangles as follows:

class Triangle()
{
  public:
    .......
    .......

    const Triangle *edge12Granne_;
    const Triangle *edge23Granne_;
    const Triangle *edge31Granne_;

    bool isGranne12NormalConsistent() const;
    bool isGranne23NormalConsistent() const;
    bool isGranne31NormalConsistent() const;
};

Now I am writing down only one of the functions whose definition is not clear to me and I believe that if someone could explain the theories behind it,I shall be able to understand the rest of the other two functions.

bool Triangle::isGranne12NormalConsistent() const
{
    // Find opposing vertex in neighbour.
    if (edge12Granne_->edge12Granne_ == this)
    {
        /*
         * two triangle are neighbors to each other by
         * the adjecent edge that constitutes both the
         * triangles.
         * */

        // v2 | w1
        // v1 | w2
        if (v1_.id_ == edge12Granne_->v2_.id_)
        {
            // Ok (reversed order compared to this.)
            return true;
        }
        else
        {
            // Not ok, same order!
            return false;
        }
    }
    else if (edge12Granne_->edge23Granne_ == this)
    {
        // v2 | w2
        // v1 | w3
        if (v1_.id_ == edge12Granne_->v3_.id_)
        {
            // Ok (reversed order compared to this.)
            return true;
        }
        else
        {
            // Not ok, same order!
            return false;
        }
    }
    else
    {
        // v2 | w3
        // v1 | w1
        if (v1_.id_ == edge12Granne_->v1_.id_)
        {
            // Ok (reversed order compared to this.)
            return true;
        }
        else
        {
            // Not ok, same order!
            return false;
        }
    }
}

The function in your third code snippet is checking for the winding order to be consistent between two neighboring triangles. It does this by looking at the order in which vertices are listed in their shared edge. If the two triangles have consistent winding (both counterclockwise for instance), they will have opposite orientations for the shared edge.

A diagram may help see this:

The vertex order is indicated by the blue arrows:

t0: v0, v1, v2
t1: v1, v3, v2

If you look at the shared edge (v1–v2), it appears in the first triangle with order v1, v2, and in the other with order v2, v1 (cycling around the end of the vertex list back to the beginning).

The function in your snippet takes a triangle and looks at one of its neighbors, checks all three possibilities for which edge in the second triangle is the shared one, and checks that the vertices in that (other triangle's) edge appear in the reverse order to the vertices in the current triangle's edge. It returns true if this is the case, and false otherwise.


How to generate a 2d histogram? I'm playing around with volume rendering and looking into multi dimensional transfer functions. I keep seeing images like this for 2d transfer functions:

I've read numerous papers and articles but haven't gotten very far because I can't seem to understand how the image on the right is generated from the volume data set. Also what would you call this image? From my understanding the two axis are data values (intensities) on the x and gradient magnitude on the y. But where is the gradient magnitude coming from? I have a basic understanding of gradient magnitude and don't see how it can be used to make that picture.
How is it calculated? How can I generate a picture like the one on the right?

Revisiting this question from awhile ago. It turns out this sort of histogram is created by iterating through every pixel value in both the normal image and gradient magnitude image plotting it in the 2d image. (x intensity, y gradient magnitude). The count/occurrences of the 2d histogram are represented by the intensity of the grayscale in figure b. So the brighter a pixel in figure b is, the more pixels with that (intensity, gradient magnitude).

For volume rendering this is called a 2d transfer function which lets users filter out or assign lower opacities to pixels with large/small gradient magnitudes. This is especially helpful in real time volume rendering with lighting. Real time volume rendering typically uses a BSDF at every point along a ray march as an approximation (Assuming that light reaches every point in the volume). IE: Dot product of the incoming light direction with the gradient as the normal. Using the gradient as the normal means that homogeneous areas are not well represented as they have low gradient magnitudes. Small noise in these areas can easily cause the gradient and normal to flip.


Could we dispense the near clipping plane? Would it be, at least theoretically, possible to render without a near clipping plane? Could raytracing ignore this perchance?

Quite many budding 3d artists have a WTF moment the first time they encounter the clipping plane. If this could be avoided then it would be a big win for usability of 3D authoring apps. Its not one of those things a normal artist would think about until explained.

The near clipping plane in rasterizing setups with perspective projection is there to avoid a divide by 0 and bound the possible depths for orthogonal projection.

With a bit of extra math you can make the depth stored in the depth buffer linear again. However you still need the guard against the divide by 0.

In raytracing the near plane can be at 0 no problem. There is nothing technical stopping you from putting it at -10 however (besides that it would be very odd).


Why does my code generate a figure whose kurtosis looks a little higher than the one I am trying to reproduce? this figure (fig_1) is an univariate normal distribution in 3D, the corresponding equation is in the top left hand corner

it seems that x varies from -5 to 5; sigma varies from 0 to 4.

This Python code is trying to reproduce this figure

ax = plt.figure().gca(projection='3d')
xx, yy = np.meshgrid(np.arange(-5,5,.1), np.arange(0.1,4,.1))
zz = norm.pdf(xx,0,yy)
ax.plot_surface(xx, yy, zz, alpha=.5)

and I got this figure

and this guy

zz = norm.pdf(xx,0,np.sqrt(yy))

gives this figure

which is better although the kurtosis looks a little higher than fig_1.

any clue

The value 's' in the original graph is the variance, which is the square of the Gaussian sigma (i.e. the scale parameter of the normal distribution). So norm.pdf(xx,0,np.sqrt(yy)) should be correct.

Note that the shape of the peak you see is very much dependent on the y value where the graph starts. Your starting point of 0.1 may be a little smaller than in the original graph, which looks like it starts at about 1/6 ~= 0.167, so you may see a narrower and taller peak for that reason. Also, the original graph doesn't quite have enough horizontal resolution to resolve the shape of the peak entirely, as you can see from how far apart the grid lines are spaced vertically on the sides of the peak.


Converting color to equivalent with opacity Assume a color X with alpha 1 and RGB 255,230,210.

How can i find another color with alpha A ( or an array of colors ) that blended with a background of color Y ( e.g 255,255,255 ) would produce the original color  ?

This depends on what function you're blending with. Since you don't say where you're doing this, I'll just take OpenGL as an example.
As you can see in the API, there are several ways of blending two colours (or rather how to interpret these two colours for a given blend function).
You speak of a colour and a background colour, so let's say your result is $c_r$, your object in the foreground has colour $c_s$ (for Source, the new colour value being "added" someway) and your background has colour $c_d$ (for Destination, the old colour value that has been in the buffer already).
Let $c_s = \left(\begin{array}{c} R_s\\G_s\\B_s\\A_s \end{array}\right)$ (and analogously for $c_r$ and $c_d$).

To determine the blended RGBA values of a pixel, the system uses the following equations:
$R_d = min(k_{R'}, R_s s_R + R_d d_R) \ G_d = min(k_{G'}, G_s S_G + G_d d_G) \ B_d = min(k_{B'}, B_s s_B + B_d d_B) A_d = min(k_{A'}, A_s s_A + A_d d_A)$

A bit confusing with the doubled $R_d$ and so forth, so I'll rewrite it to the denomination I defined earlier:
$R_r = min(k_{R'}, R_s s_R + R_d d_R) \ G_r = min(k_{G'}, G_s S_G + G_d d_G) \ B_r = min(k_{B'}, B_s s_B + B_d d_B) A_r = min(k_{A'}, A_s s_A + A_d d_A)$
The min function is there for not going above a certain constant (and $k_{R'},\ k_{G'},\ k_{B'},\ k_{A'}$ are simply constants). For our purposes, let's just assume that we don't surpass this threshold, so we end up with
$R_r = R_s s_R + R_d d_R$ (and of course the same for the other channels).
The rest of the components $s_R$ and $d_R$ which are some scaling factors for combining the two colours.

Transparency is best implemented using blend function (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) with primitives sorted from farthest to nearest.

This helps, because now we know what the factors $s_R$ and $d_R$ actually are: the alpha values of the Source.
This leaves us with
$R_r = R_s A_s + (1 - R_d A_d)$
Therefore, to come back to your example, we have (for the green channel, since that is not $255$):
$G_r = 230$, an undefined $\alpha$ value for the source, the background green $G_d = 255$ and probably a solid background, meaning $A_d = 255$. We need the values to be in $\left[0, 1\right]$, therefore we get $G_r = \frac{230}{255} = 0.90196078431$, $G_d = 1$ and $A_d = 1$.
Filling this in we get:
$0.90196078431 = G_s A_s + (1 - 1 \cdot A_s)$
Your question lets me believe, that you know $A_s$, thus only $G_s$ (and of course the same for the other colour channels) is missing. This can then be calculated by isolating the respective colour component.


Multiple light pixel Shader in GLSL I have wirtten a fragment shader that works just fine with a single light. Now I am trying to adpat it to work with 8 lights, the implement it in Processing. Clearly I am doing something wrong in the math and I cannot see what it is... I have read other posts about this and try to adapt the answer to my problem, no luck though...

////Fragment/////

    #ifdef GL_ES
precision mediump float;
precision mediump int;
#endif

varying vec4 vertColor;
varying vec3 ecNormal;
varying vec3 lightDir;

void main() {
  vec3 direction = normalize(lightDir);
  vec3 normal = normalize(ecNormal);
  float intensity = max(0.0, dot(direction, normal));
  gl_FragColor = vec4(intensity, intensity, intensity, 1) * vertColor;
}

////vertex/////

        #define PROCESSING_LIGHT_SHADER

uniform mat4 modelview;
uniform mat4 transform;
uniform mat3 normalMatrix;

uniform vec4 lightPosition;
uniform vec3 lightNormal;

attribute vec4 vertex;
attribute vec4 color;
attribute vec3 normal;

varying vec4 vertColor;
varying vec3 ecNormal;
varying vec3 lightDir;

void main() {
  gl_Position = transform * vertex;
  vec3 ecVertex = vec3(modelview * vertex);

  ecNormal = normalize(normalMatrix * normal);
  lightDir = normalize(lightPosition.xyz - ecVertex);
  vertColor = color;
}

From the error message the problem is in line 24 of your fragment shader:

vec3 transformedNormal = normalize(normalMatrix * normal);

normalMatrix is a mat4 but normal is a vec3.

The solution is to make normalMatrix a mat3 instead.


Phong Modified BRDF Implementation According to: https://www.cs.virginia.edu/~jdl/importance.doc

For a specular sample: (page 4 of the document) $$pdf(\theta_{i}) = \frac{n+1}{2\pi} \cos^n(a)$$

And this is how you choose an incoming direction according to a specular sample of the BRDF (page 4 of the document):
$$ \vec{w_{i}} = (\alpha, \phi) = (\arccos(\varepsilon_{1}^{  \frac{1}{n+1}  }), \ \  2\pi\varepsilon_{2})$$

But according to the figure on page 4, a specular sample has a BRDF favoring a direction which is highly dependent on the incoming ray, but the $ \phi $ angle is decided by uniformly sampling a circle, disregarding completely the incoming direction, which to me makes no sense

Shouldn't the $ \phi $ angle favor the specular direction, which is dependent on the incoming ray? Why we're not using at all the incoming ray to calculate the sampled direction which is supposedly sampling the brdf?

No matter where I look for it, I haven't seen an implementation using a BRDF which takes into account the incoming direction, which makes me think I've misunderstood the theory behind it

Lastly, assuming a standard backward path tracer, is this pseudocode the right way of computing the results?

color =  float3(1,1,1)
ray   =  getRayFromCamera()

for NUM_BOUNCES:
    primitive = world.intersect(ray)

    // assuming primitive is non null, I'm omitting the code where we sample a background
    albedo           = primitive.albedo

    // computeOutGoingRay will return a Ray and a PDF
    outgoingRay, pdf = primitive.material.computeOutGoingRay(ray, normal, etc..)

    // of the incoming radiance, how much is scattered along the new direction?
    brdf             = primitive.brdf(outgoingRay, ray)

    color = (color * albedo * brdf) / pdf

    // assigning the new ray to keep our backward recursion
    ray   = outGoingRay

Consider the following scheme that mimicks the generation of a sample for phong shading. where V is the viewer direction, N is the normal, R is the specular reflection direction and $S_i$ is some sample. This space (commonly called tangent space) uses the normal as one of its basis vectors. you can think of it as the 'Z axis'

Now, when we generate samples for Phong shading wo do not generate them in this space.

Rather, we use a different reference system where the Specular reflection direction is the 'z axis', thus, our angle $\theta$ is the angle between the reflected direction and the sample direction, and the angle $\phi$ is the angle between some $S_{ix}$ that is the projection of our sample direction onto the plane perpendicular to $R$ and some arbitrary but constant vector perpendicular to our reflected direction

$\phi$ is a number that is sampled uniformly in the range $(0,2\pi)$ this means that rays will be generated around the reflected direction with no bias towards any particular direction.

Generated rays will, however, be biased to have a direction close to $R$ due to the way that $\theta$ is sampled.

When we generate rays in a backwards pathtracer we use the view direction to get the directions where the light is likely to come from, instead of throwing light around and seeing where it falls. The later technique is closer to photon mapping.


Canonical Space to Screen Space: Why do we have the strange -0.5? When we convert from canonical space to screen space,

To map x from -1 to 1 to 0 to width,

-1 < x < 1
0 < x+1 < 2
0 < x+1/2 < 1
0 < (x+1)w/2 < w
0 < xw/2 + w/2 < w

Similarly for y,

-1 < y < 1
0 < y+1 < 2
0 < y+1/2 < 1
0 < (y+1)h/2 < h
0 < yh/2 + h/2 < h

What is the reason behind choosing a screen that has (xmin, ymin) = (0.5, 0.5) and (xmax, ymax) = (1920 - 0.5, 1080 - 0.5) where (nx, ny) = (1920, 1080)?

I can't really answer this without the context so post some links where this is done.

What I can tell you is that the general formula for viewport transformation as done by OpenGL is given as (taken from wiki)

Here x,y are the what your picture says xmin, ymin and width, height is xmax, ymax. Now there is no specific reason I can think of why you would wanna do a shift of -0.5 but the viewport allows us to draw to a specific portion of the screen.

For example, in video games you would want to draw the minimap on the lower left corner of the screen, hence you could provide the values

x = 0, y = 0, width = 200, height = 200

This would render a 200x200 minimap in the lower left corner. Your second picture does this but instead it chooses a value of

x = -0.5, y = -0.5, width = nx, height = ny

If you substitute it in the above matrix you'll get

$ \frac{nx}{2} * x_{ndc} - \frac{1}{2} + \frac{nx}{2}$

for the $X$ value. This equals

$ \frac{nx}{2} * x_{ndc} + \frac{nx - 1}{2}$

Which is the same as what is done in the picture you posted. Hope it helped somewhat.

EDIT:- 0.5 offset is commonly used to get to the pixel center. See this image as an example.

Here the pixel 3,5 has the center 3.5, 5.5. This is the 0.5 offset you are seeing. I was confused at first when you were talking about an offset of negative 0.5.


Why is this not a proper solution to handling transparency in deferred rendering? From the  sources that I have read about deferred rendering they all mention "handling transparency is hard" without much more information.

However a solution that I see is, handle the scene once without the transparent geometry, once assuming the geometry is fully opaque, then interpolate between the 2 based on the transparency coefficient.

Is this a bad solution because it's slow due to the additional render pass or is it that there is something else I am not seeing?

PaulHK is right in what he said: you have to consider that there may be more than 2 transparent objects behind each other.

Also, the idea of deferred shading is to render the geometry only once to be more efficient. If you render the geometries multiple times, you lose (part of) your efficiency. Moreover, the lighting is deferred, thus you'd need to do lighting calculations for all your geometries before you can do the correct color calculations from transparency. So what you're really doing is going back to forward shading.

That being said, one solution is to first render any opaque geometry with deferred shading and afterwards render the transparent objects with forward shading, so maybe you weren't too far off. Those transparent objects however shouldn't be affected by too many lights to still render frames "fast".


What is this shadow mapping error called? I have been following a tutorial, shadow mapping works more or less as expected except sometimes the shadow projects completely weirdly on nearby walls. This isn't sampling outside the lightmap because I use CLAMP_TO_EDGE and if(min(ShadowCoord.x, ShadowCoord.y) > 0.0 && max(ShadowCoord.x, ShadowCoord.y) < 1.0) in the fragment shader. Does anyone know what the official term for this problem is so I can go research it some more? Thanks.

It's not entirely clear what the scene in your picture is supposed to look like, so I'm not sure I'm interpreting it correctly. But if I am, it seems like the problem is that you're projecting the shadow properly, but onto the wrong face of the gray wall. If you want to research it, you might have luck looking up winding order (sometimes called winding number), or back-face culling.

If I'm correct in what's shown above, I think you could solve it in your shader by only applying the shadows to the current fragment if the vector from the light source to the surface points away from the surface normal. In other words, take the dot product of those 2 vectors, and if it's negative apply shadows, otherwise, don't.


Subdivision scheme where the faces and edges have weights (not necessarily scalar weights) Subdivision schemes work by considering the vertices and their connectivity information to calculate averaging weights.
However, other than specifying which vertices are connected, and perhaps which weights to use (in the case of loop subdivision for example), edges don;t really play a role.
Same as faces. I am wondering if there is a subdivision scheme that attaches "weights" (and weights could be either scalars or matrices or similar vector fields) and has the faces and edges have a controllable effect on the convergence.
For example in the case of meshes, consider a cube. If the weights of the faces are 0, then the subdivision is just a regular subdivision. But if the weights of each face go towards infinity, then the limit of the convergence should be the original cube.
I am trying to find something like this, but I cannot find any associated literature with the topic.

What you are looking for is semi-sharp creases. You can find it in section 3 of this paper: https://graphics.pixar.com/library/Geri/paper.pdf
Basically, each edge is given a sharpness value $s$. This can either be a integer or a floating point value. This value signifies how many subdivision steps this edge will be subdivided using sharp-subdivision rules. When subdividing an edge with sharpness $s$, it subdivide into two edges with sharpness level $s-1$. Until $s$ is 0 and the edge will be subdivided as normal.For fractional levels the last subdivision with $s \in [0,1]$ the vertices will obtain a blended position between the subdivided $p$ and the sharp position $p_s$, i.e. $(1-s)p + s p_s$.
The sharp subdivision rules boil down to the following. Vertices at the end of an sharpness tagged edge will remain in the same position and vertices on the sharp edge are refined using the stencil [1, 6, 1]/8 applied to previous, current and next points on the tagged curve respectively. New points are inserted at midpoints of edges.
This scheme can be generalized to faces by propagating the sharpness value of a face to its edges.


How to count aspect ratio of an image when you have dimensions? What formula is for counting aspect ratio of the image as in this site:

Aspect ratio calculator (ARC) | NinjaUnits

For example:

 1000x1200 its aspect ratio 5:6
 1000x1333 its aspect ratio 3:4
 2592x3888 its aspect ratio 2:3

I want to determine this by handling an uploaded image so it should be a formula for counting this, but did not figure out this yet.

You need to take your width and height and make them into a fraction, then reduce it. To do that, you'll need to find the greatest common divisor of the numerator and denominator, and divide them both by that.

Now take the new fraction you got by dividing both the numerator and denominator by the greatest common divisor and do it again. Repeat this process until the greatest common divisor is 1. At that point you have your irreducible fraction and you can print it.

Another way to do it is to use Euclid's Algorithm.

Example:

Let's look at your top example, 1,000 x 1,200

1,000 / 1,200 = ?

We can see by examination that they both end in "00" so they're both divisible by 100. So start there:

1,000 ÷ 100 = 10
1,200 ÷ 100 = 12

Next, we see that 10 and 12 are both even, so they're both divisible by 2:

10 ÷ 2 = 5
12 ÷ 2 = 6

Finally, we see that 5 and 6 share no common factors besides 1. (This is because 5 is prime, so its only factors are 5 and 1. 6 has the factor 2 and 3, as well as 1 and 6.) So we have reached the end. The reduced fraction is 5 / 6.

One More Thing

You might also want to be able to detect cases of irrational ratios, such as the golden ratio, which is quite popular in visual arts. Such a ratio will not be possible to divide into small integers, so checking whether width / height is within some small distance of the golden ratio may be sufficient.


Can I ignore importance, adjoint equations, GRDFs for my path tracer? I need to write a photo-realistic renderer. I have been looking at ScratchAPixel site, asking a couple of questions here on CG, and going through the Advanced Global Illumination 2nd ed book. I've read about radiometry, probability, Monte Carlo, and a bit on Russian roulette. I'm aware of the rendering equation in its hemispherical and area formulations. I've written a SAH based kd-tree, so am ok for efficient ray casting.

I'm poised to start writing some code alongside reading chapter 5 of my book which is about path tracing algorithms. However, the last half of chapter 4 is taken up with talking about the following concepts:

The Importance Function
The Measurement Equation
Adjoint equations and linear transport operators
GRDF (Global Reflectance Distribution Function)

These concepts I've not seen appearing elsewhere in my research (if you can call reading a few web pages about GI rendering proper research?). My question is, do I really need to know this stuff to progress to writing my path tracer? I suspect the answer might depend on which type of path tracer I'm going to developer. To start, based upon what little I know, I think it'll be unidirectional.

No, you don't need to know this stuff to implement basic path tracer.

Basic unidirectional path tracer is quite simple to implement. Just trace bunch of paths of length X for each pixel with uniform distribution over the normal oriented hemisphere at the path's intersection points. Then weight the remaining path with the BRDF at each intersection point and multiply with luminance of light once the path hits a light.

You'll get quite a noisy (but unbiased!) image even for large number of paths and then you can start to look into methods to reduce noise, e.g. importance sampling & bidirectional path tracing. Just validate the more optimized path tracers towards earlier validated path tracers to avoid introducing accidental bias.


How is a projector handled in a Ray Tracing environment? I want to create a scene where a room includes a projector and a white screen.

How is a Ray Tracing environment functioning in such a setup? Is the light from the projector going through a slide and then the color reflecting on the white screen? If so, wouldn't any light ray bounce on that screen creating a mess?

Just treat it as a special case of a point light + shadow ray. Whenever a ray hits the screen (or any surface onto which you'd like to project the slide), fire another "projector" ray back towards the projector. If it is not obscured, then use the X & Y angles (relative to the coordinate system of the projector) to look up a texture map of the slide.


Is there a difference between window and screen in computer graphics? I've been reading some code which setups a ray tracer and I've realized I've a doubt regarding what's a screen and what's a window in this context. Here's the relevant code using Qt:

...

void CCanvas::paint()
{
  // check if we have something to draw on
  if (!image) return;

  // camera position
  Point3d camera (0.0, 0.0, 0.0);

  // screen resolution
  int w = image->width();
  int h = image->height();

  // horizontal opening angle
  double alpha = PI/2.0;

  // horizontal window dimension [-X,X]
  double X = tan(alpha/2.0);

  // pixel size
  double s = 2.0 * X / (double)w;

  // vertical window dimension [-Y,Y]
  double Y = s * h / 2.0;
  ...

I think I know what's the screen resolution: if we think of the screen as a matrix, w and h would be respectively the number of columns and rows in this "matrix".

After obtaining the resolution of the image, we define an opening angle $\alpha$. I interpreted this opening angle as the opening angle of the camera, i.e., it defines basically what's visible from the camera.

Then we compute X, which is half of the horizontal window (according to the comments in the code). (Note this X is not the same X representing the x axis in the picture below. Yes, whoever wrote this code had no common sense!)

Now, mathematically I understood these calculations, what I don't understand is the difference between $2X$ and $w$.

Is the "window" here the same thing as "screen"?

I've this picture here

I've added the $X=\tan \left( \frac{\alpha}{2}\right)$ to the picture according to what I think $X$ is from the code above.

If the window is the same thing as the screen, I wonder why do we need $w$. I mean, couldn't we just decide how many rows and columns we have in the visible area?

Again, it's not clear if the visible area of the camera is the same thing as the screen or window...

In this code, the "screen resolution" means the number of pixels in the output image (which happens to be a window in the windowing system), while the "window dimension" means the corresponding size in the 3D scene. Computing s tells you the ratio between one unit of 3D space in your scene and one pixel of output image. You need to do this in order to know how far apart in world-space to cast your primary rays.

These uses of "screen" and "window" aren't standard; they're specific to this code. No wonder you're confused.


Correct Specular Term of the Cook-Torrance / Torrance-Sparrow Model For a while I have been doing some research on the topic of Physically Based Rendering. One reflection model that is mentioned over and over is the Cook-Torrance / Torrance-Sparrow model. It seems like in each mention or explanation of this model a different form of the specular term is used. The versions I have found are:

$${\frac {FDG}{\pi ({\vec N}\cdot {\vec V})({\vec N}\cdot {\vec L})}}$$
$${\frac {FDG}{4 ({\vec N}\cdot {\vec V})({\vec N}\cdot {\vec L})}}$$
$${\frac {FDG}{({\vec N}\cdot {\vec V})({\vec N}\cdot {\vec L})}}$$

Which one is correct, and when? In Physically Based Rendering: From Theory to Implementation by Matt Pharr and Greg Humphreys, the second one is conclusively derived, but in their original paper Cook and Torrance use the first one without any detailed explanation.

I would trust Pharr and Humphreys on this. Equation 2 also agrees with the SIGGRAPH Physically Based Rendering course notes, as well as with equation 20 in the Walter et al paper that introduced the GGX distribution.

I've read somewhere that there is an error in the original Cook-Torrance paper that led them to miss the factor of 4 in the denominator, which was corrected in subsequent papers. I couldn't find a reference to this with a quick search though (if anyone knows one, please feel free to note it in the comments).

As for the factor of π, it might appear or not, depending on conventions. Sometimes it's factored into the normal distribution function D. For instance, if you look in the Walter et all GGX paper section 5.2 where they give the equations for several D functions, you can see they all have a π in the denominator. Note that this implies that the Lambertian BRDF should have a π in the denominator as well.

In real-time graphics, the π is often left out, in which case we can interpret it as having been factored into the light colors. Either way is fine, as long as you're consistent about either putting the π in or leaving it out of all the BRDFs you use.


Raytracer help - Background color writing over some colors? I'm implementing Peter Shirley's "Raytracer in a Weekend: The next week" and I'm having some trouble with rendering a scene with a light. I'm not sure if the issue is the background color or the light itself but the raytracer seems to be working fine in any scenes that have no lights.
The image rendered is supposed to be:

but it actually turns out as:

When I try to troubleshoot and change the background color to blue the image is:

Hopefully someone who has implemented this raytracer before or who has intuition about raytracing has an idea why this might be happening??

In Lambertion::scatter there is a comment about attenuating light based on distance.
For a raytracer this is not required as the distribution of rays themselves cause attenuation over distance.


How to deal with light leaks? I'm struggling with light leaking through meshes (especially thin ones) due to interpolation between probes located on the opposite sides of those.

First approach I took was mapping all static geometry to 1 lightmap, then generating light probes for each texel center that can be mapped to some triangle. Compute probe world position using barycentric coords of a texel on a corresponding static geometry triangle and then, in fragment shader, bilinearly interpolate between probes using fragment's UV coordinate. Unfortunately, I had to throw all of this away, because automatic UV mapping (all of the tools available in Blender, for example) are absolutely not suitable for this task and the manual UV crafting is just too tedious to be viable.

Then I saw a Guerrilla Games presentation regarding GI tech in the Killzone which leverages shadow maps (kind of) to store occlusion data for each probe. I thought maybe I could pull off something similar. What I did was ray trace occlusions for each probe and store occlusion distances for respective ray directions in a huge texture atlas, then, in a fragment shader, determine which probes are occluded for the current pixel by comparing distance from probe to pixel with occlusion distance for the nearest ray and set interpolation weights for occluded probes to 0. But, there are 2 serious problems to this approach, which I found only after implementing the algorithm.

1) Let's consider the case, when distance from the current pixel to the probe is compared to ray 3 (see attached illustration). For pixels located in the range denoted 'l' distance to probe will be greater than occlusion distance for ray 3 and the probe will be considered occluded, but it's not!

2) Let's imagine the first problem is solved. Now the problem of a thin mesh arises. For such mesh pixels on the opposite sides of it will have pretty much the same world location and distance tests will result in full interpolation of probes from both sides for both pixels. I guess a solution for this will require pixel's normal, but maybe it will come after the first issue is resolved.

I would greatly appreciate any help in solving there problems. Or maybe someone could point towards more viable algorithm instead?

I've also spent a good amount of time thinking about a solution for this problem. It seems like one those fundamental mismatches of taking a sparser, lower frequency structure (probe mesh, even if augmented with depth map) and lighting surfaces per pixel which themselves are occluders.

If you look at the SIGGRAPH'17 Precomputed lighting in CoD:IW (http://advances.realtimerendering.com/s2017/index.html) you'll see they partitioned the precomputed lighting into light maps for large surfaces (which have lots of occlusion) and using a probe mesh for small static objects as well as characters. They take a similar approach of storing an occlusion depth map per tetrahedra face, however because they're not lighting the occluders with the probe mesh, the discontinuities introduced are likely not noticeable.

Trying to light the occluders with the same map is extremely tricky because the probe mesh faces may not align well with the occluders. I.e. literally "corner" case where 3 planes meet doesn't encode well into a depth map. I have feeling they still get light leaks, but their density is probably fairly high and their evaluation points are distant enough from the pathologically bad cases.

Sometimes it might be worth asking level artists to not make single sided meshes or thin walls XD, at least for the time being.

Best of luck, if you find a robust solution to this, you should submit a paper to a conference and be graphics industry famous :D


Confusion around Lambert's Cosine Law in Ray Tracing in One Weekend Peter Shirely's Ray Tracing in One Weekend presents the following function to calculate the color of a lambertian material (See chapter's 7 and 8):

bool lambertian::scatter(ray ray_in, hit_record rec, vec3& attenuation, ray &scattered) {
  vec3 target = rec.p + rec.normal + random_in_unit_sphere();
  scattered = ray(rec.p, target - rec.p);
  attenuation = this->albedo;
  return true;
}

I've read through other material that mentions Lambert's Cosine Law when discussing diffuse materials. Why is the cos term omitted from Peter Shirely's text? Is it canceled out somehow? My expectation is that the attenuation term should look something like:

  attenuation = this->albedo * dot(rec.normal, ray_in.direction);

It is cancelled out by the probability density function in the estimator. The pdf in their case is exactly: $\frac{\cos\theta}{\pi}$, which is in the denominator: att = albedo * cos_theta / pdf = albedo * pi. They have absorbed pi in the albedo. Note that there's an update on github to the code, since random_in_unit_sphere() actually generates a $\cos^3\theta$ distribution, they fixed it to random_on_unit_sphere() which generates a $\cos\theta$ distribution. So technically you are working with a $\cos^2\theta$ brdf here and not constant.


intersection between line segments - narrowed precondition I believe that the issue may already have been discussed here. I want to find if two line segments does intersect and if they do then find and store and intesection points. Now it is already confirmed that one of the line segments will always be parallel to x-axis or y-axis.
So in that case how should we reformulate the basic algorithm that checks the line segments intersection?

The arbitrary line can be expressed as y = a*x+b (assuming it's not parallel to the y axis).

If the other line is parallel to the y axis then you can simply fill in the x coordinate of any point on the line into the formula of the first line. Then you can check whether they intersect by ensuring the result lies between the endpoints of the second line.

For the second line parallel to the x axis you just flip the x and y coordinates.


Probability density for explicit light sampling When implementing explicit light sampling, I am separating the monte-carlo approximation to two parts
$$\frac{1}{N+M}(\sum^N \text{direct} + \sum^M \text{indirect})$$
$$\text{direct} = \frac{\text{bsdf} \times C \times cos(\theta)}{p}$$
I am confused as how to obtain $p$ for direct lightning.  My light source is an arbitrary triangularized mesh and I am sampling random points from it's surface.
A solution was presented here: Path weight for direct light sampling
$$\frac{A}{r^2} (N_\text{light} \cdot -L)$$
Where does this come from?  I'm asking for a reference or a derivation. Thanks!

It comes from the relation between the area formulation and the solid angle formulation of the rendering equation: $d\omega = \frac{\cos\theta_L}{r^2}dA$:
https://arxiv.org/abs/1205.4447


How to determine if SceneKit can produce sufficiently 3D graphics like Minecraft? Inspired by Minecraft and LEGOs, we created an app to promote creativity on smartphones. Many people like being creative but find a blank canvas daunting. "Recipes" inside the app let users create like the most talented individuals -- similar to how a cooking recipe lets anyone cook like Thomas Keller.

The problem so far is users say the app doesn't look "3D" enough.

Here's the relevant code that creates the scene and creates blocks: https://gist.github.com/panabee/dca88a591f94a3904023dacabd020ad9.

What they mean is they expect the app to look more like apps like this:

With encouragement from one of the admins, these screenshots from our app illustrate the disparity in "3Dness":

We're using SceneKit on iOS, but are having trouble identifying what is needed to make the app look more like the images above. We played with lighting and textures, but are new to 3D graphics as well as SceneKit and are unsure what to do.

We have tried to find consultants on StackOverflow, LinkedIn, Github, but this expertise appears not so common.

A few questions:

1) For iOS, is SceneKit capable of producing the images above, or do we need to use something like Metal?

2) If SceneKit is capable, what are the key concepts to learn?

3) (Happy to strike this question if it's considered off-topic) Which websites/forums could we find people to help provide advice if we get stuck?

What is missing?

Now that we can see images from your app, the difference that stands out most to me is the lack of shadows and ambient occlusion. This gives a very flat look as the colour of a surface does not vary according to its surroundings. This makes a big difference to how much an image appears to have depth.

Another effect which may be less important is that the surfaces are crystal clear all the way up to the horizon. Adding a mist/fog effect so that objects become gradually more obscured the further they are from the camera helps to give the impression of distance.

Both these effects can be seen most obviously in the first example image in the question. The nearest building shows ambient occlusion on top of the stepped red roof, and also the grass is in partial shade near to the buildings and animals. The trees in the distance look more obviously far away due to the distance fog making them only partially visible.

Can SceneKit provide these?

I haven't used SceneKit myself, but a search online reveals that both ambient occlusion and distance fog effects are both possible using SceneKit.

Another effect not used in the example images, but that you might like to look into, is depth of field. The example images all have everything in focus at the same time, whereas photographs often have a focal distance at which everything is in clear focus, with anything nearer or farther being out of focus. It seems SceneKit can also handle this. Since your competitors don't appear to be using this, you could either view this as unimportant, or as a feature that could give you an advantage.

I'd expect narrow depth of field to make a building app awkward, as parts of the object you are building would be out of focus. You may be interested in a wide depth of field that only creates a subtle blur for objects too distant to be part of what is currently being built, if that suits your purposes.

As you guessed, asking for off site resources is off topic. However, you can ask further questions about computer graphics programming here on this site, if you run into any problems implementing these effects.


How to determine the object-to-world matrix I want to ray trace a triangular mesh that I load from a file. The vertices' coordinates are expressed in the reference frame associated with the object  (i.e the mesh) and I don't know the object-to-world matrix. Is there a way to calculate this matrix given these coordinates? If no, how can I ensure that the mesh's triangles are transformed to the world's reference frame?

No! If you dont have this info and must have a extra frame, just assume the reference frame is a identity matrix*. The point of the extra reference frame is just to be able to move the object but if you have no info about how the object should relate to other objects it does not really matter operator can move it where they want. Everything is relative, if theres no relation then no problems.

* choosing identity is like choosing a arbitrary constant to be 1, its numerically easy to work with.


Why is work-efficiency desired in GPU programming? I've been reading the following article on how to do a parallel scan in CUDA:

https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch39.html

In the article, there is an emphasis on making the scan "work efficient". In other words, a GPU algorithm should perform no more additions than a CPU algorithm, O(n). The authors present two algorithms, one "naive" that does O(nlogn) additions, and one that they consider "work efficient". However, the work efficient algorithm does twice as many loop iterations.

From my understanding, GPUs are simply giant SIMD processors and should operate in lock-step. Doing twice as many loops in the "work efficient" algorithm seems to imply that many threads will be idle and decrease performance in the long run. What am I  missing?

First of all, re: "GPUs are simply giant SIMD processors and should operate in lock-step", it's a bit more complicated than that. The entire GPU does not run in lockstep. Shader threads are organized into groups of 32 called "warps" (on NVIDIA; on AMD they're groups of 64 called "wavefronts", but same concept).  Within a warp, all the threads do run in lockstep as a SIMD array. However, different warps are not in lockstep with each other. In addition, some warps may be actively running while others may be suspended, much like CPU threads.  Warps can be suspended either because they're waiting for something (such as memory transactions to return or barriers to clear), or because there isn't a slot available for them (since the GPU can only actively run a certain number of warps at a time).

Now, back to your question. I can see two ways that the "work-efficient" algorithm from that paper looks like it would be more efficient than the "naive" algorithm.

The work-efficient version requires half as many threads to begin with. In the naive algorithm, they have one thread per array element; but in the work-efficient version, each thread operates on two adjacent elements of the array and so they need only half as many threads as array elements. Fewer threads means fewer warps, and so a larger fraction of the warps can be actively running.
Although the work-efficient version requires more steps, this is offset by the fact that the number of active threads decreases faster, and the total number of active threads over all the iterations is considerably smaller. If a warp has no active threads during an iteration, that warp will just skip to the following barrier and get suspended, allowing other warps to run. So, having fewer active warps can often pay off in execution time. (Implicit in this is that GPU code needs to be designed in such a way that active threads are packed together into as few warps as possible—you don't want them to be sparsely scattered, as even one active thread will force the whole warp to stay active.)

Consider the number of active threads in the naive algorithm.  Looking at Figure 2 in the article, you can see that all the threads are active except for the first 2k on the kth iteration.  So with N threads, the number of active threads goes like N − 2k. For example, with N = 1024, the number of active threads per iteration is:

1023, 1022, 1020, 1016, 1008, 992, 960, 896, 768, 512

If I convert this to number of active warps (by dividing by 32 and rounding up), I get:

32, 32, 32, 32, 32, 31, 30, 28, 24, 16

for a sum of 289.  On the other hand, the work-efficient algorithm starts with half as many threads, then it halves the number of active ones on each iteration until it gets down to 1, then starts doubling until it gets back up to half the array size again:

 512, 256, 128, 64, 32, 16, 8, 4, 2, 1, 2, 4, 8, 16, 32, 64, 128, 256, 512

Converting this to active warps:

16, 8, 4, 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 4, 8, 16

The sum is 71, which is only a quarter as many. So you can see that over the course of the entire operation, the number of active warps is much smaller with the work-efficient algorithm. (In fact, for a lengthy run in the middle there are only a handful of active warps, which means most of the chip is not occupied. If there are additional compute tasks running, e.g. from other CUDA streams, they could expand to fill that unoccupied space.)

All that being said, it's unfortunate that the GPU Gems article does not clearly explain any of this, instead focusing on big-O "number of additions" analysis that, while not entirely irrelevant, misses a lot of the details about why this algorithm is faster.


What is a stencil buffer? Wikipedia states that a stencil buffer is some arbitrary buffer a shader can use.

However, it hints that it's used for clipping, or otherwise "tightly binding" the depth and pixel buffers, slightly contradicting itself.

What does the stencil buffer really do, and how is it practically used in modern applications?

The stencil buffer definition by Wikipedia is indeed not great, it focuses too much on the details of modern implementations (OpenGL). I find the disambiguated version easier to understand:

  A stencil is a template used to draw or paint identical letters, symbols, shapes, or patterns every time it is used. The design produced by such a template is also called a stencil.

That's what stencil meant before Computer Graphics. If you type stencil on Google Images, this is one of the first results:

As you can see, it is simply a mask or pattern that can be used to "paint" the negative of the pattern onto something.

The stencil buffer works in the exact same way. One can fill the stencil buffer with a selected pattern by doing a stencil render pass, then set the appropriate stencil function which will define how the pattern is to be interpreted on subsequent drawings, then render the final scene. Pixels that fall into the rejected areas of the stencil mask, according to the compare function, are not drawn.

When it comes to implementing the stencil buffer, sometimes it is indeed coupled with the depth buffer. Most graphics hardware uses a 1 byte (8 bits) stencil buffer, which is enough for most applications. Depth buffers are usually implemented using 3 bytes (24 bits), which again is normally enough for most kinds of 3D rendering. So it is only logical to pack the 8 bits of the stencil buffer with the other 24 of the depth buffer, making it possible to store each depth + stencil pixel into a 32 bit integer. That's what Wikipedia meant by:

  The depth buffer and stencil buffer often share the same area in the RAM of the graphics hardware.

One application in which the stencil buffer used to be king was for shadow rendering, in a technique called shadow volumes, or sometimes also appropriately called stencil shadows. This was a very clever use of the buffer, but nowadays most of the rendering field seems to have shifted towards depth-based shadow maps.


How to handle different data types (uint8, uint16, float) in Volume Datasets? So I've written a personal volume renderer which currently visualizes datasets (CT-Scans mostly) that have a .raw extension and the values are in the range 0-255 or uint8.

Now I searched the net and found some datasets that had uint16 (ranging from 0-4095) and float which was arbitrary but went from like 0-20 or something. I think the uint16 dataset corresponds to values in Hounsfield scale but I'm unsure of how do I map these to 0-255.

I haven't implemented any fancy transfer functions currently. For the uint8 datasets I just read the isovalues from the dataset and use that value for all the channels (r,g,b) as it is giving me a grayscale image.

EDIT:- To clarify further, I have implemented a gpu based ray marching algorithm taking help from this site here, I'm taking the datasets from here, As mentioned in the comments I don't really know how you should interpret these values, but a quick read shows that normally the CT data is based on Hounsfield units which represents density afaik.

Ok after searching a whole lot on the internet I found a slide which answered most of my questions and doubts. I'm dumping it here in-case anybody else is curious what these datasets usually contain and what should be (in my opinion) the correct way to handle mappings.

So apparently the CT scanner always records the densities of the substances in Hounsfield Scale. The Hounsfield Scale itself is a little interesting. For body organs and stuff, the scale has the range from -1024 to 3000. This was the scale I talked about earlier as when normalized the values usually lie between 0-4096. However metals usually go past this 3k boundary and some metals like stainless steel and gold having HU number as high as 13k and 30k respectively. This is usually called the "extended" Hounsfield Scale.
Some volumetric datasets contain data for computer simulations mostly fluids etc. For these type of datasets either density or flow velocity is captured. Precision is important for these datasets hence float is used.

Hence the uint8 datasets are probably grey-scale data converted from the original HU units. uint16 datasets contain the original Hounsfield data while float may contain density or velocity.

Now for the mapping part, as others stated one way might be to just divide by the max value of the data type or the max value of the data itself, to get the uint16 or float data in 0-1 range then multiply by 255 to get a grey-scale image. However doing so may lower the contrast and hide minute details. For example, dividing a uint16 dataset that contained only HU units in range 300-2000 with the $2^{16}$ will map the HU units to only a small range of grey values. Dividing by 2000 in this case is surely better. However after reading how CT-scanners work, it's better to let the user set the minimum and maximum range of Hounsfield units they wish to observe. This is usually called the window width. And then map this window width to the whole 0-255 range of grey levels as YardenJ2R mentioned in his second formula. This is more in line with how radiologists visualise the data.

For fancy transfer functions which I haven't implemented yet, we might have to convert the Hounsfield units to opacity or grey scale as mentioned above then map that opacity to color or there might be a way to directly map it to colors using a LUT as defined by the user's control point settings.


Choosing Reflection or Refraction in Path Tracing I am trying to implement refraction and transmission in my path tracer and I'm a bit unsure on how to implement it. First, some background:

When light hits a surface, a portion of it will reflect, and a portion will be refracted:

How much light reflects vs. refracts is given by the Fresnel Equations

In a recursive ray tracer, the simple implementation would be to shoot a ray for reflection and a ray for refraction, then do a weighted sum using the Fresnel.
$$\begin{align*}
R &= Fresnel()\\
T &= 1 - R\\
L_{\text{o}} &= R \cdot L_{\text{i,reflection}} + T \cdot L_{\text{i,refraction}}
\end{align*}$$

However, in path tracing, we only choose one path. This is my question:

How do I choose whether to reflect or refract in a non-biased way

My first guess would be to randomly choose based on the Fresnel. Aka:

float p = randf();
float fresnel = Fresnel();
if (p <= fresnel) {
    // Reflect
} else {
    // Refract
}

Would this be correct? Or do I need to have some kind of correction factor? Since I'm not taking both paths.

TL;DR

Yes, you can do it like that, you just have to divide the result by the probability of choosing the direction.

Full Answer

The topic of sampling in path tracers allowing materials with both reflection and refraction is actually a little bit more complex.

Let's start with some background first. If you allow BSDFs - not just BRDFs - in your path tracer, you have to integrate over the whole sphere instead of just the positive hemisphere. Monte Carlo samples can be generated by various strategies: for the direct illumination you can use BSDF and light sampling, for the indirect illumination the only meaningful strategy usually is the BSDF sampling. The sampling strategies themselves usually contain the decision about which hemisphere to sample (e.g. whether reflection or refraction is computed).

In the simplest version, the light sampling usually doesn't take care much about reflection or refraction. It samples the light sources or the environment map (if present) with respect to the light properties. You can improve sampling of environment maps by picking just the hemisphere in which the material has non-zero contribution, but the rest of the material properties is usually ignored. Note that for and ideally smooth Fresnel material the light sampling doesn't work.

For BSDF sampling, the situation is much more interesting. The case you described deals with an ideal Fresnel surface, where there are only two contributing directions (since Fresnel BSDF is in fact just a sum of two delta functions). You can easily split the integral into a sum of two parts - one reflection and one for refraction. Since, as you mentioned, we don’t want to go in both directions in a path tracer, we have to pick one. This means that we want to estimate the sum of numbers by picking just one of them. This can be done by discrete Monte Carlo estimation: pick one of the addends randomly and divide it by the probability of it being picked. In an ideal case you want to have the sampling probability proportional the the addends, but since we don't know their values (we wouldn't have to estimate the sum if we knew them), we just estimate them by neglecting some of the factors. In this case, we ignore the incoming light amount and use just the Fresnel reflectance/transmittance as our estimates.

The BSDF sampling routine for the case of smooth Fresnel surface is, therefore, to pick one of the directions randomly with probability proportional to the the Fresnel reflectance and, at some point, divide the result for that direction by probability of picking the direction. The estimator will look like:

$$
\frac
   {L_{i}\left(\omega_{i}\right)F\left(\theta_{i}\right)}
   {P\left(\omega_{i}\right)} =
\frac
   {L_{i}\left(\omega_{i}\right)F\left(\theta_{i}\right)}
   {F\left(\theta_{i}\right)} =
L_{i}\left(\omega_{i}\right)
$$

Where $\omega_{i}=\left( \phi_{i}, \theta_{i} \right)$ is the chosen incident light direction, $L_{i}\left(\omega_{i}\right)$ is the amount of incident radiance, $F\left(\theta_{i}\right)$ is either the Fresnel reflectance for the reflection case or 1 - Fresnel reflectance for the refraction case, $P\left(\omega_{i}\right)$ is the discrete probability of picking the direction and is equal to $F\left(\theta_{i}\right)$.

In case of more sophisticated BSDF models like those based on microfacet theory, the sampling is slightly more complex, but the idea of splitting the whole integral into a finite sum of sub-integrals and using discrete Monte Carlo afterwards can usually be applied too.


What does GPU serialized mean? With GPU isolation : multiple processes use the single GPU will be serialized, cause OOM easily
link
https://www.slideshare.net/mobile/Hadoop_Summit/deep-learning-on-yarn-running-distributed-tensorflow-etc-on-hadoop-cluster-v3
slide number 17

It's saying that if you're using a single GPU, the multiple processes attempting to use the GPU will be queued up and run one at a time rather than some or all being run in parallel. If you have multiple GPUs some of the processes can be run at the same time.


Silhouette below 3D model There are some 3D applications which can cast shadow or silhouette below 3D models. They render pretty fast and smooth. I wonder what kind of technology is the standard procedure to get 3D model shadow/silhouette.
For example is there any C++ library like libigl or CGAL to get shadow/silhouette pretty fast? Or maybe GLSL shading is used? Any hint would be appreciated on the standard technology stack.

There is a trick to shadows like this, but only this exact usecase. While you can certainly draw a true shadow using many techniques. The trick is you can just scale the object flat onto the floor plane.
So all you need to do is draw the object 2 times one time with a matrix that is scaled to zero on the floor axis. You can also use this technique to draw shadows that arent entirely top down by adding skew to counter the now rotated shadow.
This is super easy to implement. No extra cameras. But works only on one plane. This is a special case of shadow volumes without the need to draw the volumes.


Converting from RGB Color Picker to Surface Reflectance I am trying to derive the material reflectance values from a color picker tool(say MS Paint).
My understanding is to first gamma decode(raise to the power of 2.2) the RGB values from the color picker, so to bring it back to linear space, and then use it in Radiance mat definition.
However in the “Rendering with Radiance” book by Greg Ward section 5.1.4 "Estimating Surface Color"
page 294 and page 295, the rgb values from the color picker are raised to the power of inverse of gamma (1/2.2) and then used as in the material definition.
Here is a cropped snapshot of the formula from the above links to the scan copied of the two pages.

My question is, should the sRGB values from the color picker not be raised to the power of 2.2(decoding) rather than 1/2.2(encoding)?
For example for BS4800 “18-B-25 Dark admiralty grey / Merlin” colour sample, this webpage shows sRGB=104,112,116 and LRV value as approx. 16%. This matches closely to my understanding of (112/256) ^ 2.2 = 0.16 (just for the green component).

Yes, I think you're right. The values from a typical color picker would be in a gamma encoding or sRGB encoding and would need to be converted to linear to use as reflectance. This is probably just a mistake in the book.
(BTW, in case you're not aware, sRGB isn't actually gamma 2.2 but a more complicated transfer function.)