Why are tessellation shaders disliked? I often hear that tessellation shaders aren't great for performance reasons. Something along the lines that if you try to use a tessellation shader, you'll find that the data you get out isn't sufficient and you'll have to re-do calculations anyways. Or it might have been that this path was less efficient than going another route and wasn't a good use of bandwidth?

Can anyone explain the details of why tessellation shaders are not advised, and if there are any exception cases where tessellation is a good thing?

An answer with details gotten elsewhere:

It's bad on mobile: ARM doesn't have a tessellation unit so is emulated in software. Tiled based architectures will write vertex data to main memory before pixel shading which is a bandwidth issue. qualcom switches to direct rendering mode if you enable tessellation which isn't cheap.
It's bad on AMD / Console
There is complexity in setting the tessellation factors to give good results
it sounds like it's difficult not to get too much resolution in some ways (aliasing), and not enough in others (low quality results). You essentially need to know screen space derivatives of vertices to calculate good tessellation factors, and this is not easy to get even decent approximations of.
A bummer about tessellation shaders is that they basically work by sampling discrete sample points (at vertices) so for a better result, you need a denser mesh to start with, which sort of defeats the purpose of dynamic tessellation.  Can be addressed a bit with mipmapping type strategies.
Tessellation is still used by major games in specific cases though, and terrain does seem to be one usage case


Variance of estimator (Monte Carlo Integration) So I was reading this paper by Lafortune, "Mathematical Models and Monte Carlo algorithms" and in it he writes.

We have a function or integrand I we want to estimate given as,

$I = \int f(x) dx$

We then have a primary estimator for this as,

$\hat{I_p} = f(\xi)$

where $\xi$ is a random number generated in the interval of the integrand $I$.

The secondary estimator is defined as,

$\hat{I_{sec}} = \frac{1}{N} \sum f(\xi_i)$

Then there is an explanation that goes to show how the expected value of the secondary estimator is equal to the function/integrand I. i.e,

$E(\hat{I_{sec}}) = I$

All fair. The problem comes when he tries to find the variance of this secondary estimator as follows. Im posting a screenshot of the paper here since it's getting a little complex.

I don't understand how did step 2 follow from step 1 of Equation 3.5 in the variance calculation. Note that he has assumed $PDF = 1$ for now

I was trying to calculate the variance using a more standard approach and ignoring the multiple integrals. We know the variance of an estimator $\hat{\theta}$ is given as

$Var(\hat{\theta}) = E[(\hat{\theta} - E(\hat{\theta}))^2]$

If we apply this to above we get

$Var(\hat{I_{sec}}) =  E[(\hat{I_{sec}} - E(\hat{I_{sec}}))^2] $

$\hspace{20mm}      = E[(\hat{I_{sec}} - I)^2]$

$\hspace{20mm}      = E[(\hat{I_{sec}})^2 - 2* I *(\hat{I_{sec}}) + I^2 ] $

$\hspace{20mm}      = E[(\hat{I_{sec}})^2] - 2* I *E[\hat{I_{sec}}] + I^2 $

$\hspace{20mm}      = E[(\hat{I_{sec}})^2] - 2* I^2 + I^2$

$\hspace{20mm}      = E[(\hat{I_{sec}})^2] - I^2 $

$\hspace{20mm}      = E[(\frac{1}{N} \sum\limits_{i=1}^{N} f(\xi_i))^2] - I^2$

Now I am stuck here, I could do this,

$\hspace{20mm}      = E[\frac{1}{N^2} \sum\limits_{i=1}^{N} f^2(\xi_i) + \frac{1}{N^2}\sum\limits_{i\neq j}^{N} f(\xi_i) f(\xi_j) ] - I^2  \hspace{10mm} \because
[\sum\limits_{i=1}^{N} f(x_i) ]^2 =  \sum\limits_{i=1}^{N} f^2(x_i) + \sum\limits_{i\neq j}^{N} f(\xi_i) f(\xi_j)$

$\hspace{20mm}      = \frac{1}{N^2} \sum\limits_{i=1}^{N} E[f^2(\xi_i)] + \frac{1}{N^2}\sum\limits_{i\neq j}^{N} E[f(\xi_i) f(\xi_j) ] - I^2$

$\hspace{20mm}      = \frac{1}{N^2} \sum\limits_{i=1}^{N} \int f^2(x)dx + \frac{1}{N^2}\sum\limits_{i\neq j}^{N} E[f(\xi_i) f(\xi_j) ] - I^2$

$\hspace{20mm} \because E[f(X)] = \int f(X) p(X) dx$

Note $p(X) = 1$,

$\hspace{20mm}      = \frac{1}{N} \int f^2(x)dx + \frac{1}{N^2}\sum\limits_{i\neq j}^{N} E[f(\xi_i) f(\xi_j) ] - I^2$

Don't know what to do next from here, any tips? Or did I do something wrong?

Quoting from "Advanced global illumination" book:

The Monte Carlo estimator is defined as:

And its variance according to the above definition(continuous random variable) would be:

Edit:

An example for how the variance of an estimator G is calculated:

(note how the summation is written as N times the integrand and is cancelled out later.)


What is the state of art in geometric LOD in games? How do modern games do geometry level-of-detail for object meshes like characters, terrain, and foliage? There are two parts to my question:

What does the asset pipeline look like? Do artists make a high-poly model which is later decimated? If so, what decimation algorithms are most popular? Are LOD meshes sometimes done by hand?
How do engines transition between different object LODs at run time? Are there any smooth or progressive transitions?

The answer might be "different studios use different techniques." If so, please identify some of the most common practices. It would also be great if you could point me to whitepapers/slides that cover specific examples.

For the geometry LOD most games simply switch between a number of predefined LOD meshes. For example "Infamous: Second Son" uses 3 LOD meshes (Adrian Bentley - "inFAMOUS: Second Son engine postmortem", GDC 2014) and "Killzone: Shadow Fall" uses 7 LOD meshes per character (Michal Valient - "Killzone: Shadow fall demo postmortem", Devstation2013). Most of them are generated, but more important ones (like main character) can be hand made. Meshes are often generated using a popular Simplygon middleware, but sometimes they are simply generated by graphics artists in their favorite 3D package.

Games with a large draw distance additionally use imposters for foliage, trees and high buildings (Adrian Bentley - "inFAMOUS: Second Son engine postmortem", GDC 2014). They also employ hierarchical LODs, which replace a set of objects with one. For example in "Just Cause 2" trees are first rendered individually as normal LOD meshes, then individually as imposters and finally as a single merged forest mesh (Emil Persson, Joel de Vahl - "Populating a Massive Game World", Siggraph2013) and in "Sunset Overdrive" distant parts of the world are replaced by single automatically offline generated mesh (Elan Ruskin - "Streaming Sunset Overdrive's Open World", GDC2015).

Another component of a LOD system is simplification of materials and shaders. For example "Killzone: Shadow Fall" disables tangent space and normal mapping for the distant LODs (Michal Valient - "Killzone: Shadow fall demo postmortem", Devstation2013). This usually is implemented by disabling globally a set of shader features per LOD, but for engines with shader graphs, where artists can create custom shaders, this needs to be done manually.

For the LOD transitions some games simply switch meshes and some use dithering for smooth LOD transitions - at the LOD switch two meshes are rendered: first gradually fades out and second fades in (Simon schreibt Blog - "Assassins Creed 3 – LoD Blending"). Classic CPU progressive mesh techniques aren't used as they require a costly mesh update and upload to GPU. Hardware Tessellation is used in a few titles, but only for the refinement of a most detailed LOD, as it's slow and in general case it can't replace predefined geometry LODs.

Terrain LODs are handled separately in order to exploit it's specific properties. Terrain geometry LOD is usually implemented using clipmaps (Marcin Gollent - "Landscape creation and rendering in REDengine 3"). Terrain material LODs either are handled similarly to mesh LODs or using some kind of a virtual texture Ka Chen - "Adaptive Virtual Texture Rendering In Far Cry 4.

Finally if you are interested to see real game LOD pipelines then just browse through documentation of any of the modern games engines: Unreal Engine 4 - "Creating and Using LODs", CryEgnine - Static LOD and Unity - LOD.


How is Depth of Field Implemented? I know that depth of field involves blurring.

There's some great information about how to do a Gaussian blur in the question How is Gaussian Blur Implemented?

But, other than that, how is depth of field implemented?

What are the rules about how you blur each pixel, and how do you handle the case of when there is a pixel that wants to blur a lot next to a pixel that doesn't want to blur as much?

I've also heard of the "circle of confusion" but have no idea what that is.

Can anyone explain DOF in a plain, easy to understand way?

The depth of field is a characteristic of a camera lens setting, although the name "Depth of Field" is commonly used to describe the effect caused by such characteristic.

Camera lenses can only perfectly focus on one single point, but there is a distance for which the image will still look reasonably sharp. Such distance is what actually the Depth Of Field is. This distance is variable depending on various factors, but let's say for the sake of brevity that the easiest way to adjust it is by changing your camera aperture.

As for the circle of confusion it is helpful to look a pic from wikipedia:

In all of the three cases a single point is "projected" onto our image plane, but as you can see that is at different distances from the lens.

Basically because, as I said, not everything is perfectly in focus (that is the focal point is on the image plane), a point can be projected to our image plane not to a single point but to an area. This, as it is clear from the pic, is because the focal point is either before or after the image plane. This "circle" that is created on the image plane is what is called the circle of confusion.

Now from this should be clear that varying the depth of field you vary the CoC size, hence it can be used as a measure of the DoF itself and an intuitive one when thinking in terms of blurring.

As per how it is implemented in rendering there are really lots of methods out there. Usually in games is done as a post process using the depth information to split your scene in different "planes"; for example a focus plane that does not need to be blurred, a near plane and a far/background plane that need to be blurred. Once you blurred your planes you can composite them back together to get your final image.
How much you blur the various planes it is dependent on the effect you want to achieve. Advanced system usually implement this and other "lenses" effect using appropriate parameters coming from the photography world like the aperture.

Note that there are many ways to implement the effect, the one I described is just one way that also has variations (for example performing the whole thing at lower res). The limit is dependent on what your target is and how much you budgeted for this effect. You can go from adopting the "3 planes" method at extremely low res to computing the circle of confusion for each pixel and applying the consequent ad-hoc blur to that pixel.

EDIT: The comment from @NathanReed needs to be more in evidence as part of the answer:

  Matt Pettineo's blog posts How To Fake Bokeh and Bokeh II: The Sequel are great introductions to how to practically implement post-process DoF and address the typical artifacts you get from it.


Feeding vertex data to a compute shader I'm writing a CAD program in vulkan. In my current design, I have line paths represented as large vertex buffers for rendering. I would like to implement an algorithm for extruding the path outwards, based off of cross products. Ideally I would like to implement this algorithm within a compute shader, using the vertex buffer in place of a storage buffer. The algorithm would work like this: for each vertex in original path, calculate cross product based off of the two surrounding vertices in the buffer, then normalize and multiply by a scalar.

I'm fairly certain that this could be done by copying the vertex buffer to a storage buffer temporarily, but I doubt that's optimal, unless it's not actually physically copied. I don't see why I should move large chunks of data from VRAM to VRAM like that. It would probably, be faster to implement this algorithm locally on the CPU in that case. I don't have much experience using compute shaders; any input is appreciated.

You will need a new chunk of memory for the new surface anyway, so why not read from the original and store directly in the result.

Having precomputed normals will reduce a lot of the computation requirements though (especially needing to find&read each neighbour), the user action of the extrusion will probably be interactive so you will have multiple frames with different extrusion amounts but the normals of the original surface won't change. Having the normals be around will likely be helpful for other operations as well. You could make the temporary transformed geometry just a vertex shader trick as well and only commit the result once needed.

Computing normals is much simpler by going face per face and then summing the the non-normalized normals per vertex and normalizing after.


What program can bake transparent light maps? So I have this racing game I'm trying to modify that uses lightmaps that fade to transparency for light sources. An example from the game:

I've been trying to produce a similar effect but with no luck. I've tried blender but that doesn't support transparency off of light sources, if anyone has any program ideas I'd greatly appreciate it!

Not sure what sort of lightmaps are produced by blender. Perhaps if you can get it to render a non-transparent lightmap (for example one that has all the colours you want but on a black background), you could then use a program to remove all of the black. I believe GIMP and Photoshop both have a colour to alpha tool. Be sure to pick a background colour that will not be the same colour as any of the lights.


Solving a set of Geometry Constraints - What's that Method called? I am looking for the correct term for a type of problem solver. Since its hard to google for something without knowing its name, I hope that the good people of this stackexchange may be able to help me.

When defining shapes in CAD systems based on constructive solid geometry, you start out with a (usually 2D) sketch, from which you extrude, rotate, or otherwise generate a 3D shape. You make this sketch by defining a set of curves (lines, arcs, bezier, etc.) and then defining a set of constraints. This is co-linear to that, this line is parallel to that line, the distance here is so and so.

If you define a valid combination of constraints, the CAD system can almost always find a configuration of the curves matching those constraints. What is the part of the CAD system called, that finds those solutions? Is there a good open-source implementation that I could use?

I tried search terms like "geometry solver" or "geometric solver", but I couldn't find anything relevant.

Its called a geometric constraints solver (a good primer on subject). You can find a open source solver as part of Open Cascade but its a bit convoluted to get going. A simpler solution for just solving, but also 3D solver capable, is geosolver.

Making your own (algebraic solution being easiest to write) is also not that hard, just a bit of work to make it automatically write the equations (and be efficient at solving). Take the following drawing as a example (made and solved with Mathematica):

You can simply conclude that its a sum of vectors that has to loop back at the origin thus:

$$
7\begin{pmatrix}\sin(15^\circ)\\ \cos(15^\circ)\end{pmatrix}+5\begin{pmatrix}\sin(120^\circ)\\ \cos(120^\circ)\end{pmatrix}+2\begin{pmatrix}\sin(195^\circ)\\ \cos(195^\circ)\end{pmatrix}+4\begin{pmatrix}\sin(\delta)\\ \cos(\delta)\end{pmatrix}+x\begin{pmatrix}\sin(-90^\circ)\\ \cos(-90^\circ)\end{pmatrix}=0
$$

And solve the 2 unknowns from 2 equations, either algebraically or by using a gradient descent solver of some kind.


What does "Bidirectional" in BRDF mean? So Bidirectional essentially means "two directions", right? But what are these two direction (left-right? up-down? north-south?) we are talking about in regards to bidirectional reflectance distribution function (BRDF), bidirectional scattering distribution function (BSDF), or bidirectional transmittance distribution function (BTDF) in computer graphics?

In general the two directions in BxDF are incoming $\omega_i$ and outgoing $\omega_o$ radiance directions, often defined in spherical coordinates $[\theta, \phi]$ or as a 3D unit vector in Cartesian coordinates $[x, y, z]$. The BxDF $f(\omega_i, \omega_o)$ defines how much of the incident radiance $L_i$ from direction $\omega_i$ hitting the surface is scattered towards direction $\omega_o$. What $\omega_i$ and $\omega_o$ exactly represent depends on the context.

Often $\omega_i$ represents the direction of a light source, but it could also be the direction of a surfel reflecting light or some other light emitting/scattering element in space. $\omega_o$ often represent the direction of the viewpoint (camera/eye), but it could as well be the direction of a surfel or other point in space whose incident radiance we are interested in calculating.

While $\omega_o$ is valid for normal oriented hemisphere above the surface, there are differences in valid $\omega_i$ domain between BxDF's. Because BRDF defines the amount of reflected light $\omega_i$ is valid for the normal oriented hemisphere above the surface. On the other hand, BTDF defines the amount of light transmitted through the surface so $\omega_i$ is valid for the normal oriented hemisphere below the surface. BSDF is the sum of the two so $\omega_i$ is valid for the entire sphere.


Is depth of field incongruous in a 3D still image? If rendering an image in 2D, adding depth of field effects (blurring objects further from the focal distance) adds realism and draws the eye to the object of the image. With a 3D (i.e. stereo) image, looking at an object in the image at a given depth will makes objects at all other depths defocused (not blurred, but incorrectly aligned by the eyes, giving a double image). This means that if depth of field effects are used, there will be conflicting results: looking at an object that is at a different depth will cause that depth to be the only depth not having a double image, but it is also a depth that is blurred. This gives the object a property of being focused upon, and a property of not being focused upon. In a 3d still image, are depth of field effects detrimental to the acceptance of the image by the eye, or are there ways around this?

In traditional stereo 3D, I don't believe that there is a way to make a fixed focal plane feel natural to the viewer. When looking at an out-of-focus object in stereo 3D, the object remains out-of-focus, causing conflicting cues. The lens in the eye tries to adjust to bring the object into focus, but of course it won't succeed, causing eye strain and headaches.

However, there is hope outside stereo 3D: Lightfield displays, such as this nvidia prototype, go a different route. In stereo 3D, the light in the scene is already captured by two virtual (or physical) cameras, "baking" in the focal plane. Head-mounted displays like Oculus Rift then attempt to tape two displays in front of your eyes in such a way that the retina receives the exact same image that was captured by the camera. Lightfield displays go a different route: Instead of capturing two images ahead of time, they reproduce the entire 4D light field in front of your eyes, allowing your eyes to capture the image as if they were sitting directly inside the virtual scene. This has a number of benefits, including much smaller and lighter hardware as well as giving your eyes the ability to refocus.

If there is a way to make lightfield displays technically and commercially viable, then I believe they can remove the need for depth of field and fixed focal planes entirely and make VR feel a whole lot more comfortable for the viewer. However, it is likely not possible to construct lightfield screens, so televisions and cinemas won't be able to use this technology.


Why does my code to plot a cone actually generate a roof? per this post, this formula

$Z = 1 - \sqrt{X^{2} + Y^{2}}$

generates a cone where the point is at (0, 0, 1) and it spreads out below that. It meets the x-y plane at the unit circle

I am trying to reproduce with Python

ax = plt.figure().gca(projection='3d')
xx, yy = np.meshgrid(np.arange(-1,1.1,.1), np.arange(-1,1.1,.1))
zz = 1 - np.sqrt(xx**2, yy**2)
ax.plot_surface(xx, yy, zz, alpha=.5)

and get this "roof"

each of xx, yy, zz is a 21 by 21 matrix, even if I increase them to 210 by 210, nothing changes, what am I missing?

You need to change the call to sqrt from:

np.sqrt(xx**2, yy**2)

to:

np.sqrt(xx**2 + yy**2)

Otherwise you're passing an array of values to the function and will be returned an array of values for the answer.

See here for details:

  >>> np.sqrt([1,4,9])

  array([ 1.,  2.,  3.])


Incorrect values written in stencil buffer I am trying to pick objects on mouse click. For this I have followed this tutorial, and tried to use the stencil buffer for this purpose.

Inside "game" loop I am trying to draw 10 (5 pairs) 'pick'able triangles as follows:

...

glClearColor(red, green, blue, 1.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

glClearStencil(0); // this is the default value

/* Enable stencil operations */
glEnable(GL_STENCIL_TEST);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

/*Some other drawing not involving stencil buffer*/

GLuint index = 1;
for (GLshort i = 0; i < 5; i++)
{
    //this returns 2 model matrices
    auto modelMatrices = trianglePairs[i].getModelMatrices();

    for (GLshort j = 0; j < 2; j++)
    {
        glStencilFunc(GL_ALWAYS, index, -1);
        glUniformMatrix4fv(glGetUniformLocation(ourShader.Program, "model"), 1, GL_FALSE, glm::value_ptr(modelMatrices[j]));
        glDrawElements(GL_TRIANGLES, 3, GL_UNSIGNED_INT, BUFFER_OFFSET(2));
        index++;
    }
    /*Some other drawing not involving stencil buffer*/
}

/*Some other drawing not involving stencil buffer*/
...

However, when I am trying to read back the stencil values, I am getting wrong values. I am reading back the values as (this is also a part of the above-mentioned tutorial):

GLuint index;
glReadPixels(xpos, Height - ypos - 1, 1, 1, GL_STENCIL_INDEX, GL_UNSIGNED_INT, &index);

Whenever, I click the first triangle of the pair I am getting values as i+1, whereas the correct value should have been i, and for the second triangle of the pair, I am getting 0 as index.

Please let me know what am I missing here?

Update
I have found that stencil values can be applied on quads. When I tried to apply the stencil value on rectangle it worked correctly. Though it is not the solution I was looking at, still, I can live with that. However, I would still like to know what was the problem with triangles.

OK, so this not really an answer, rather just sharing what I have learned.

To clarify, shape of the object has nothing to do with this problem at all. Be it a triangle or rectangle, and wherever you click on that object, this method should work (the 'But' for this part is explained below).

I think stencil buffer values are associated with each pixel on screen, and it does not directly maps to the OpenGL window space. Initially I was using a window size that is smaller than my screen size and on top of that it was not full screen. Each time I started (debugging from VS 2015), the window appeared not exactly at (0, 0) on the screen. Some of the screen space is also consumed by the window bar at the top. I think this is the reason for getting erratic values when reading back the data from stencil buffer.

So as a simple solution to this problem what I did is to make the window full screen, and everything worked like charm. As I use GLFW, here is how I have done it:

glfwCreateWindow(Width, Height, "Title", glfwGetPrimaryMonitor(), nullptr);

Just used glfwGetPrimaryMonitor(), while creating the window. And that's it.

And yes, it works for triangles too.


Apply Projective Transformation 2D Rectangle I want to write code from scratch about Projective Transformation in order to apply this transformation to 2d rectangular. I searched about this transformation, but I could not figure out completely. These are some formulas I have seen:

   x' = xH
   y' = yH
   A.H = 0
   A = UΣV^T

According to these formulas,SVD is calculated from A matrix, and then H is obtained from V. Then target points should be found by H. But this is a bit confusing. Could you explain it? Also is there any alternative solution like composition of transformations like Similarity Transform which can be obtained by multiplying rotation, translation and scale matrixes?

EDIT

I watched this youtube video (https://www.youtube.com/watch?v=54Qtu3S9HJU) . It explains how calculate homography by the equations above after 12 minutes.This method seems very complex. I want to learn how exactly we calculate homography and obtain another rectangle by applying Projective Transfrom?

Since homography is not my topic, I can't really tell if I am missing some important edge cases, but after some research, I think I got the basics. I will use the same variable names, the guy in the video uses on slide 15 (slide numbers are in the bottom right corner on the red background).

You start by defining 4 points in the plane $X$ where you say you know their projected points in plane $x$. You assume, that the transformation between $X$ and $x$ is linear and can be described by a matrix $H$. Means
$$x = HX$$

Now you want to calculate the Matrix $H$ that perfoms the projection:

$$H = \begin{bmatrix}h_1&h_2&h_3\\h_4&h_5&h_6\\h_7&h_8&h_9\end{bmatrix}$$

$h_1$ to $h_9$ are unknowns. So how do you get them? Well, you have 4 times:

$p_i' = Hp_i$

where $p_i'$ and $p_i$ are the projected and the original points. This full equation looks like this:

$$\begin{bmatrix}x_i'\\y_i'\\1\end{bmatrix}= \begin{bmatrix}h_1&h_2&h_3\\h_4&h_5&h_6\\h_7&h_8&h_9\end{bmatrix}\begin{bmatrix}x_i\\y_i\\1\end{bmatrix}$$

Perform the multiplication and you will get this:

$$\begin{bmatrix}x_i'\\y_i'\\1\end{bmatrix}= \begin{bmatrix}x_ih_1+y_ih_2+h_3\\x_ih_4+y_ih_5+h_6\\x_ih_7+y_ih_8+h_9\end{bmatrix}$$

You can also write this in some kind of nested vector fashion:

$$\begin{bmatrix}x_i'\\y_i'\\1\end{bmatrix}= \begin{bmatrix}\bar{h_1}^T p\\\bar{h_2}^T p\\\bar{h_3}^T p\end{bmatrix} = \begin{bmatrix}p^T\bar{h_1}\\p^T\bar{h_2}\\p^T\bar{h_3}\end{bmatrix}$$

where
$$\begin{matrix}\bar{h_1} = \begin{bmatrix}h_1\\h_2\\h_3\end{bmatrix}&\bar{h_2} = \begin{bmatrix}h_4\\h_5\\h_6\end{bmatrix}&\bar{h_1} = \begin{bmatrix}h_7\\h_8\\h_9\end{bmatrix}\end{matrix}$$

Now here comes the trick (got it from this link). If  $p'$ is equal to the vector $Hp$ their cross product must be the zero vector $\bar{0}$:

$$p' \times Hp = \bar{0}$$

Applying the cross product will give you three equations:

$$\begin{matrix}y_i'p^T\bar{h_3} -  p^T\bar{h_2} &= 0 \\ p^T\bar{h_1} - x_i' p^T\bar{h_3} &= 0\\x_i' p^T\bar{h_2}-y_i' p^T\bar{h_1}&=0\end{matrix}$$

Write this as a matrix equation and you get:

$$\begin{bmatrix}\bar{0}^T &-p^T&y_i'p^T\\p^T&\bar{0}^T&-x_i'p^T\\-y_i' p^T&x_i' p^T&\bar{0}^T\end{bmatrix} \begin{bmatrix}\bar{h_1}\\\bar{h_2}\\\bar{h_3}\end{bmatrix}= \begin{bmatrix}0\\0\\0\end{bmatrix}$$

Now, if you look at the matrix, you might notice, that the 3 columns are not linear independent. $-y_i'$ times the first column minus $x_i'$ times the second one, yields the third column. So you can drop one.

If you drop the last column and replace all the vectors with their corresponding entries, you get:

$$\begin{bmatrix}0&0&0 &-x_i&-y_i&-1&y_i'x_i&y_i'y_i&y_i'\\x_i&y_i&1&0&0&0&-x_ix_i'&-y_ix_i'&-x_i'\end{bmatrix} \begin{bmatrix}h_1\\h_2\\h_3\\h_4\\h_5\\h_6\\h_7\\h_8\\h_9\end{bmatrix}= \begin{bmatrix}0\\0\end{bmatrix}$$

You can multiply the second row by $-1$ and swap it with the first one (that does not change the solution of the system) to get the exact same formula as presented in the video on the slide 17.

If you apply the formula to all 4 points you get the system:

$$\begin{bmatrix}
0&0&0 &-x_1&-y_1&-1&y_1'x_1&y_1'y_1&y_1'\\
x_1&y_1&1&0&0&0&-x_1x_1'&-y_1x_1'&-x_1' \\
0&0&0 &-x_2&-y_2&-1&y_2'x_2&y_2'y_2&y_2'\\
x_2&y_2&1&0&0&0&-x_2x_2'&-y_2x_2'&-x_2' \\
0&0&0 &-x_3&-y_3&-1&y_3'x_3&y_3'y_3&y_3'\\
x_3&y_3&1&0&0&0&-x_3x_3'&-y_3x_3'&-x_3' \\
0&0&0 &-x_4&-y_4&-1&y_4'x_4&y_4'y_4&y_4'\\
x_4&y_4&1&0&0&0&-x_4x_4'&-y_4x_4'&-x_4' \\
\end{bmatrix} \begin{bmatrix}h_1\\h_2\\h_3\\h_4\\h_5\\h_6\\h_7\\h_8\\h_9\end{bmatrix}= \begin{bmatrix}0\\0\end{bmatrix}$$

The problem is now, that you have 9 unknowns but just 8 formulas. According to the answers in this question, one possible solution is to constraint $h_9$ to be 1 and solve the resulting linear system. However, another way is to solve it as an underdetermined least-squares problem. This might be done using the SVD. However, as I said in the comments, this is a rather complicated algorithm. An easier one is to use QR decomposition, which can be a substep of the SVD, but this is also not too easy to code yourself. Since you don't seem to be that experienced in advanced linear algebra, you might try the constraint technique first, before trying the SVD.

You can also use more than 4 points, but then you get an overdetermined system, which can be solved with the SVD, QR decomposition or any other least-squares solver.


Subtle bugs in Fortune's algorithm implementation I am in the middle of a process of implementation of Fortune's algorithm for a Voronoi diagram generating on a plane. Here the last C++ debug version. Very rarely I catch (visually) such a bugs:

Debugging process of a geometrical algorithms considered very complex by me. I even don't know which runtime checks (assert-ions) should I add to catch this bugs on the fly.

The question is addressed to those who have tried to implement Fortune's algorithm. I sure, if one faced similar bugs in his implementation somewhere in the middle of a way, then one may remember how to avoid it.

What is the source of the bug? What is the main corner cases (except rectangle mesh, concentric points, etc) can I face?

This commit contains the fix of the bug.

In short, it is wrong to disable event for second edge, during event checking, when first edge may be truncated sooner then stated by its current associated event.


Should I prefer an AMD or Nvidia GPU for studying graphics programming, especially with respect to vendor specific debugging tools and ecosystem? My current laptop has a fairly old Intel integrated GPU which was good enough for following simple tutorials.

I am now keen on getting to grips with advanced techniques and newer APIs, and so I am looking for a new discrete GPU.

I want to ignore the usual price/performance metric for GPUs, as I assume that's going to be irrelevant for a hobbyist programmer.

Is there a pragmatic reason to prefer AMD over Nvidia or vice versa?

A cursory search shows that some tools are exclusive to one vendor (e.g. Nvidia Nsight), but is one significantly better than the other or are the differences minor?

Will drivers make a difference here also? And finally will I run into a lot of example code specific to one vendor?

If you want to learn OpenGL X.Y or DirectX A.B then shop for hardware that supports that specification.

If you're using other software (Blender, Maya, 3DSMax, etc) you'll want to make sure it's supported.

If your OS has a history of issues with a particular vendor, adjust accordingly.

Beyond that, it's not going to matter much, especially at the hobbyist level.
Price / performance metrics are still relevant, it's just that the definition of performance is different.


Can you avoid querying for a texture location in OpenGL? The way I learnt to send a texture to a shader is more or less

//Use program
//Bind texture to the texture unit to its appropriate target
glActiveTexture(GL_TEXTURE0 + texture_unit);
glBindTexture(target, textureID);
//Get the uniform location in the program and attach the texture unit
GLuint location = /*get the location somewhow*/;
glUniform1i(location,texture_unit);

However when dealing with buffers and SSBOS rather than trying to find a location through a string name, the object is declared as:

layout(std430, binding = 2) buffer texture_meta_data

For example. This means that the SSBO is to be bound at binding point 2

And then on the C++ side one does something like

glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 2, SSBOID);

To bind the object to the binding point.

How can you do the same with textures?

By using the exact same syntax as you did for SSBOs:

layout(binding = 0) uniform sampler2D texture_name;

This is a GL 4.2 feature, so if you can use SSBOs, you can use this.


How does Smith multiple scattering interact with diffuse subsurface scattering? The Multiple Scattering Microfacet BSDFs with the Smith Model paper describes a statistical model for replacing the masking-shadowing functions in microfacet BSDFs (which account for paths with more than one surface intersection by setting their contribution to 0) with a distribution which can be path traced and allows the a ray to intersect a microfacet surface several times before exiting.

They do this by modifying a volumetric (microflake) model to behave like a heightfield: to never collide with anything "above" the surface and to always collide with anything "below" the surface.

The result, using wording from their slides, is that "the ray can never go through the Smith volume; the model creates an opaque surface-like interface."

Is this model compatible with traditional diffuse subsurface scattering, where a path can travel a perceptible (macro-scale) distance through a surface before exiting? Or is this simply a specular BSDF with no intention of modeling long paths internal to a surface, to be combined with a diffuse BSDF that adds that component?

(Does the microflake model of volumetric rendering normally separate diffuse and specular, or is "diffuse" simply a path that bounces around so many times that the outgoing direction is uniformly distributed?)

The goal of Heitz et al.'s model is pretty much the opposite of subsurface scattering: They only consider surface scattering, i.e. the ray can never enter the material.

Because microfacets are statistical in nature, they can recast their problem in such a way that it can be solved by microflakes, which allows them to compute properties such as the mean free path to derive a heightfield sampling procedure. However, even though they use microflake theory, they still solve the same problem of multiple scattering on microfacets, and their result is still a BSDF, not a subsurface scattering model.

Because it is a BSDF, the path exits at the same location as it hit the surface. This is also mentioned in the introduction:

  Note that our volume-scattering process models the interaction with a surface, but its application in rendering is virtual in that no displacements occur, i.e. the incident and exitant location are the same and the resulting plane-parallel radiometry produces a BSDF.


Phong: Real-time rendering of Metal and Plastic Materials Using a shader that works like Phong, what would be the difference between painted Metal and plastic?

I seem to remember reading in books years ago, that I no longer have access to, the following:

Specular color for painted metal is same color as the paint.
Specular color for plastic is always white.
Ambient for metal should be darker than for plastic.

This also seems to correlate with what I see when I observe objects (in real world)

But I tried to verify this also by checking the Internet, and found conflicting versions, some said the complete opposite.

So what is true?

Diffuse colours on materials typically come from within the material, while the specular colour is from the very surface. Coloured plastic materials are made by embedding particles of dye inside a colourless medium, so the diffuse colour is the colour of the dye, while the specular colour is white from the colourless surface. With metals, all of the reflection is at the surface, so the diffuse and specular colours are the same.

There are many different kinds of paint, but most commonly they're made like plastics: putting dye particles inside a white or colourless medium. Emulsion, acrylic, and enamel paints are all made this way. For this reason, painted or varnished metal surfaces have white specular as if they were plastic. There are some kinds of paint (metallic paints being the most obvious example) that aren't made this way, so it's not always true.

Ambient doesn't correspond to a physical property of the material: it's a cheap substitute for global illumination. It represents all of the indirect light that falls on the shading point and is reflected in the viewing direction. There's no physically correct way to set it: you should just do what looks best in your scene. That said, I'd offer a rule of thumb that shinier objects probably want less ambient, because less of the indirect light is going to be reflected diffusely to where you can see it. However, you might consider not using ambient at all, and pointing dim fill lights at places that will come out too dark without indirect light.


How does state of the art real time hair rendering work? It is common knowledge that hair simulation and rendering is particularly challenging and in fact rare are the examples in games that propose believable hairs. It is pretty easy to imagine how hard it is to actually simulate the high amount of fibres and to simulate the various scattering events that may happen between so many strands, let alone the fact that each of them is translucent.

Can someone formalize what are the main challenges in hair rendering, especially for real time scenarios? How these are overcome by state-of-art techniques? Are there some de-facto standard theoretical models used?

To clarify, for real time I don't necessarily mean in a game context.

Sorry for the bad quality of my answer. I do not have access to a computer currently and editing from my phone is not a straightforward task. In particular I would love to be able to paste images.

I would say that the main challenges of simulating hair are:

replicating their very specific reaction to lighting (as a material)
replicating their volumetric property (as a geometry)
animating them in a realistic manner

Here is a list of references I gathered on the subject, ordered chronologically (mainly about the rendering part):

JAMES T. KAJIYA, TIMOTHY L. KAY, Rendering Fur With Three Dimensional Textures, 1989
JEROME LENGYEL, EMIL PRAUN, ADAM FINKELSTEIN, HUGUES HOPPE, Real-Time Fur over Arbitrary Surfaces, 2000
STEPHEN R. MARSCHNER, HENRIK WANN JENSEN, MIKE CAMMARANO, Light Scattering from Human Hair Fibers, 2003
ARMIN BRUDERLIN, SUNIL HADAP, TAE-YONG KIM, NADIA MAGNENAT-THALMANN , ULRICH NEUMANN, YIZHOU YU, STEVE WORLEY, Photorealistic Hair Modeling, Animation, and Rendering, Siggraph 2003 Course
THORSTEN SCHEUERMANN, Hair Rendering and Shading, ShaderX3, 2004
THORSTEN SCHEUERMANN, Hair Rendering and Shading, ATI / GDC 2004
THORSTEN SCHEUERMANN, Practical Real-Time Hair Rendering and Shading, ATI / Siggraph 2004
THORSTEN SCHEUERMANN, Practical Real-Time Hair Rendering and Shading, ATI / Siggraph 2004 (slides)
MARTIN KOSTER, JORG HABER, HANS-PETER SEIDEL, Real-Time Rendering of Human Hair using Programmable Graphics Hardware, 2004
HUBERT NGUYEN, WILLIAM DONNELLY, Hair Animation and Rendering in the Nalu Demo, GPU Gems 2 (Chapter 23), 2005
LENA PETROVIC, MARK HENNE, JOHN ANDERSON, Volumetric Methods for Simulation and Rendering of Hair, Pixar 2006
CHRISTOPHE HERY, RAVI RAMAMOORTHI, Importance Sampling of Reflections from Hair Fibers, Pixar 2007
SARAH TARIQ, LOUIS BAVOIL, Real-Time Hair Rendering on the GPU, Siggraph 2008.
ARNO ZINKE, CEM YUKSEL, ANDREAS WEBER, JOHN KEYSER, Dual Scattering Approximation for Fast Multiple Scattering in Hair, Siggraph 2008
IMAN SADEGHI, HEATHER PRITCHETT, HENRIK WANN JENSEN, RASMUS TAMSTORF, An Artist Friendly Hair Shading System, 2010
EUGENE D’EON, GUILLAUME FRANCOIS, MARTIN HILL, JOE LETTERI, JEAN-MARIE AUBRY, An Energy-Conserving Hair Reflectance Model, 2011
XUAN YU, JASON C. YANG, JUSTIN HENSLEY, TAKAHIRO HARADA, JINGYI YU, A Framework for Rendering Complex Scattering Effects on Hair, 2012
JIAWEI OU, FENG XIE, PARASHAR KRISHNAMACHARI, FABIO PELLACINI, ISHair: Importance Sampling for Hair Scattering, 2012
SARAH INVERNIZZI, On Physically Based Hair Rendering, 2013
WOLFGANG ENGEL, Hair Rendering in Tomb Raider, 2013
EUGENE D’EON, STEVE MARSCHNER, JOHANNES HANIKA, Importance Sampling for Physically-Based Hair Fiber Models, 2013
TIMOTHY MARTIN, WOLFGANG ENGEL, NICOLAS THIBIEROZ, JASON YANG, AND JASON LACROIX, TressFX: Advanced Real-Time Hair Rendering, GPU Pro 5, 2015
LEONID PEKELIS, CHRISTOPHE HERY, RYUSUKE VILLEMIN, JUNYI LING, A Data-Driven Light Scattering Model for Hair, Pixar 2015

I did not read all of these material yet, but here is a little of what I remember (I am more interested by the real-time solutions) :

Kajiya-kay laid the basis for realistic hair / fur rendering
Marschner proposed an improved model, and noted that hair has two distinct specular highlights (one white due to direct reflection, the other colored and sparkling due to reflection inside of the hair fiber)
Scheuermann proposed approximations to simulate the behaviour described by Marschner, approximations suited for real-time (using tricks such as two separate specular highlights, artist-designed specular offset and exponent, noise textures). He also proposed using three rendering passes (opaque, transparent back and front) of polygons to simulate the "volumetric" qualities of hair
Most of the games I heard of seem to use Scheuermann approaches for hair shading. However, this is not an energy-conserving solution. Nevertheless, I think d'Eon attempts to fix this problem in the 2011 paper.
An other topic of interest is the use of tessellation, that has also been used recently to replace the polygonal approach and simulate hair strands individually.

The offline rendering world attempts to solve more advanced challenges. I think most of them are described in the references above.


Shader storage buffer indexing by 4 bytes instead of 3? I am trying to send rendering information for an instancing rendering using shader buffer objects. My current problem is that I have 2 shader buffer objects, one which works just fine, and another that is not quite working.

The idea is to render squared faces at different positions in the screen. For this, I create a vector of glm::vec3 that represent the position of each face, put all of them in an std::vector and use them as the data for the Shader buffer object.

In other words I am creating the data as follows:

face_positions.push_back(vec3(0,0,0));
visible_faces.push_back(Left);

face_positions.push_back(vec3(0,-10,0));
visible_faces.push_back(Front);

face_positions.push_back(vec3(0,0,0));
visible_faces.push_back(Back);

Then updating the buffers as follows:

glBindBuffer(GL_SHADER_STORAGE_BUFFER, cube_rendering_VBOs[3]);
glBufferData(GL_SHADER_STORAGE_BUFFER, visible_faces.size()*sizeof(Face), visible_faces.data(), GL_DYNAMIC_COPY);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 3, cube_rendering_VBOs[3]);

glBindBuffer(GL_SHADER_STORAGE_BUFFER, cube_rendering_VBOs[4]);
glBufferData(GL_SHADER_STORAGE_BUFFER, face_positions.size()*sizeof(vec3), face_positions.data(), GL_DYNAMIC_COPY);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 4, cube_rendering_VBOs[4]);

And finally rendering by doing:

glDrawElementsInstanced(GL_TRIANGLES, MESH->indices->size(), GL_UNSIGNED_INT, (void*)0, visible_faces.size());

Obviously there is more code associated with passing the actual mesh geometry used for the instanced rendering and enabling the correct layouts in the vertex shader, but I cannot put my enire code base here and I have more throuroughly tested those, so I beleive the mistake is in how I am passing or reading the shader buffer objects.

The behaviour is, if you look at the lines:

face_positions.push_back(vec3(0,0,0));
visible_faces.push_back(Left);

face_positions.push_back(vec3(0,-10,0));
visible_faces.push_back(Front);

face_positions.push_back(vec3(0,0,0));
visible_faces.push_back(Back);

The face_positions() should be the poisitions of the faces.

However what I have noticed is, the first 3 floats (i.e face_positions[0]) work just as expected. Then one value is skipped (i.e face_positions[1].x) and the next values read are the next 3 (i.e face_positions[1].y face_positions[1].z face_positions[2].x) then once again one value is skipped, and it starts reading the next value (i.e face_positions[2].z).

So it seems somewhere something is padding the array with 0's to make everything 4 bytes aligned.

Does anyone have any idea of where to start looking?

Does anyone ahve[sic] any idea of where to start looking

You fell victim to one of the classic blunders. Never use a vec3 in a UBO or SSBO.

While vec3s take up 12 bytes, they always have the alignment requirements of a vec4. Yes, even under std430 layout, they always have vec4 alignment. So if you have an array of vec3s in a U/SSBO, what you really have is an array of vec4s where one of the elements doesn't get used.


How to find Light Direction from a Point on a surface to an Area Light How can I a find the Light Direction/Ray from a Point on a surface to an Area Light? Is there a universal technique to do this for all types of lights (circle, rectangle, sphere & tube)?

Although this is for area lighting in a ray tracer, is there another way to implement area lighting through an emissive value that I can apply to any object or surface (I have seen this mentioned online)? As opposed to doing it like this.

For example for a point light I know it can be done like this:

class PointLight : public Light
{
    Vec3f pos;
public:
    PointLight(const Matrix44f &l2w, const Vec3f &c = 1, const float &i = 1) : Light(l2w, c, i)
    { l2w.multVecMatrix(Vec3f(0), pos); }
    // P: is the shaded point
    void getDirectionAndIntensity(const Vec3f &P, Vec3f &lightDir, Vec3f &lightIntensity) const
    {
        lightDir = pos - P; // compute light direction
        float r2 = lightDir.norm();
        lightDir.normalize();
        lightIntensity = intensity * color / (4 * M_PI * r2);
    }
};

If you want to use only one sample to approximate analytical area lighting (e.g. for real-time applications), you can use Most Representative Point (MRP) approximation as described in "Lighting in Killzone Shadow Fall", "Real Shading in Unreal Engine 4" and "Moving Frostbite to PBR". This works reasonably well for specular approximation, though the energy conservation is a challenge. What I have found working best is to find a point on a light source with smallest angle from the view reflection vector, and the solution depends on the shape of the light (for example here's my solution for rectangular lights).

For diffuse evaluation there are numerical and analytical approximations depending on light shape presented in the papers, which also properly handle the horizon case. You can also try to use MRP for diffuse lighting, but I haven't found good approximations for it. Depending your application though diffuse MRP might result in feasible quality.

There's also some more recent work for evaluating area lighting integrals not using MRP. "Accurate Analytic Approximations For Real-Time Specular Area Lighting" deals with the specular evaluation via edge integrals, though I recall this method is patented. "Real-Time Polygonal-Light Shading with Linearly Transformed Cosines" looks quite promising, but I haven't checked the paper in more details.

For non-real time applications and ray tracing area lighting is fundamentally and mathematically much simpler problem. For your integrator you just check the incident radiance at each sample (ray intersection with the scene) and accumulate results for total radiance towards the output direction. The challenge is how to reduce variance of your integrator, but getting basic area lighting to work in a ray tracer is relatively simple.


Midpoint Algorithm Fast Direction I've been exploring the Midpoint algorithm for drawing lines and curves, and I have a quick question: How should you determine the fast direction for a curve? I've seen that if the shape is simple, you can draw one section and repeat the rest, but what if the shape does not have this property?

Right now I'm just using the derivative(slope), so I wonder if this is correct or cheaty? :P

I have never done this so this is just my best guess. I think you are on the track with the slope but I'm going to expand it little with some calculus. If you parameterize your curve by some variable t:

x(t) = ...

y(t) = ...

Then you can compute dx/dt and dy/dt which is how much x or y changes as you change t. This runs into some problems on a general purpose curve though. For example dx/dt might be a function of t. This means that the rate of change of x itself is changing as you move along the curve. But then you can also find dx/dt^2 (rate of change of the rate of change) and so on. This method is known as forward differencing.

Then what you can do is have an x, y starting point and draw a pixel there. Then you add the first order difference (dx/dt, dy/dt) to x and y. Then you add the second order difference to the first order difference and so on.

Here is a link I found on forward differencing:
http://www.nptel.ac.in/courses/122104018/node103.html

I haven't read it but it looks promising.

--edit--

Here's an example that was derived in my computer graphics class:
Some of the algebra can be tricky so it may or may not be correct.

Q(t) = a*t^3 + b*t^2 + c*t + k
0 < t < 1
segment size d
num segments: 1/d

Q1(t) = Q(t + d) - Q(t)
= 3*a*d*t^2 + 3*a*t*d^2 + a*d^3 + 2*b + d + b*d^2 + c*d

Q2(t) = Q1(t + d) - Q1(t)
= b*a*t*d^2 + 6*a*d^3 + 2*b*d^2

Q3(t) = Q2(t + d) - Q2(t)
= b*a*d^3      <==== no longer depends on t!

Iterate:

init variables:
Q  = Q(0)  = k
Q1 = Q1(0) = a*d^3 + b*d^2 + c*d
Q2 = Q2(0) = b*a*d^3 + 2*b*d
Q3 = Q3(0) = b*a*d^3

repeat 1/d times:
    drawLine(Q, Q + Q1)
    Q  += Q1
    Q1 += Q2
    Q2 += Q3

Q is just for one of the x or y variables. You could have a QX, QX1, ... and QY, QY1, ... and forward difference them both in the same loop. Then draw using (QX, QY) and (QX + QX1, QY + QY1). You could also extend this into as many dimensions as you like.


Reasons of the assumptions for the microfacet distribution function? The paper Microfacet Models for Refraction through Rough Surfaces (among others) reminds us the following assumptions about the microfacet distribution function D:

Microfacet density is positive valued
Total microsurface area is at least as large as the corresponding
macrosurface’s area
The (signed) projected area of the microsurface is the
same as the projected area of the macrosurface for any
direction v

I can see why 1) a distribution density is a positive value, and intuitively believe that 2) means that the total area of sloped microfacets cannot be smaller than their projection.
However I am not sure to understand the justification for 3). What does the third condition mean mean?

It's a geometric assumption like the other two. Consider a flat macrosurface. Its projected area in any direction $v$ is just $v\dot\ \hat N$ times its area (where $\hat N$ is the surface normal). In particular, the case where you're looking at it along the normal is simplest: the projected area is equal to the area of the surface.

Now split the macrosurface into microfacets. The total area of the microfacets is at least as much (assumption 2), but each 'kink' in the surface bends the normals of the separate microfacets away from the original normal. Whatever the shape of the microfacets, the sum of their projected areas doesn't change. In the case where you're looking along the normal, it's easy to see that the total projected area is the same: the surface would have to get larger or smaller for it to change.

For any direction, the microfacet has to cover a portion of the original projected area of the surface. Changing the orientation of the microfacet while still filling that portion doesn't change its projected area.

There's one tricky case, which is where the microfacets overhang each other. In this case, the total area is greater, because some area is covered by more than one microfacet. But in this case, at least one of the microfacets has to end up pointing away from the view direction, back into the surface. In this case, the dot product is negative, so this cancels out the area covered by more than one microfacet. This is why the text is careful to single out that it's the signed projected area.

There's one more tricky case, which is where the microfacets extend past the silhouette of the object. This might happen when you're looking from very glancing angles, or where overhanging facets overhang outside the perimeter of the surface. In this case, the projected area of the microfacets will be greater, violating the third assumption. We don't generally consider this case. Intuitively, it matches up with the fact that techniques like bump-mapping don't change the shape of the silhouette of the object.


How to get Ray Tracer working in Visual Studio Right now I am reading Peter Shirley's Ray Tracing in One Weekend book, I want to be able to use the provided source code but I am not sure what platform or IDE it was written on, because of this it does not compile in Visual Studio.

Can somebody help me? Whenever I try compiling the code it asks me to 'attach to process'.

How can I get this working? In a few reviews I have read online it says that I'll need to do some tweaking to get it working but I honestly don't know how.
Would it mean rewriting the entire ray tracer as a Win32 Project?

No MakeFile or Visual Studio Solution file is provided. So you will need to do the setup yourself in the Visual Studio IDE. Note that you could just drag and drop the files (.h and .cc are known extensions to Visual Studio) in a new console application if you are not/less familiar with Visual Studio.

The code itself is just plain C++ code. No special stuff. So it does not matter which platform or IDE was used during development. In fact the code is pretty self containing except for

#include <math.h>
#include <stdlib.h>
#include <iostream>
#include "float.h" -> <float.h>

But if your Visual Studio is installed and configured correctly, every console application can just include these headers without any problems. (Just try a basic hello world console application, since the problem has in my opinion nothing to do with the code you want to use.) Note that you need a Win32 Console application and NOT a Win32 Project.

It could be possible that the pre-compiled header file is enabled (StdAfx.h or pch.h) in which case you could disable this or include the header as the first included header in every .cpp file.


Spline interpolation library in cpp Have been searching a lot for a good spline interpolation library in cpp, came across one, which is the famous Eigen library , having the unsupported counterpart for spline fitting.<Here>.
I need spline fitting API that also accepts derivatives.
Eigen's InterpolateWithDerivatives API claim to offer that but it is not without bugs. What alternatives do I have?

So you have a series of points and, at each point, a supplied derivative?
Is a piecewise cubic sufficient or does it need higher derivative continuity?  If the former is ok, then Cubic Hermite Splines will do the job. (If you need, they can be trivially mapped into cubic Beziers)


How do we know if a ray intersected a triangle in one in a million I've taken a class about ray-plane intersection on this: https://www.khanacademy.org/computing/pixar/rendering/rendering-2/v/rendering-5
Suppose we shoot a ray at the scene and this scene contains millions of triangles. How can we know if the ray intersected one of the triangle? Do we just run a check on this millions of triangle if it intersects with the ray? That would be inefficient!!

If you want to know all triangles that are hit by the ray, you have to check all of them. However, that doesn't mean that you have to check each of them individually. Usually, you apply some sort of space partitioning to your scene and group related objects in a bounding volume hierarchy. Then it gets a lot more efficient because you can drop multiple triangles with a single check.
For example, consider a scene with a rectangular area. You might subdivide this area into a 10x10 grid. Now check which parts of the grid are hit by the ray. All objects located in grid cells that aren't hit by the ray can be discarded and you only need to check the remaining objects. Now let's say that one of the hit objects is a human. Before you test the individual triangles, you narrow down the affected area in multiple substeps by checking against its bounding volume hierarchy. The traversal of the hierarchy might look like this: upper body hit -> left arm hit -> hand hit -> thumb hit. So with 4 checks, you can discard all triangles of the humans' mesh that do not belong to the thumb.
In case you only want to find the first triangle that is hit by the ray, you can also sort all hit cells and objects by distance before performing detailed tests. Then you start your checks at the closest object and continue until you get the first hit.


Finite difference wave equation shown in paper is wrong? I'm trying to produce wave surface animations, and I came across this paper: Fast_Water_Animation_Using_the_Wave_Equation_with_Damping.  In the paper they go to provide the following equation:

At first I just blindly implemented this, but soon saw some strange bugs I realized weren't from me. When you evaluate the equation, say, with delta t = 1/60, delta x = 1, c = 0.01, and k = 1.0, on a grid with all zeros except the center point, you realize there's something off about this.  If you are at that center grid point, and say your value is 1.0, the acceleration component( the delta t^2 stuff in the above equation) is positive when you assume all the surrounding neighbors are zero. That makes no sense. Peaks in waves do not accelerate upwards when the values around them are smaller. And further more, normally when I see finite difference of the laplacian, it is the average of the surrounding samples minus the current sample.

Indeed, when I reverse the sign on the acceleration component, I get what appears to be the correct result (before vs after on 16x16 grid)

This is what I changed the equation to:

Was this just some pervasive typo or am I completely missing something here?

Yes, it looks like a sign error in the paper. It looks like it was introduced in equation (4) and then carried through the following ones.

Probably the authors wrote the source code first and the paper after, so did not notice that their equations did not match what their code was doing. Ideally it should have been caught in the paper review, but, mistakes happen.


How to calculate camera distance so that after projective transformation a quad is filling the whole screen First of all, I'm using OpenGL and GLM.

Now, I have a square perpendicular to the z-axis and centered around it. Let's say that it has Z-value (offset along the Z-axis) X. Now, I want to set up a projective camera, its position is (0, 0, Y) looking at (0, 0, 0). The perspective is: viewing angle=45 degrees, far=100, near = 0.1, aspect=1) I'm using glm::lookat() and glm::perspective().
How can I calculate the value for Y from X (given the parameters I've described)?

Fortunately your scenario is rather simple, your camera is on a line directly perpendicular above and centered on the quad you're concerned about. So what you want is for the quad to fill the whole screen, I assume (more or less) exactly the whole screen, i.e. the size of your projected quad in window space is your entire viewport.

The mathematics of this are rather easy if you understand how the perspective matrix is actually constructed and what it does and if you know some basic trigonometry. The matrix constructed with glm::perspective takes as viewing angle the field of view along the y-axis and is based on goold old gluPerspective. And if we take a look at the actual matrix, we can see that it transforms a $1$ high object sitting on the view axis at distance $1$ from the camera to a height of $\cot \frac{\alpha}{2}$ ($\alpha$ being your view angle). If you move it further away it obviously gets smaller and if you increase its height, it gets larger, so it's $\frac{h}{d}\cot\frac{\alpha}{2}$ for camera distance $d$ and object height $h$. We want that height to be equal to $1$ (the height, from the center, of the view plane in NDC space).

So all you need to know in addition to that is the size of your square in the y-direction. We take half that height (because the square is centered) and multiply that by $\cot\frac{\alpha}{2}$ to get the distance it has to be from the camera. So if your square vertices have the y-coordinate +/-H and given your other values:

$$Y-X = H\cot22.5°$$
$$Y = X+H\cot22.5°$$
$$Y \approx X+2.41421H$$


Scaling models for 3D print So I need to print out this model, however with default sizes, one of the pieces is too large for my printer (M3D). I will need to scale it down.. the question is, do I need to scale the rest of the individual objects (pieces) in the same proportions, so I will be able to assemble all the parts afterwards ? This might be a bit of a stupid question, but I don't have any experience with 3D printing yet.. thank you.

Just like for any reduction or drawing, if you scale one part and not the other, their relative proportion is changed, so yes, aesthetic, matching, functionality are affected: scale everything in the same proportion. Nothing specific to 3D printing here.


How can I combine two filter kernels for image processing to create a single kernel? Is it possible to create a single kernel which can help us to perform an 2 step image processing?

For example, I want to first perform a median filter on my image to remove salt and pepper noise and then perform a sharpen operation on it.

For this purpose, I have to perform the following operation

(image * Median_Kernel) *  sharpening kernel = final_image

I want to have a kernel which combines the effect of Median_kernel and sharpening it such that

image * combined_kernel = final_image

Is this mathematically possible? If so, could someone please explain to me how to do this mathematically?

Why not just do them one after the other?

Img = (Img * median) * sharpen;

I think you should look into how these filter operations actually work before attempting to use them. It's pretty simple, use wikipedia or even any image processing textbook. Even the Matlab docs have good explanations.

Median filtering is a non linear filtering operation which means that there is no fixed kernel function for it. You may define a neighborhood size but other than that, nope, no kernel. Have you looked into how it works? I dont know how you could generate a "kernel" for a non-linear filtering operation.

Maybe a basic refresher in Image Processing is in order? Perhaps even going over basics of Signal Processing / DSP? I would suggest looking over 2D filtering if you can.


inside and outside region identification from 2D contour A 3D model is sliced against a plane and the resulting 2D contour is projected onto the plane. I am looking for an efficient algorithm to identify the inside and outside region of the contour. Simultaneously I need to identify outer and inner loops within the contours.

Any references from articles/books is earnestly requested.

Thanks

In general, I recommend the book Real-Time Collision Detection.

For your particular case, my first choice would be to use the 3D test for point-inside-mesh. I assume that the contour is a sequence of 2D segments (a sequence of vertices, essentially) and that the model mesh is "sealed", i.e. there are no holes in its surfaces that would make distinguishing the "inside" and "outside" of it impossible.

Find a bounding box for your mesh.
Cast rays from an edge of the bounding box to the tested point. In your case, the ray would be coplanar with the slicing plane, and the point would be one of the vertices of the contour.
Count how many times the ray crosses mesh triangles.
Odd number of crossings means that the point is on the inside; even number means it's on the outside.
Repeat points 2-4 for all contour vertices.
Classify contour regions by whether their vertices are on the inside or outside.


Why is it easier to inverse transform every object in a scene than to transform a camera? My lecturer for a computer graphics (raytracing) paper has stated that 'It is easier to apply the inverse transform to the world than it is to apply the transform to the camera.' The example given was that is it easier to shift all objects in a scene 5 units to the left than it is to shift the camera 5 units to the right.

This seems to imply that it's easier to iterate through and transform multiple objects in a scene than it is to apply a single transformation to the camera.

Is this correct, and if so, why is this the case?

That's actually incorrect. You can transform every ray of your camera if you wish (and numerous implementations do so). There are some advantages and disadvantages to each method (e.g. if your rays are more than your vertices you end up doing more transformations, however you don't incur cache misses by running over all vertices).


pseudoinverse Jacobian and adding more control in computer animation I am studying inverse kinematics.
But I wonder the suggested equations.
(The following equations are taken from 'Computer Animation Algorithms and Techniques Third Edition Page, 181')

To better control the kinematic model such as encouraging joint angle constraints, a control expression can be added to the pesudoinverse Jacobian solution.

($J$ is Jacobian, $J^+ = J^T(JJ^T)^{-1} = (J^TJ)^{-1}J^T$, and I will explain $z$ later)

The form for the control expression is the below eqution 5.25

$ \theta = (J^+J - I)z\cdot\cdot\cdot\cdot(5.25) $

But, the $\theta$ is always zero vector because $J^+J = I$, so $\theta = \mathbf 0z$.

The book said that a change to pose parameters in the form of Equation 5.25 does not add anything to the velocities, so the control expression can be added to the pseudoinverse Jacobian solution without chaining the given velocities to be satisfied.

$ V = J\theta \\
V = J(J^+J - I)z \\
V = \mathbf0z \\
V = \mathbf0 \cdot\cdot\cdot\cdot(5.26)$

And the book defined $z$ (Equation (5.27)) to bias the solution toward specific joint angles.

$z = \alpha_i(\theta_i - \theta_{ci})^2\cdot\cdot\cdot\cdot(5.27)$ where $\theta_i$ is the current joint angles, $\theta_{ci}$ are the desired joint angles, and $\alpha_i$ is joint gains.

Finally, The form of the conventional pseudoinverse of the Jocobian added by the contorol expression is Equation 5.28.

$\theta = J^+V + (J^+J - I)z \cdot\cdot\cdot\cdot(5.28)\\
\theta = J^+(V + Jz) -z \\
\theta = J^T[(JJ^T)^{-1}(V+Jz)] -z
$

However, I could not understand why the book derived the above equation. Because the equation 5.25 is  zero vector, the equation 5.28 of red part($\theta = J^+V + \color{red}{(J^+J - I)z}$) is zero vector and $z$ doesn't have any effect on the result. We just have $\theta = J^+V$, no difference from the real original conventional pseudoinverse.

What is wrong with me?

I think I found your misunderstanding, but it's IMHO based on a little inconsistency (or at least lack of clarity) in the book.

  But, the θ is always zero vector because J+J=I, so θ=0z.

This is not true, $J^+J$ is not necessarily $I$. $JJ^+=I$ but the multiplication by the pseudoinverse is not commutative. Let's look at this in more detail:

$JJ^+ = JJ^T(JJ^T)^{-1}= (JJ^T)(JJ^T)^{-1} = I$

$J^+J = J^T(JJ^T)^{-1}J = ?$

The underlying misunderstanding comes from the assumption that the matrices $J^T(JJ^T)^{-1}$ and $(J^TJ)^{-1}J^T$ are equal (since the latter would produce $I$ if right-multiplied by $J$), which is not true. They are both pseudoinverses of $J$ but they are not necessarily the same matrices, in fact they don't necessarily both exist.

As the book says, the $(JJ^T)^{-1}$ only exists if the rows of $J$ are linearly independent, and in turn Wikipedia says that $(J^TJ)^{-1}$ exists if the columns of $J$ are linearly independent. However, they can't both exist if the matrix is not square, since one of the dimensions would necessarily have more vectors than the dimension, which can't all be linearly independent. Specifically, for a normal redundant joint system you probably have more columns than rows, making only the pseudoinverse $J(JJ^T)^{-1}$ exist.

We can, however, see that for a square matrix they are indeed the same and actually the real inverse:

$J^T(JJ^T)^{-1} = J^TJ^{-T}J^{-1} = (J^{-1}J)^TJ^{-1} = I^TJ^{-1} = J^{-1}$

$(J^TJ)^{-1}J^T = J^{-1}J^{-T}J^T = J^{-1}(JJ^{-1})^T = J^{-1}I^T = J^{-1}$

(But in that case the system would have a unique solution and there's no point to angle control anyway.) But you cannot resolve the brackets with a non-square matrix, since the individual matrices aren't square and don't have an actual inverse. The only situation when both matrices exist are square $J$s and then they are indeed equal, but then there's also no point to angle control.

But I understand why you are confused and I am a little, too, since Parent actually seems to use the alleged identity $J^T(JJ^T)^{-1}=(J^TJ)^{-1}J^T$ in equation 5.20, where he derives the pseudoinverse's relevance for solving the inverse kinematics problem. He computes everything with the left inverse but than concludes with using the right inverse. I guess that works conceptually because they're both pseudoinverses, even if it doesn't make complete mathematical sense (to me and you at least). He seemed to have used the assumption of a square matrix as a trick for transforming the problem and then used the conclusion for a non-square matrix, which might be conceptually correct.

I guess it simply doesn't matter which of the two pseudoinverses we use, since only one of them ever exists (for a non-square Jacobian) anyway and if the left inverse would exist (making your above assumption true), that means you have more constraints than DoF and thus your system is overconstrained and there's no point in angle control.


Only show opaque objects behind transparent objects I would like to make certain objects in my scene transparent, but only the parts visible to the camera when opaque.

Currently, when I make objects transparent, I can see other transparent objects behind them, which I don't want. I only want to be able to see opaque objects behind transparent objects.

This is what happens now:

This is what I want:

The simplest solution is to do three passes:

Render opaque meshes to a buffer (front-to-back, depth read/write on)
Render translucent meshes to another buffer (front-to-back, depth read/write on)
This makes sure that only the closest translucent mesh is rendered.
Alpha blend translucent mesh on top of opaque buffer using opaque depth buffer. (depth read on)
This will take care of opaque meshes blocking translucent meshes and preserving translucent opacity levels. Depth test for values less or equal for small early z test (potential) optimisation.

EDIT: As mentioned below you could use the same depth buffer for the translucent pass and do the alpha blend then, the key is rendering the translucent pass front-to-back and depth write/testing.


How to keep an object constant in screen space? If you open the modern Blender UI and select for example scaling, and then you use the scroll wheel to "zoom" (move the camera along the radial axis). You will notice that the size of the 3D
widget remains constant in screen space (it becomes bigger or smaller in regards to all the other objects).

I am wondering how this can be achieved. I assume there is an obvious matrix cancellation you can do, e.g setting the w component to 1 after projecting. But I am not sure.

As you can see the widget's size in screen space is identical:

Scale the object proportional to its depth (z in camera space) and it will retain the same size on screen regardless of its position in world space.

Additionally, you might also wish to scale the object proportional to the field of view so that it retains the same screen size regardless of the camera zoom. (Specifically, scale it by tan(fov/2)).

Finally, you might also wish to scale it inversely proportional to the pixel size of the window if you want the widget to retain a constant size in pixels regardless of how large the window is. Otherwise it will be constant as a fraction of the window size.

I wouldn't recommend trying to do this in post-projective space (by manipulating w, etc) because it will scale the object's distance from the camera at the same time as scaling its size. Effectively you're scaling it in camera space. What you really want to do is scale it in its own local space before applying world-view-projection transformations.


Ray tracing: transforming ray to object's local space for intersection test Im writting a raytracer, for which I want to be able to test intersections with implicit surfaces (in my case spheres). Without applying any transformation it works great, but I would like to upgrade it to test intersections against non-uniform scaled spheres. For such purpose, I have computed the object's local to world matrix, as well as the world to local matrix.

During the intersection test, I first transform the ray origin and direction to the object local space:

    Vector o = ray.getOrigin();
    Vector l = ray.getDirection();

#ifdef _RT_TRANSFORM_RAY_TO_LOCAL_SPACE
    o = worldToLocalMatrix * o;
    l = worldToLocalMatrix * Vector(l.x, l.y, l.z, 0.0f);
    l = l.Normalize();
#endif

I perform the test, compute the intersection point in object's local space and then transform it back to world space:

Vector hitPoint(o + (l * distance));
        Vector hitNormal((hitPoint - center) / radius);
        hitNormal = hitNormal.Normalize();
#ifdef _RT_TRANSFORM_RAY_TO_LOCAL_SPACE
        hitPoint = localToWorldMatrix * hitPoint;
        hitNormal = localToWorldMatrix * Vector(hitNormal.x, hitNormal.y, hitNormal.z, 0.0f);
#endif
        hitNormal = hitNormal.Normalize();

If I run the raytracer, it works great:

However, from some points of view (not all), I obtain wrong renders from the dielectric sphere (center in the above image) and the perfect specular (right in the above image), as well as one of the cubes:

If I just apply the affine transformations to the object's vertices and normals during start up(in the case of the sphere, I only apply translation since its an implicit surface), the render works flawlessly, so my guess its that the problem comes from the transformations applied to the ray.

To compute the local to world matrix:
Model = rotation * translation * scale

To compute the world to local matrix:
InvModel = inverse(scale) * inverse(translation) * inverse(rotation)

I would like to know if I'm applying the transformation to the ray as well as to the output hit properly

I figured out the problem. It was not related to the ray / point transformation itself, but to a problem with the camera update after moving it (Up vector wasn't being computed properly).

I leave the answer in case someone have similar issues.


Does it matter whether I learn OpenGL or Direct3D? Are the differences between these two APIs minor implementation details that mean once I have learned one I can use it for everything? Or are there reasons for learning one rather than the other if I want to be able to use it in general without having to relearn another API in future? Is one or other more general?

In particular I would like to be able to write for any graphics card, so code is not restricted to only running on a particular manuafacturer's cards or a specific model. I'd also like to be able to write code that still works in the absence of a graphics card (albeit slower).

Is there a difference in how portable code will be across different platforms (operating systems/architectures)? I'm interested in the availability of other libraries that work with these, and whether one or the other leads to fewer licensing restrictions in its wider environment. Anything measurable that would make a difference to whether one of these could be the only one I learn without restricting me.

I don't think it matters much, which API you want to use when leaning to "program graphics". The more important things to learn are the typical concepts you use when working with 3D scenes. For example you want to learn how to write a simple (and later more sophisticated) Scene Graph. These concepts are much more important than the specific API method names you have to call.

Compare it when learning how to write programs. If you are quite good at writing programs in e.g. Java, you won't have that much trouble learning C++, Objective-C, Swift, or some other object-oriented language. Because it's the concepts and the thinking which you learn.

The choice between OpenGL, Direct3D and Metal is primarily the choice, which operating system you target. Direct3D is primarily available on Windows, Metal on OS X and iOS, and OpenGL is supported on most systems including OS X, iOS, Windows and Linux.

The graphics card probably has little to no influence on this decision as I don't know a card that supports only one or two. If you have no dedicated graphics card, then rendering in real time will be a problem soon. Although a Intel HD Graphics and the AMD companion can already do quite much.

So, choose your platform.

Disclaimer: As of now, I did neither use Direct3D nor Metal.


OpenGL task/mesh shader extension NV_mesh_shader not behaving as I expected I am trying to get up to speed with the NV_mesh_shader extension from NVidia (on my RTX2080). Samples are rare, so I tried to come up with my own small sample.

I am rendering a sphere (20 triangles) using an index buffer and a vertex buffer. I can render it the "usual" way using "vertex pulling" with two shader storage buffers, one for vertices, one for indices, (like I want to do it with a mesh shader) just fine.

When using a mesh shader (no task shader so far) I get the following problem: calling glDrawMeshTaskNV(0,1)with the mesh shader set to layout(local_size_x=20) in I only get a single triangle rendered. It looks like the mesh shader is not being invoked 20 times as I think it should?
If I call glDrawMeshTaskNV(0,20) and set layout(local_size_x=1) in it renders just fine with nothing else changed in the shader. I use gl_GlobalInvocationID.x to pull the proper vertices for a triangle.

Here's the complete shader for the non-working variant:

#version 450
#extension GL_NV_mesh_shader : require

#define GROUP_SIZE 20
// mesh shader setup
layout(local_size_x=GROUP_SIZE) in;

layout(max_vertices=3, max_primitives=1) out;
layout(triangles) out;

// vertex/index data setup
struct Vertex
{
    vec4 position;
    vec4 uv;
};

layout(std430, binding=1) readonly buffer vertex_data
{
    Vertex vertex[];
};

layout(std430, binding=2) readonly buffer index_data
{
    uint indices[];
};

uniform mat4 model;
uniform mat4 view;
uniform mat4 projection;

// mesh output
out gl_MeshPerVertexNV {
    vec4 gl_Position;
} gl_MeshVerticesNV[];

out uint gl_PrimitiveCountNV;
out uint gl_PrimitiveIndicesNV[];

void main()
{
    // get starting index
    const uint primitive_start_index = gl_GlobalInvocationID.x * 3;

    uint index_A = indices[primitive_start_index];
    uint index_B = indices[primitive_start_index+1];
    uint index_C = indices[primitive_start_index+2];

    // get vertices
    vec3 A = vertex[index_A].position.xyz;
    vec3 B = vertex[index_B].position.xyz;
    vec3 C = vertex[index_C].position.xyz;

    gl_MeshVerticesNV[0].gl_Position = projection * (view * (model * vec4(A, 1.0)));
    gl_MeshVerticesNV[1].gl_Position = projection * (view * (model * vec4(B, 1.0)));
    gl_MeshVerticesNV[2].gl_Position = projection * (view * (model * vec4(C, 1.0)));

    gl_PrimitiveIndicesNV[0] = 0;
    gl_PrimitiveIndicesNV[1] = 1;
    gl_PrimitiveIndicesNV[2] = 2;

    gl_PrimitiveCountNV += 1;
}

Keep in mind this is only a learning sample, I know this is maybe overkill for a mesh shader :)

Thanks for any hints.
pettersson

I managed to figure it out. A coworker told me that I was misinterpreting the documentation.
layout(max_vertices=3, max_primitives=1) out;
actually means max_vertices and max_primitives per workgroup, not per shader invocation. So I was always only outputting a single triangle, regardless of what I set local_size_x to.

Changing the values to proper numbers resulted in a proper rendering.


Render with camera perspective off-center So ordinarily we have this

But I'm after this!

Is it possible to do this? Ideally by changing the projection matrix?

The reasoning is kinda interesting - I am developing for mobile VR, and I have found that putting my phone (and surely many others) in headsets that are too wide for them makes its so that your horizontal perspective convergence point is in the wrong place.

Yes, you can use an off-axis projection matrix.
This is what I use in my code (note: I shift the centre upwards, not left as you do in your example.)

void camera_setAspectRatio(float aspect, float zNear, float zFar, bool offaxis)
{
        // create a projection matrix
        const float f = 1.0f / tanf(fovy_radians/2.0f);
        fovx_radians = 2.0f * atanf( aspect * tanf(fovy_radians/2.0f) );
        float* mout = proj.data;

        mout[0] = f / aspect;
        mout[1] = 0.0f;
        mout[2] = 0.0f;
        mout[3] = 0.0f;

        mout[4] = 0.0f;
        mout[5] = f;
        mout[6] = 0.0f;
        mout[7] = 0.0f;

        mout[8] = 0.0f;
        mout[9] = offaxis ? -0.25f : 0.0f; // off axis projection!
        mout[10] = (zFar+zNear) / (zNear-zFar);
        mout[11] = -1.0f;

        mout[12] = 0.0f;
        mout[13] = 0.0f;
        mout[14] = 2 * zFar * zNear /  (zNear-zFar);
        mout[15] = 0.0f;
}

Stereo Projection screens (not head-mounted) tend to use Off-Axis projection matrices as well. Paul Bourke has a great write up on these projections.


Is `groupshared` memory stored in L2 cache of GPU? The article says that L1 cache is shared by work items in the same work group(aka. SM) and L2 cache is shared by different work groups.

In Direct3D, it seems that a thread group (which is specified by numthread(x, y, z)) is mapped to multi work groups, and groupshared could be accessed by all threads of the thread group (which belong to different work groups).

So, is groupshared the L2 cache of GPU? If it is, could I manipulate L1 cache directly in Direct3D?

There seems to be some confusion of terminology here. In Direct3D, you have threads and thread groups. "work item" and "work group" are generally encountered in OpenCL terminology, where a "work item" would be what a thread is in Direct3D and a "work group" corresponds to a Direct3D thread group. groupshared memory is memory accessible to all threads that are part of the same thread group.

It is unclear what exactly you mean by "manipulating L1 cache directly". A cache is a hardware component that is used to transparently optimize memory access. The word "transparent" is crucial here. While, at the hardware level, caches necessarily contain some form of memory, they are generally not explicitly "accessible" as a region of storage. You can't have a pointer that points into a cache. You can have a pointer that points into memory. Memory accesses will typically go through a cache, i.e., any memory request will be handed off to the next cache, which can then check whether it can serve the request directly or has to go further down the memory hierarchy. There's more to a cache than just the memory it uses to cache stuff. A cache also contains additional logic for managing the cache on top of just the storage itself. Caches are implicit by nature. As you manipulate memory, you will typically also cause some caches to manipulate their memory along the way. But you generally can't "manipulate" a cache directly in the sense of explicitly defining what is when to be stored where in the cache. As soon as you can explicitly point into it and read/write from/to it, it kinda stops being a cache and starts being just some form of storage.

That being said, groupshared memory is generally located in fast on-chip storage just like caches are. The whole point of groupshared memory is to take advantage of the knowledge that certain threads reside on the same part of the chip to allow them to communicate faster. And on some hardware (e.g., most of the recent NVIDIA architectures), groupshared memory and the L1 cache will actually use the same physical memory. However, that just means that one part of that memory is used as "normal" memory, accessed directly via addressing through some load-store-unit, while another part is used by the L1 cache to cache stuff…


Modulo vs Scaling vs Capping when performing filters I'm currently taking a graphics course and we've recently covered cross-correlation with regards to using a filter matrix that is applied to a region of pixels in a sort of continual raster-type scan across a larger input image to produce some output image.

The formula is denoted here:

G is the output image, h is the kernel/box filter matrix, F is the input image

The classic one that was introduced is a 3x3 box filter for blurring an input image. The box filter is a 3x3 matrix with each cell value being equally set to 1/9. We've been dealing primarily with grayscale images with values 0 (black) to 255 (white).

In this case, the largest possible output pixel value would be if the 3x3 input image area contained all white pixels in which case we'd have:

[(1/9) * 255 ] * 9 = 255

This is fine, but what if our kernel filter matrix instead had 9 cell values all equal to 0.5?

Then the largest potential output value would be:

[0.5 * 255] * 9 = 1147.5

Once we've completely finished producing our new output image matrix, how do we handle values above 255 and below 0?

It seems like there are three possible approaches:

Capping

Cap all output values to 0 and 255. Any value below 0 is capped to 0, any value above 255 is capped to 255.

Modulo

Use modulo. In this case it would be 1147.5 % 255 = 125.5 rounded to 126.

Scaling

Take the highest output value and use it to scale all the values proportionally. If our example output image pixel values were 1147.5, 100, and 255, then we would perform the following operations:

(1147.5 / 1147.5) * 255 = 255
(100 / 1147.5)    * 255 = 22.2 = 22
(255 / 1147.5)    * 255 = 56.6 = 57

I just don't get which one of these I'm supposed to be using for common graphics operations. I have the same question when combining images together (such as multiply, divide, screen, etc).

When doing a filter operation such as a blur, in many cases the filter itself will be normalized so its values sum to 1.0, precisely to avoid this problem.

In cases where out-of-range outputs do need to be handled, the most common way is what you called capping. In graphics it's more commonly known as clamping or saturating. This is the default behavior if, for instance, you run a pixel shader on a GPU and output out-of-range values.

Renormalizing the output image might be necessary in some special cases depending on what you're doing. I can't think of anytime that I've seen the modulo behavior used—it would be a pretty weird operation to do on a visual image.


Get binding point of sampler/image in shader I have an image and a sampler declared in a shader as follow
layout(binding = 5) uniform isampler2D heightmap;
layout(binding = 3) uniform writeonly uimage2D info;

Which are correctly bound using the following code
glBindTextureUnit(5, heightmapId);
glBindImageTextures(3, 1, &infoId);

For introspection purposes, I am trying to retrieve the binding points of these objets, 5 and 3, but I didn't find any way to do this for uniforms. Is it possible using the API?
I first tried to use glGetProgramResourceIndex, since it worked with storage buffers, but it doesn't the results are get are incorrect, as if the explicit binding was ignored.
I then looked at glGetProgramResourceiv, but the binding related enums are only available for storage buffers and atomic counters.

The binding value for uniforms of opaque types is, from the perspective of the OpenGL API, the uniform's value. You set it by using glProgramUniform1i.
As such, you query it by querying the value of the uniform with glGetUniformiv.


How different can the number of pixels in a straight line be to its real length? I'm working on a game with a programmable robot. It uses a laser range finder to get the distance to walls.

I'm using DDA to generate a line from the robot, going in the direction it is pointing, then checking each pixel along the line to see if it hits anything. At that point, the number of pixels checked so far is used as the range, to save having to calculate the proper length.

If the line is horizontal or vertical, then there's no discrepancy, but how greatly does the number of pixels in a line differ from its real length in the worst case?

Notes:

The length of the line is never just a few pixels, say always at least 50px.
Longer lines obviously have larger discrepancies, but the question is really about relative differences.

The worst case is when the line is at a 45-degree angle; then its length (in pixel-sized units) is a factor of $\sqrt{2}$ times the number of pixels, or about 40% longer.

You can see this if you imagine starting with a vertical line of say 50px, then gradually sliding the endpoint away horizontally. Note that the number of pixels drawn doesn't change! You still have exactly one pixel turned on in each row that the line crosses. The pixels will adjust their horizontal positions but no new pixels will turn on, until the line surpasses 45 degrees from the vertical. The same is true if you start with a horizontal line and slide the endpoint vertically.

The number of pixels in the line effectively only looks at the vertical or horizontal distance, whichever is larger, and ignores the smaller dimension.
Mathematically, the number of pixels is $\max(|\Delta x|, |\Delta y|)$ while the Euclidean distance is $\sqrt{\Delta x^2 + \Delta y^2}$.


How to perform additive blending in Metal. What do these terms mean? Forgive me, I dont really understand the basis of alpha blending. Like I know the formula but when it comes to the terms blending operation, source blend factor, and destination blend factor I am absolutely lost.

The goal is to do additive blending like in photoshop so black items would be invisible and items with any color would add their color resulting in an effect that pixels with many layers underneath are close to white.

The problem is I have no clue how to translate this to metal. For example this is how I am currently doing alpha blending (although this currently isnt working for whatever reason). What should I be doing for additive blending?

descriptor.colorAttachments[0].blendingEnabled = YES;
descriptor.colorAttachments[0].rgbBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].alphaBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].sourceRGBBlendFactor = MTLBlendFactorSourceAlpha;
descriptor.colorAttachments[0].sourceAlphaBlendFactor = MTLBlendFactorSourceAlpha;
descriptor.colorAttachments[0].destinationRGBBlendFactor = MTLBlendFactorOneMinusSourceAlpha;
descriptor.colorAttachments[0].destinationAlphaBlendFactor = MTLBlendFactorOneMinusSourceAlpha;

The “blend factors” are the amount that the specified component is multiplied by while blending. “Destination” is the existing content that’s being blended “on top of”, while “source” is the new content that’s being blended “onto” it. When the blend operation is set to “add”, the final pixel value is calculated like this:
value = source * sourceBlendFactor + destination * destinationBlendFactor

…so a blend factor of “one” means the value is used unchanged (multiplied by 1), “[x]Alpha” means the value is multiplied by the alpha value of x (source or destination), and “oneMinus[x]Alpha” means the value is multiplied by (1 - the alpha value of x).
For additive blending, you most likely want the following:
Objective-C:
descriptor.colorAttachments[0].blendingEnabled = YES;
descriptor.colorAttachments[0].rgbBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].alphaBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].sourceRGBBlendFactor = MTLBlendFactorSourceAlpha;
descriptor.colorAttachments[0].sourceAlphaBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].destinationRGBBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].destinationAlphaBlendFactor = MTLBlendFactorOne;

Swift 5:
descriptor.colorAttachments.colorAttachments[0].isBlendingEnabled = true
descriptor.colorAttachments.colorAttachments[0].rgbBlendOperation = .add
descriptor.colorAttachments.colorAttachments[0].alphaBlendOperation = .add
descriptor.colorAttachments.colorAttachments[0].sourceRGBBlendFactor = .sourceAlpha
descriptor.colorAttachments.colorAttachments[0].sourceAlphaBlendFactor = .one
descriptor.colorAttachments.colorAttachments[0].destinationRGBBlendFactor = .one
descriptor.colorAttachments.colorAttachments[0].destinationAlphaBlendFactor = .one

There’s a really helpful tool for visualizing the different options here—it’s written for OpenGL, but the Metal stuff works the same way.


Bug: Overly Sharp Highlights in PBR I'm still working on my BRDF equations. Theoretically, all equations are correct, but the hilights are super sharp. I'm not sure if this is a result of  me not using Image-Based Lighting (I currently use just an env map with a LOD function). I'm using materials from here: http://freepbr.com/c/base-materials/ - I use Specular/Gloss but the materials use the fragment shader to convert it.

Material Frag Shader (Excerpt) (Specifically, the Metalness-Roughness shader):

float Roughness = baseSpecular.a;
if (uTexRoughEnabled == 1)
    Roughness = texture(texRoughness, UV).r;

float Metalness = texture(texSpecular, UV).r;

// 1 - Roughness becomes Glossiness
specular = vec4(mix(vec3(0.4), albedo.rgb, Metalness), 1-Roughness);
albedo = vec4(mix(albedo.rgb, vec3(0), Metalness), 1-Roughness);

Lighting Functions:

vec3 Light_F(in vec3 Specular, in float VH) {
    // Fresnel Schlick
    return Specular + (1-Specular) * pow(1-VH, 5.0f);
}

float Light_D(in float alpha, in float NH) {
    // GGX
    float alphaSqr = alpha*alpha;
    float denom = NH * NH * (alphaSqr - 1) + 1;

    return alphaSqr / (pi * denom * denom);
}

float Light_V( in float NL, in float NV, in float alpha ) {
    // Frostbite's GGX
    float Lambda_GGXV = NL * sqrt (( - NV * alpha + NV ) * NV + alpha );
    float Lambda_GGXL = NV * sqrt (( - NL * alpha + NL ) * NL + alpha );

    return 0.5f / ( Lambda_GGXV + Lambda_GGXL );
}

vec3 Light_Diffuse(in float NV, in float NL, in vec3 Normal, in float alpha, in vec3 Albedo, in vec3 eyeDir, in vec3 lightDir) {
        // Oren-Nayar
        float gamma = dot( eyeDir - Normal * NV, lightDir - Normal * NL);
        float A = 1.0f - 0.5f * (alpha / (alpha + 0.57f));
        float B = 0.45f * (alpha / (alpha + 0.09));
    float diffAlpha = max( acos( NV ), acos( NL ) );
    float diffBeta  = min( acos( NV ), acos( NL ) );
    float C = sin(diffAlpha) * tan(diffBeta);
    return Albedo * (A + B * max( 0.0f, gamma ) * C) / pi;
}

Final lighting function in deferred renderer:

vec3 LightCalc(in vec3 Albedo, in vec4 WorldPos, in vec4 Specular, in vec3 Normal, in samplerCube envMap, in vec3 lightPos, in float lightAtten, in vec4 lightColor) {
    vec3 lightDir   = WorldPos.xyz - lightPos;
    vec3 eyeDir = normalize(eyePos - WorldPos.xyz);
    vec3 eyeReflect = reflect(-eyeDir, Normal);

    float Distance  = length(lightDir);
    lightDir        = -normalize(lightDir);

    float Attenuation = clamp(1 - pow(Distance / lightAtten, 4), 0, 1);
    Attenuation = Attenuation*Attenuation/(Distance*Distance+1);

    vec3 AmbientColor = vec3(Albedo.xyz)*0.01;

    float Roughness = 1-Specular.a;
    float alpha = Roughness * Roughness;

    vec3 H = normalize(eyeDir + lightDir);

    float NL = clamp(dot(Normal, lightDir), 0, 1);
    float NH = clamp(dot(Normal, H), 0, 1);
    float NV = clamp(dot(Normal, eyeDir), 0, 1);
    float LH = clamp(dot(lightDir, H), 0, 1);
    float VH = clamp(dot(eyeDir, H), 0, 1);

    float lod = compute_lod(alpha, 1, NH);
    vec3 reflectDir = normalize(eyeReflect);
    vec3 reflectPix = textureLod(envMap, reflectDir, lod).rgb;

    float D = Light_D(alpha, NH);
    vec3 F = Light_F(clamp(Specular.rgb, 0.018, 0.95), LH);
    float Vis = Light_V(NL, NV, alpha);
    vec3 Spec = (D * F * Vis) * reflectPix;

    vec3 Diffuse = Light_Diffuse(NV, NL, Normal, alpha, Albedo.rgb, eyeDir, lightDir);

    vec3 lightModifier = lightColor.xyz*lightColor.w*Attenuation;
    return NL*(Spec + max(Diffuse, AmbientColor))*lightModifier;
}

Feel free to point out any mistakes I made, even if not related to the issue. I've tested the materials in Unreal, and they looked correct. I doubt this is solely due to the lack of Image-Based Lighting. I also doubt this is an issue with using point lighting. Additionally, if I turn down the roughness by a lot, it becomes too dark a high roughness to be the issue.

Just to comment on the 0.16*Specular^2 term that was mentioned in the comments by Karim:

Frostbite only remaps the specularity for their internal purpose so that they can pack gemstones specularity into the 8bit channel. They just lose some precision in favor of more variation. The specularity value is the original one when rendering.

Check the Disney BRDF viewer which is a awesome start for PBR, since it contains the whole BRDF from Disney.
BRDF Explorer | Walt Disney Animation Studios

Also, if you are looking at a better Fresnel function than

vec3 Light_F(in vec3 Specular, in float VH) {
    // Fresnel Schlick
    return Specular + (1-Specular) * pow(1-VH, 5.0f);
}

Use this instead since it takes roughness/gloss into account:

vec3 Light_F(in vec3 Specular, in float VH, in float gloss)
{
    return Specular + (max(vec3(gloss), Specular) - Specular) * pow(1-VH, 5.0f);
}