How to obtain generalized barycentric coordinates for n-sided polygon? Quadrilateral linear basis functions(generalized barycentric coordinates) are defined as:

What are the steps to calculate the above functions?

Is it possible to obtain basis functions for a 5 sided polygon(pentagon) or in general n-sided polygon?

Quadrilateral basis functions can be calculated using an outer product of the basis of two linear functions (1-r, r) and (1-s, s). The same thing does not apply for arbitrary $n$-sided domains. For this we use generalised barycentric coordinates. Wachspress coordinates are arguably the simplest generalised barycentric coordinates available, are fast to evaluate and are defined for arbitrary $n$.

For a point $p$ on a planar convex polygon defined by vertices $v_1, v_2, \ldots, v_n$, ordered counter-clockwise, you can find weights

$$w_i(p) = \frac{A(v_{i-1}, v_i, v_{i+1})}{A(v_{i-1}, v_i, p)A(v_{i}, v_{i+1}, p)}. $$

Here, $A(v_1, v_2, v_3)$ denotes the signed triangle area. The coordinate functions $\phi_i$ can then be found be normalising the weights $w_i(p)$

$$\phi_i(p) = \frac{w_i(p)}{\sum_{j=1}^n w_j(p)}.$$

Now, the coordinate functions correctly parametrise the point (linear reproduction property)

$$p = \sum_{i=1}^n v_i \phi_i(p).$$

At vertices the coordinates satisfy the Lagrange property $\phi_i(v_j) = \delta_{ij},$ where $\delta_{ij}$ is the Kronecker delta, meaning that the corresponding coordinate function evaluates to 1 at its vertex and linearly interpolate on the edges. It is also important to notice that the coordinate partition unity $\sum_{i=1}^n \phi_i(p) = 1.$

For, non-convex polygons you could use mean value coordinates.


What is the use of epsilon in the Möller–Trumbore intersection algorithm? The Möller–Trumbore intersection algorithm compares the determinant with an epsilon. What is this epsilon and how is its value chosen?

The epsilon is there to counter floating-point accuracy problems.  Note that the code checks that the algorithm is between $-\epsilon$ and $\epsilon$; that is, close to zero.  If the determinant is zero, then the ray lies in the plane of the triangle and can be safely said to not intersect it (as the alternative, an edge-on intersection, should be caught by neighbouring triangles in the mesh).

The Cramer's rule computation of the intersection point then divides by the determinant, and here's where the floating-point accuracy comes in.  If the determinant is merely very close to zero, not exactly equal to it, then its reciprocal will be very large.  We multiply the other determinants by this reciprocal, so small floating-point errors in their values will be blown up into significance and make the difference between an intersection and a non-intersection.

I haven't tested it, but I suspect that leaving out the epsilon will cause intersections on the silhouette of a mesh (i.e. with triangles close to parallel with the ray direction) to become very nondeterministic.  This would result in nearby rays either missing or hitting the triangles unpredictably and giving you speckling.

The value of the epsilon depends on the floating-point precision you're using. It should be small enough that only very grazing intersections are culled, but large enough that it avoids the speckling problem.  It can be determined by experiment, or by just eyeballing it.  The implementation you linked to uses $\epsilon = 10^{-7}$, which is a reasonable choice for single-precision arithmetic; you may want to use $10^{-9}$ or $10^{-10}$ for double-precision computations.


What are some new Real-Time Graphics Rendering Techniques to get closer to real life? So recently after implementing PBR in my engine, I was wondering what are some new & old  Rendering Techniques that people are implementing and researching to get Real Time Graphics closer to what things look like in real life? A list of things that come to my mind were:

Physically Based Rendering/Lighting

Dynamic Reflections

Shadow Mapping (also PCSS)

Ambient Occlusion (SSAO, HBAO)

Bloom & HDR

Depth of Field

Lens Flares

Motion Blur

Particle Effects (Fog, Fire)

What are some other techniques that are being used in very graphically demanding games like Battlefield 1, Star Wars: Battlefront, Assassins Creed Unity & other such games. The past few years PBR has been the biggest thing for the gaming industry, are there any new things like that 'on the rise'?

PBR isn't just a feature you "add to a rendering engine" but an entire philosophy how to approach solving rendering problems. This seems to be a common prevailing misconception when people talk about PBR. Quite often people assume that when they have implemented physically based BRDF (e.g. GGX) into their engines, they are "done implementing PBR". They are not - they are barely getting started.

Let's take some of the topics you list and how do you approach implementing them in the "physically based" way.

Depth of Field, Lens Flares, Motion Blur and Bloom & HDR for example. For years these effects have been implemented and slapped as post effect on the screen with some adhoc factors exposed to artists. However, as you know these are effects happening in the camera system and they should be exposed and implemented considering how the light interacts with this system. How camera aperture, shutter speed, ISO, camera sensor and lense system influence the effects and how they should be implemented and controlled considering these physical characteristics of the camera.

This same principle applies to all the aspects of rendering when you go PBR. For example shadows you mention, how would you implement shadows in physically based way? First of all you would have to consider all lights as area light sources as opposed to infinitesimally small lights that people have been doing for ages. Recently there has been push to implement physically based area lights in real-time which is an important step towards more complete PBR, but they are completely omitting proper area shadowing. This is a difficult problem in real-time rendering and there has been only little research done in the area of area shadows. "Real Time Area Lighting - Now and Next" by Sam Martin is a good thing to read if you are interested in the topic.

Talking about dynamic reflections or lighting in general. How should all the different lighting techniques such as SSLR, local/global environment maps, (textured) area lights, SSAO, GI, etc. be implemented and integrated together to form a cohesive lighting system? That you doesn't do double counting and that they are properly accounted in diffuse and specular lighting calculations.

When implementing real-time PBR algorithms, you will eventually have to introduce some bias due to performance constraints and resort to some kind of approximations. But the important thing is that you know what the real result should be and are aware of the compromises you are making. PBR is a paradigm shift from the classic artistic observation-based real-time rendering algorithm development towards more scientific way of solving the rendering problems and this shift in mentality and the direction it pushes you helps you to get more realistic results in the end.


Unlit objects in deferred rendering What is the best way to draw unlit objects with deferred? Like skybox, particles or just glowing objects?
I thought about mask in alpha channel, check it in light pass and just set diffuse color but it won't work with light blending.
My second idea was stencil buffer with pass after light pass that only fill unlit space. But switching pso (dx12) only for color copy doesn't sound like a good idea.
And of course pass only for unlit objects after light pass, but I don't want to create another list only for this (and again, i have to switch pso).

Typically in a deferred engine, you would render unlit objects directly to the color buffer, bypassing the lighting system entirely. You would likely also output zeroes to the G-buffer so that deferred lights don't add anything to those pixels (and possibly also mark them in the stencil buffer, to avoid running the lighting shaders on them, for efficiency).

Don't worry about switching PSOs for this. I'm guessing you're worried about the performance, but it's not a big deal. Game engines typically switch PSOs (or switch shaders, in DX11/GL) hundreds or thousands of times per frame. Switching a couple of times for an extra color rendering pass isn't even a blip on the radar.

However, if you want to really simplify your code (possibly at the expense of performance), you could just bind the color buffer together with your G-buffers and render to it all the time, as if it were an extra G-buffer. Ordinary deferred shaders would output zero to it, while unlit objects would output their color to it.


How does graphics api like OpenGL determine which triangle is back face to cull? When I was writing a software rasterizer for my university class, we did culling in  eye coordinates. Essentially, we use cross product to get a normal from the 3 vertices (in eye coordinates), then we dot product that normal with one of the 3 vertices (since we are in eye coordinate, that vertex is essentially a vector from eye [0, 0, 0,.1] to triangle surface). If the dot product is > 0, meaning the normal and the eye vector point in the same direction, we cull that triangle.

OpenGL however doesn't do culling in eye space, since culling is after vertex shader. So all they have are vertices in clip space. So how do they know which triangle to do culling? Our triangles, when fed into the graphic pipeline, are always in CCW order. How do some turn into CW order?

It's actually fairly simple (and in essence 'equivalent' to what you said above). If you represent triangle ABC's projected XY coordinates as a matrix...

$$M_{ABC} =   \begin{bmatrix}
    X_A & X_B & X_C \\
    Y_A & Y_B & Y_C \\
    1 & 1 & 1 \\
    \end{bmatrix}
$$

...then the sign of the determinant e.g. $X_A (Y_B - Y_C)+X_B(Y_C-Ya)+X_C(Y_A-Y_B)$ will allow you to decide how to cull.
Further if the determinant is zero, then the on-screen triangle has zero area and can clearly be removed.

Another reason to use this notation is that the adjoint of $M_{ABC}$  gives you equations for the edges.


Role of PDF of Uniform Random Sampling in a path tracer I am trying to implement my own path tracer but before arriving to the question I want to give you a short overview:
In the implementation of the rendering equation I use some particular technique in order to sample surfaces. For example: when one of my rays hits a diffuse surface, the next ray bouncing from that surface will be calculated using a Cosine-weighted Random Direction. This implies that in my rendering equation I have to take into account the PDF that this specific random distribution implies and specifically divide for this PDF in my equation. So far everything is okey.

Now, my question. I want to implement a particular technique called "next event estimation" which simply samples the lights. In order to do so, I want to pick a random point on my light, which is spherical, by using the following code (C++):

Vec3<float> randomPoint() const
{
    float x;
    float y;
    float z;

        // random vector in unit sphere
    do
    {
        x = ((float)rand() / (RAND_MAX)) * 2 - 1;           // random coordinates between -1 and 1
        y = ((float)rand() / (RAND_MAX)) * 2 - 1;
        z = ((float)rand() / (RAND_MAX)) * 2 - 1;
    } while (pow(x, 2) + pow(y, 2) + pow(z, 2) > 1);        // simple rejection sampling

    return centerOfSphere + Vec3<float>(x, y, z) * radius;
}

As far as I understand (please correct me if wrong), this implements a uniformly random sampling. Also, as far as I have read, this kind of sampling has a PDF = 1 / (b - a) . My question: do I have to use this PDF as well like I do for my Cosine-weighted Random Distribution or not, since it is a uniform distribution? if yes, what's the range (b - a) that PDF talks about? Thanks in advance!

Firstly, as @trichoplax correctly pointed out, your randomPoint function calculates a point in a cube, then uses rejection sampling to return all points that are inside a unit sphere. In order to return points on a sphere, you would need to change the greater than to an equals. That said, rejection sampling is very inefficient.

A better way to sample a sphere, is to sample in spherical space, then transform to cartesian. ie:

float phi = PI * randf();
float theta = 2 * PI * randf();

float x = radius * std::sinf(phi) * std::sinf(theta);
float y = radius * std::cosf(phi);
float z = radius * std::sinf(phi) * std::cosf(theta);

However, this is not uniform, and will cause samples to clump at the poles. To prevent this, we transform phi:

float phi = std::acosf(2.0f * randf() - 1.0f);
float theta = 2 * PI * randf();

We can use some trig identities to make it a bit more efficient to calculate with the computer:
$$\begin{align*}
\sin(\cos^{-1}(x)) \equiv& \ \sqrt{1 - x^2}\\
\cos(\cos^{-1}(x)) \equiv& \ x\\
\\
\phi =& \ \cos^{-1}(2\xi - 1)\\
\sin(\phi) =& \ sin(\cos^{-1}(2 \xi - 1))\\
=& \ \sqrt{1 - (2 \xi - 1)^2}\\
\cos(\phi) =& \ 2 \xi - 1
\end{align*}$$

So the full change is:

float cosPhi = 2.0f * randf() - 1.0f;
float sinPhi = std::sqrt(1.0f - cosPhi * cosPhi);
float theta = 2 * PI * randf();

float x = radius * sinPhi * std::sinf(theta);
float y = radius * cosPhi;
float z = radius * sinPhi * std::cosf(theta);

However, in the case of next event estimation, uniform sampling the whole sphere is inefficient, because a ray can only 'see' half of the sphere at a time. So if we generate a point on the 'back' of the sphere, the sphere will occlude the point, and your calculation will be wasted.

Instead, you generate samples in a cone, which covers the great circle of the sphere, as viewed from the starting point of the ray.

// Sample sphere uniformly inside subtended cone
float rand1 = randf();
float rand2 = randf();

// Compute theta and phi values for sample in cone
float distanceSquared = DistanceSquared(rayOrigin, sphereCenter);

// We use geometry to calculate the angle of the cone (aka, the maximum phi can be when we sample)
// It's easier / cheaper to use geometry to calculate sin/cos phi directly, rather than generating phi and using sin/cos
float sinPhiMaxSquared = radius * radius / distanceSquared;
float cosPhiMax = std::sqrt(1.0f - sinPhiMaxSquared);
float cosPhi = (1.0f - rand1) + rand1 * cosPhiMax;
float sinPhi = std::sqrt(1.0f - cosPhi * cosPhi);

// Phi can be anything in 2 PI
float theta = 2 * PI * rand2;

float x = radius * sinPhi * std::sinf(theta);
float y = radius * cosPhi;
float z = radius * sinPhi * std::cosf(theta);

NOTE: x, y, z are in LOCAL coordinate space.

So we need to transform to world coordinates:

float3 zAxis = normalize(sphereCenter - rayOrigin);
float3 xAxis;
if (std::abs(zAxis.x) > std::abs(zAxis.y))
    xAxis = float3(-zAxis.z, 0.0f, zAxis.x) / std::sqrt(zAxis.x * zAxis.x + zAxis.z * zAxis.z);
else
    xAxis = float3(0.0f, zAxis.z, -zAxis.y) / std::sqrt(zAxis.y * zAxis.y + zAxis.z * zAxis.z);
yAxis = cross(zAxis, xAxis);

float3 samplePoint = x * xAxis + y * yAxis + z * zAxis;

And the pdf is calculated as follows:

float pdf = 1 / (2 * PI * (1 - cosPhiMax));

The code here is heavily influenced / copied from PBRT v3. They have a series of classes and functions for sampling from shapes.

Finally, the pdf. In monte-carlo integration, we need to combine the pdf's of each integration we do. In path tracing, we can integrate over many many things. For example, the general rendering equation integrates the incoming light over the hemisphere, depth of field can be treated as an integration over a focal distance, etc.

For next event estimation, you explicitly split the rendering equation into two integrands, direct lighting, and indirect lighting.

Standard rendering equation:
$$ L_{\text{o}}(p, \omega_{\text{o}}) =  L_{e}(p, \omega_{\text{o}}) \ + \  \int_{\Omega} f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}} $$

Next Event Estimation:
$$ L_{\text{o}}(p, \omega_{\text{o}}) =  L_{e}(p, \omega_{\text{o}}) \ + \ \int_{\Omega} f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i, direct}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}}\ \ + \ \int_{\Omega} f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i, indirect}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}} $$

In simple naive forward path tracing, everything is is treated as indirect light. In next event estimation, we directly calculate the direct lighting and add it to the indirect lighting. And if we hit a light, we ignore the contribution, since we're calculating the direct lighting.

Since we have two integrations, each will have its own pdf. Aka:
$$L_{\text{o}}(p, \omega_{\text{o}}) =  L_{e}(p, \omega_{\text{o}}) \ \ + \ \sum_{k=0}^{\infty } \frac{f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i, direct}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | }{pdf_{direct}}\ \ + \ \sum_{k=0}^{\infty } \frac{f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i, indirect}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | }{pdf_{indirect}}$$

If you want to see how this is implemented, you can check out my implementation here. Note: my light sampling is a bit more complicated, since it does multiple importance sampling. But it can be as simple as:

float3 Renderer::EstimateDirect(Light *light, UniformSampler *sampler, float3a &surfacePos, float3a &surfaceNormal, float3a &wo, Material *material) const {
    float pdf;
    float3 wi;

    // Sample a point on the light and get the pdf
    float3 Li = light->SampleLi(sampler, m_scene, surfacePos, surfaceNormal, &wi, &pdf);

    // Calculate the brdf value
    float f = material->Eval(wi, wo, surfaceNormal);
    return f * Li * / pdf;
}


Path weight for direct light sampling I originally wrote a pathtracer that just bounces by the BRDF and each sample is added to a buffer that divides by the current number of frames.

Now I'd like to do next event estimation at each bounce, directly sampling the light along the path. How do I "weight" the samples taken with regular pathtracing along with the direct light samples? Even if I weight the direct light by the solid angle of the light, if I have more than 1 sample taken I can no longer just divide by the frame count right?

By "bounces by the BRDF", I assume you mean picking random directions in the hemisphere and weighting by the BRDF, then averaging over those samples. I guess you're accumulating one sample per frame, so then you divide by the current number of frames.

To be theoretically "correct", you should also be weighting those samples by $2\pi$, as that's the solid angle of the hemisphere you're integrating over. That's not too important when you just have a single sampling method, but when you're going to combine multiple ones it's easier to be sure everything's consistent if you have these factors right.

Then, to incorporate explicit direct light sampling, you have to convert from the light-sampling probability density to the solid angle probability density at the receiver point. If the light samples are uniform over the light source's area, then in place of the simple $2\pi$ weight, you'd use this weight factor instead:
$$\frac{A}{r^2} (N_\text{light} \cdot -L)$$
where $A$ is the area of the light source, $r$ is the distance between the receiver point (current path point) and the chosen light source point, $N_\text{light}$ is the normal of the light point, and $L$ is the unit vector from the path point, toward the light point.

This factor accounts for how dense the sampled light source points will be in solid angle of the receiver point, varying both with distance and with the angle on the light source's surface (the points will bunch up in areas where the source surface slopes away from the receiver, such as around the edges of a spherical area light).

After weighting by this factor, the direct light samples are in the same solid angle domain as the BRDF samples, so you can just add them together and divide by the total number of samples—which would now be twice the number of frames, if you're doing one BRDF sample and one light sample per frame.

Also, an important thing to get right is not double-counting the light's illumination. Since you're explicitly sampling the lights now, you have to make sure not to include the emissive color if you do happen to randomly hit a light while doing the BRDF samples! Otherwise you'd end up with the lights appearing twice as bright as they should.

A refinement of this is that you may wish to apply explicit light sampling only to the diffuse part of the BRDF, and not the specular part. A narrow specular highlight is better handled through importance-sampling the BRDF instead. This implies that when doing explicit light samples, you would only evaluate the diffuse component of the BRDF, and then if you hit a light source through a specular reflection ray, you would include its emissive color there.


Procedural terrain: What's the best approach to calculate noise in the GPU? A few years ago I started creating a procedural planet engine/renderer for a game in Unity, which after a couple of years I had to stop developing due to lack of time. At the time i didn't know too much about shaders so I did everything on the CPU. Now that I have plenty of time and am more aware of what shaders can do I'd like to resume development with the GPU in mind.

For the terrain mesh I'm using a cubed-sphere and chunked LODs. Since terrain never changes, heights for each chunk need to be calculated just once.The way I calculate heights is rather complex since it's based on a noise tree, which is created from an artist/designer-provided XML for each main planet/asteroid. Leaf nodes in the tree would be noise generators like Simplex, Value, Sine, Voronoi, etc. while branch nodes would be filters and modifiers (FBM, abs, neg, sum, ridged, billow, blender, selectors, etc).

To calculate the heights for a mesh you'd call void CalcHeights( Vector3[] meshVertices, out float[] heights ) on the noise's root node, somewhere in the game code.

This approach offers a lot of flexibility but also introduces a lot of load in the CPU. The first obvious thing to do would be (I guess) to move all generators to the GPU via compute shaders, then do the same for the rest of the filters. But, depending on the complexity of the noise tree, a single call to CalcHeights could potentially cause dozens of calls back and forth between the CPU and GPU, which I'm not sure it's a good thing.

How should I go about this?

r a similar problem (a tree of combined noise functions, evaluated on the GPU), I found a good method is to generate a shader from the expression tree.  Each predefined node corresponds to a single shader function, e.g.

float simplexNode(vec3 pos) {
  // ... implementation of simplex noise
}

or

float sumNode(float val1, float val2) {
  return val1 + val2;  // ... possibly clipped or whatever
}

You'd then create a shader like

#version 300 core
// ... include node definitions here
in vec3 vertexPos;
out float vertexHeight;
void main() {
  float f1 = simplexNode(vertexPos);
  float f2 = worleyNode(vertexPos);
  float f3 = sumNode(f1,f2);
  // .. a bunch more
  vertexHeight = f76;
}

Constant parameters could either be uniforms, or just pasted straight into the code.

One limitation of this method is that the noise functions and methods have to be local, i.e. the evaluation can be done pointwise.  In my case, some nodes were blends and convolutions, so I had to evaluate pieces of the expression tree into textures, blend or convolve the textures into new textures, then use these textures as leaves for the rest of the expression tree.  All of the combinators you mentioned seem to be local, so this may not be a problem for you.


Does glTextureStorageSubImage2D exist? The OpenGL SuperBible mentions glTextureStorageSubImage2D; however, a Google search currently shows no other mention of this function and I can not find any prototype or signature anywhere.

Does this call actually exist and if so, what are its parameters?

There is no such function. Given the context of the text, it is almost certainly referring to glTextureSubImage*.


Latency difference between texture copy on GPU vs CPU? Does anyone happen to know the latency difference between copying a texture on cpu (using, say, memcpy) compared to gpu (using, say, CopyResource)? I would do a perf test myself, but I'm not actually sure how to do the trace for the CopyResource. A rule of thumb would be fine, it doesn't have to be exact, I just figure that the GPU is faster than the cpu for this but am just curious how expensive a copy of, say, a 1920x1080 texture is.

You are talking apples and oranges here. Copying memory to memory on the CPU is using completely different hardware then memory to memory copies on the GPU, even in a unified memory architecture GPU's tend to have specialized memory.
The actual difference is going to depend heavily on the model of the GPU, a high end gddr5 gpu will easily beat DDR4 dual channel memory on the CPU.
Also, the driver implementation is going to have an effect on the gpu copy since you are really asking the driver for the copy on the gpu, which it then translates into code that does the actual copy.
The size of the chunk of memory also plays a major role in how fast the copy happens, as well as how large the cache on the cpu vs gpu is.
You should generally expect copies that are GPU to GPU to be much faster then CPU to CPU copies. CPU to GPU along with GPU to CPU copies have to cross the AGP bus on the PC while unified architecture systems have an advantage they still have synchronization issues and all systems have to synchronize access and flush cache's.
What are the actual differences?(very roughly) DDR4 runs from 1600 to 3200 MB per second and DDR5 run's at roughly 3200 to 6400 MB per second. So very roughly, a poorly balanced system will have up to 4x performance diffence, but you should expect about a 2x performance differnce with the GPU being faster. This difference fades somewhat(mostly) with unified systems. (But the gpu will probably still have memory set aside so it can work with framebuffers and what not in an efficient manner)


Is it possible to access adjacent vertex data in tessellation evaluation shader? I'm trying to render thick cubic Bezier curves using the whole OpenGL 4.5 pipeline:

I render a patch with 4 control points.
Vertex shader is simply pass-through to tessellation control shader.
TCS - sendng 4 vertices to tessalator with gl_TessLevelOuter[1] = 10
TES - calculating position for every of those 10 tessellated isoline points using Bezier curve equation
Geometry shader - making quads out of every line it receives extruding vertices along line's normal vector.

So far so good but this will obviously lead to cracks between adjacent quads. To solve this problem I would need to know the position of adjacent lines in TES or GS to correct edges. Is this information available?

If you want to prevent cracks between the tessellated lines in a single curve, what you need to know is how to miter the quads correctly. Your GS's extrusion is presumably based on taking the 2D perpendicular to the line direction and extending the quad in that direction. Well, don't do that.

Instead, have the TES compute the perpendicular direction for each vertex in the tessellation. You can compute this by taking the first derivative of the Bezier curve (its tangent), which can be computed exactly, and then finding the perpendicular of that. Each vertex will have its own perpendicular value.

It should be this perpendicular which you feed to the GS to do the quad extrusion. And you should be guaranteed to have invariant results. The TES's computation of the position and perpendicular will be based solely on the positions of the control points and the tessellation coordinates. The control points are fixed for all invocations of a TES over a patch, and the tessellator guarantees the invariance of gl_TessCoord for the "same vertex".

Invariance between patches will be something of a problem. That is, invariance between the end of one set of 4 control points and the beginning of an attached one. The tessellation system only guarantees invariance at edges between patches if the computations that lead to the positions of the vertices are all based solely on data that is binary identical between the two edge vertices. And that's not generally the case for bezier curves; you achieve C1 continuity by ensuring that the slope between boundary control points is the same. And that slope is not passed as a parameter; it is computed. And thus, two edge vertices on different patches could have slightly different slope computations, which leads to different vertices, which leads to invariance.

The only way to handle this is to have your Bezier spline defined by six control points rather than four. You pass as the first and last control points the previous/next control points for the Bezier spline (you'll have to make something up at the edges). This allows your computations to be invariant, since at the edges, you will be doing computations based on the exact same data.


How to texturize a 3D model from video? I have a base 3D mesh of an object that I want to texturize. I also have a 360 degrees video of the real-world object.

What are good ways to use the video to texturize the mesh? Are there existing tools to do that? (paid or free)

Edit: I have complete control over the camera and object positions, I can measure the distances and whatever needed. I can also put markers on the object.

One method is to do projection mapping. I understand that most 3D rendering applications can do that. I don't have much experience with them, but I have done it using AfterEffects using the Camera Mapper in Buena Depth Cue.

Algorithmically, it's achieved by projecting the geometry back onto the image to generate texture coordinates. So one way to do that is as follows:

Line up the image of the scene in front of the corresponding geometry
For each triangle in your geometry, project the 3 corners onto the image
The coordinates where the projection hits your image are the texture coordinates you need for that triangle


Are there common materials that aren't represented well by RGB? In graphics we use RGB and other color spaces as an approximation to the full spectrum of light wavelengths. This evidently works pretty well in general, but are there any reasonably common objects/materials/phenomena, things you might encounter in your everyday life, whose appearance isn't represented well by RGB rendering due to having a complex emission/reflection/absorption spectrum?

While the current answers are focusing mainly on colors outside a given RGB gamut, I'm also interested in hearing if there are examples where, for instance, the color of an object appears "wrong" when rendered in RGB because of an interaction between the light source spectrum and the object's reflection spectrum. In other words, a case where a spectral renderer would give you more correct results.

Credit: I liked this question in the previous private beta so I'm reproducing it here. It was originally asked by Nathan Reed

There are various different types of limitation to take into consideration.

Effects for which the path of a ray is dependent on its wavelength

These are a class of effects for which spectral rendering is required, and a number of interesting examples have already been given in Benedikt Bitterli's answer. A simple example is a prism splitting white light into a spectrum, giving rainbow colours. Rays of different wavelength are refracted by different angles as they pass through the prism, resulting in the light striking the wall behind the prism being split into its constituent colours.

This means that in real life, shining monochromatic yellow light through a prism will result in yellow light coming out, but shining a mixture of red and green light that approximates yellow will result in separate red and green light emerging. When rendering using only 3 primary colours, white light will split into only those three colours, giving rainbow effects that look discontinuous, and monochromatic light that should not split at all will split into its approximating primary colour components. The splitting of white light can be improved by using a larger number of primary colours, but this will still give discontinuities close up, and the results for monochromatic light will still be split, albeit more narrowly. For accurate results a continuous spectrum must be sampled, with wavelengths distributed according to the light source being simulated.

Surface effects that cannot be captured in a single still image

Iridescence, for example, shows a different colour to each eye so that a still image will not look the same as the original object. There are many everyday examples that you might not notice at first. Many common birds have iridescent feathers even though they appear black or grey from a distance. Close up they are surprisingly colourful.

A renderer using only 3 primary colours will not be able to produce the spreading of light based on wavelength required for this effect. A spectral renderer can simulate the spreading correctly, but the full effect still cannot be captured in a single image. Even a 2d photograph cannot capture this correctly, whereas a 3d photograph of an iridescent object will give that shimmering effect as the photographs corresponding to left and right eyes will be coloured differently. This is a limitation of 2d images rather than the RGB colour space itself. However, even in a 3d image there will be colours in the iridescent object that are not displayed correctly, due to the inability of RGB to display monochromatic colours as described below.

Colours that the human eye can detect that cannot be displayed in RGB

RGB was historically device dependent and therefore unreliable between platforms. There are device independent perceptually uniform improvements such as the colour space Lab, but these are still trichromatic (having 3 components). It is not immediately obvious why three components is insufficient to display all the colours that can be perceived by a trichromatic eye, but this paper explains it well, and accessibly. From page 7:

  For example, using a modern laser-display system with monochromatic primaries at 635 nm (red), 532 nm (green), and 447 nm (blue), lets see if we can simulate the perception of a monochromatic light at 580 nm (an orange color). Since the monochromatic orange stimulus excites the greenish and reddish cones, a contribution is required by both the green and red primaries, while no contribution is required from the blue primary. The problem is that the green primary also excites the bluish cones, making it impossible to exactly replicate the orange stimulus

The diagram of human eye cones sensitivities (also on page 7) shows how wide the overlap is and helps to visualise this explanation. I've included a similar graph from Wikipedia here: (click on the graph for the Wikipedia location)

In short, the overlap between the range of colours that can be picked up by each of the three different cones (colour sensors) of the human eye means that a monochromatic colour can be distinguished from an approximating mixture of primary colours, and therefore mixing primary colours can never accurately display all monochromatic colours.

This difference is not usually noticeable in everyday life as most of our surroundings emit or reflect light across a wide range of frequencies rather than single monochromatic colours. However, a notable exception is sodium lamps. If you live in a part of the world that uses these yellow-orange street lights, the light emitted is monochromatic and will look subtly different from a printed photograph or an image on a screen. The wavelength of sodium light happens to be the 580 nm from the example quoted above. If you don't live somewhere that has sodium street lights, you can see the same single wavelength light by sprinkling finely crushed table salt (sodium chloride) over a flame. The scintillating yellow points of light cannot be accurately captured on film or displayed on a screen. Whatever three primary colours you choose, there will always be a range of monochromatic colours that cannot be displayed.

Note that this limitation applies equally to mixing 3 primary colours of paint, using 3 photoreactive chemicals on a camera film, or taking a photograph with a digital camera with 3 different colour sensors, or a single sensor with 3 different primary colour filters. It isn't just a digital problem, and isn't just restricted to the RGB colour space. Even the improvements introduced by the Lab colour space and its variants cannot recover the missing colours.

Miscellaneous effects

Multiple diffuse reflections (colour bleeding)

If a brightly coloured matt surface is near a white matt surface, the white surface will show some of the colour of the other surface. This can be modeled reasonably well using purely red, green and blue components. The same combination of red, green and blue that gave the colour of the coloured surface can reflect off the white surface and show some of that colour again. However, this only works if the second surface is white. If the second surface is also coloured, then the colour bleeding will be inaccurate, in some cases drastically.

Imagine two surfaces that look a similar colour. One reflects a narrow range of wavelengths around yellow. The other reflects a wide range of wavelengths between red and green, and as a result also looks yellow. In real life, the light showing on one surface due to the other will not be symmetrical. Most of the light reaching the wide wavelength range surface from the other will be reflected again, as the narrow range of incoming wavelengths are all within the wider range. However, most of the light reaching the narrow wavelength range surface from the other will be outside the narrow range, and will not be reflected. In an RGB renderer, both surfaces will be modelled as a mixture of monochromatic red and monochromatic green, giving no difference in reflected light.

This is an extreme example where the difference will be instantly noticeable to the eye, but there will be at least a subtle difference in most images that include colour bleeding.

Materials that absorb one wavelength and emit another

joojaa's answer describes the absorption of ultraviolet light by snow, to be re-emitted as visible light. I hadn't heard of this happening with snow before (and frustratingly I've been unable to find any evidence to support it - although it would explain why snow is "whiter than white"). However, there is plenty of evidence of it happening with a wide range of other materials, some of which are added to clothes washing detergents and paper, to give extra bright whites. This allows the total visible light outgoing from a surface to be more than the total visible light received by that surface, which again is not modelled well using only RGB. If you want to read more about it, the term to search for is Fluorescence.

Eyes with more than 3 primary colours

There are animals that have more than 3 types of cones in their eyes, allowing them to perceive more than 3 primary colours. For example, many birds, insects and fish are tetrachromats, perceiving four primary colours. Some are even pentachromats, perceiving five. The range of colours that such creatures can see dwarfs the range displayable using only RGB. Far beyond them is the mantis shrimp, which is a dodecachromat, seeing colours based on 12 different cones. None of these animals would be satisfied by an RGB display.

But more seriously, even for images intended for human eyes, there are believed to be human tetrachromats who see in 4 primary colours, and possibly some who see as many as 5 or 6. At present, such people don't seem to be present in sufficient numbers to make displays with more than 3 primary colours commercially viable, but if in future it becomes easier to identify how many primary colours a person can see, this may become an attractive trait leading to it spreading throughout the population in future generations. So if you want your great grandchildren to appreciate your work you may need to make it compatible with a hexachromatic monitor...

Not really relevant to this question, but related: If you want to see colours that are not available in either the real world or RGB images, have a look at Chimerical Colours...


How to properly combine the diffuse and specular terms? As far as I understand, in a BRDF the Fresnel term is telling us the probability for a photon to be reflected or refracted when it hits a surface.

The reflected photons will contribute to the specular term, while the refracted ones will contribute to the diffuse term. Therefore when determining, in a physically based manner, the contribution of a light to the color of material, I feel tempted to just write:

// Assuming for example:
//   diffuse = dot(L, N);
//   specular = pow(dot(H, N), alpha) * (alpha + 2.0) / 8.0;
//   fresnel = f0 + (1.0 - f0) * pow(1.0 - dot(E, H), 5.0);
color = lightIntensity * Lerp(diffuse * albedo, specular, fresnel);

Yet, I don't think I've ever seen it written this way. I have seen the specular term being weighted according to the Fresnel term, but not the diffuse one. In his largely referenced article on PBR, Sébastien Lagarde even states that using $(1 - F)$ to weight the diffuse term is incorrect.

What am I missing?
I would much welcome an explanation that highlights in an obvious manner why this would be incorrect.

Using two Fresnel terms is correct in the sense that any given diffuse path will pass through the surface twice. If you're solving diffusion by tracing a path through the medium until it bounces out again then that you will get two (or more) Fresnel terms for that path as it interacts with the surface.

However, that's not what you're doing with a diffuse BRDF. A diffuse BRDF is intended to represent the average of all those possible diffusion paths. In the case of an Lambertian that average is modeled as uniform reflection and a single albedo value measuring the internal energy loss during diffusion, but more complicated models are possible. Crucially: a diffuse BRDF will already include the aggregate effect of some paths being reflected back into the medium to diffuse further and some passing out immediately. $1 - F_{out}$ is already "baked in" to the BRDF¹ and you do not need factor it in again.

What the Lambertian term does not include is the portion of energy that is lost by being reflected before light enters the medium of diffusion. This is view-dependent, but depends on the precise glossy lobe above it. There is no energy loss at a (non-metallic) surface interface so everything that isn't reflected will be refracted, meaning that what you actually want is to integrate the total energy loss at the surface over all outgoing directions, i.e. $1 - \int \texttt{glossy_bsdf}(\texttt{in}, \texttt{out}) \: \mathrm{d} \texttt{out}$.

It's possible to pre-compute approximations to that integral for specific BRDFs. The end result will in general depend on view direction, material roughness and IOR at least. As a first approximation you can assume that the glossy lobe is a perfectly specular reflector. That gives a weighting of $1 - \int \texttt{glossy} \: \mathrm{d}\texttt{out} = 1 - F_{in}$, which is exactly what you first suggested.

Additionally, note that the Lambertian BRDF is the albedo divided by $\pi$ and that the cosine term is a measure of how attenuated the incoming light is on the surface; it applies to both glossy and diffuse reflectance.

So, roughly:

// Assuming for example:
//   diffuse = albedo / PI;
//   specular = my_favorite_glossy_brdf(in_dir, out_dir, roughness);
//   fresnel = f0 + (1.0 - f0) * pow(1.0 - dot(E, H), 5.0);
//   total_surface_reflection = fresnel
color = lightIntensity * dot(L, N) * Lerp(diffuse, specular, total_surface_reflection);

¹ Really the integral of $F$ over all possible incident internal diffusion exit paths that result in your out direction, but I digress.


Tessellation Controll Shader how to get maximum subdivisions limit? I have a low-poly mesh and I want to add structure on the surface.
The setup:
The mesh consists of triangles.
Within the TCS the pixel-length for each of the three edges will be calculated and subdivided according to the given allowed u_MAX_LENGTH_TRIANGLE_EDGE. This will be done in the following way:
vec4 vertexPosScreen[3];
for (int i = 0; i < 3; i++)
{
    vertexPosScreen[i] = vec4(mvp * gl_in[i].gl_Position);
    vertexPosScreen[i] = vertexPosScreen[i] / vertexPosScreen[i].w;
    vertexPosScreen[i].x *= u_viewportWidth;
    vertexPosScreen[i].y *= u_viewportHeight;
}
float edgeLength[3];
edgeLength[2] = length(vertexPosScreen[0].xy - vertexPosScreen[1].xy);
edgeLength[1] = length(vertexPosScreen[0].xy - vertexPosScreen[2].xy);
edgeLength[0] = length(vertexPosScreen[1].xy - vertexPosScreen[2].xy);

float edgeLod[3];
for (int i = 0; i < 3; ++i)
{
    edgeLod[i] = max(1.0, edgeLength[i] / u_MAX_LENGTH_TRIANGLE_EDGE;
}

gl_TessLevelOuter[0] = edgeLod[0];
gl_TessLevelOuter[1] = edgeLod[1];
gl_TessLevelOuter[2] = edgeLod[2];

gl_TessLevelInner[0] = max(edgeLod[0], max(edgeLod[1], edgeLod[2]));

This guarantees, that in the final image all triangles edges are smaller than u_MAX_LENGTH_TRIANGLE_EDGE.
But:
The GPU is not able to subdivide as mush as wanted. So if we are too close to a triangle, or a triangle is too tall at all, the TCS will subdivide it as mush as possible, but not enough.
My Idea:
After tessellation is done, the new generated triangles should be stored into a new VBO during the TES step. Afterwards the fresh stored triangles in the new VBO will be rendered again and subdivided as well until all of them are small enough.
Now the big question
How are the tessellation limits defined? per edge division? And how can I ask for the maximum division on CPU? glGetInteger? Or is it limited during the number of generated vertices?
I think the easiest solution would be check directly after "edgeLod" is calculated, if edgeLod is too high. If this is the case, write something into a SSBO and loop the shaderprogram again.

The tessellation limit is GPU dependent..Any number above the GPU limit is technically an error but most gpu's just clamp to the upper limit. To get beyond the limit either use a Level Of Detail (LOD) approach where you have different models that are used based on distance, or save and recompute like you suggest.
To get the max just set all the edges to the limit AND set all the internals to the limit in the tessellation control shader:
#version 460 core

  layout(vertices = 4) out;

void main(void)
{
   gl_TessLevelOuter[0] = 64.0;
   gl_TessLevelOuter[1] = 64.0;
   gl_TessLevelOuter[2] = 64.0;
   gl_TessLevelOuter[3] = 64.0;

   gl_TessLevelInner[0] = 64.0;
   gl_TessLevelInner[1] = 64.0;

   gl_out[gl_InvocationID].gl_Position = gl_in[gl_InvocationID].gl_Position;
}

And how can I ask for the maximum division on CPU?

Yes use glGetInteger. At the bottom of this page is all the values you would be intrested in: Khoronos Tessellation limits, search for tessellation on the page


It is common for modeling software to compute the normals when they aren't included in the loaded file (regardless of the format). This is usually highly dependent on the settings chosen when the file is loaded. Vertex normals can be computed fairly easily. The software can also compute smooth or faceted normals so you can switch between the two.
Your example includes faces...the f 3 2 1 is a face. Each number refers to  one of the vertex values which are numbered starting from the top. v -2.22045e-16 -0.291306 -2.22045e-16 is vertex number 1 in this case. The software can easily use that information to compute normals.
A normal is computed from the cross product of the edge vectors. Stictly speaking the obj format doesn't compute the actual normals, it just provides the information that Meshlab needs so that Meshlab can compute the normals. which looks something along the lines of:
facted_normal = normalize( cross( 2-3, 1-3))
Where the numbers for 1, 2 and 3 are replaced with the actual values for those vertices. Other types of normals are usually computed based of that.
I loaded this file into Blender and it is well behaved, so odds are meshlab has some options turned on that are showing split normals or vertex normals or one of the many other possible variants out there.


Why does opengl perform clipping in clip coordinates? Is there any reason why OpenGL performs primitives' clipping in clip coordinates? Can it be at least theoretically done in normalized device coordinates?

The space after clip space is normalized device coordinate space, which is obtained by dividing clip.xyz by clip.w. If W is zero... oops.

Oh sure, you could put a conditional test to see if W is zero or very close to zero. Or you can just do the clipping math in clip space and never have to worry about it. No post-clipping vertex can have a W of zero, because the clip box for each vertex is based on being in the closed range (-W, W). And if W is 0, then the closed range is an empty set, and thus the vertex is not on that range.

Also, negative values of W are outside of the clipping space. The closed range (-W, W) is inverted where W is negative, but the meaning of the range itself is not. Consider a W of -1; the range becomes (1, -1). There are no numbers that are simultaneously greater than 1 and less than -1. Therefore that space is empty.

And yet, division by a negative W can still land NDC points in the [-1, 1] NDC-space range, where they will not be clipped if you didn't clip them beforehand.


How do I build my tangent space matrix? I'd like to understand how the process of building the "tangent space" matrix for normal mapping works. I'm following several tutorials, and other stackexchange questions, but I'm unfamiliar with the math involved, and a lot of resources seem to be telling me "just do this and it'll work".

According to the LearnOpenGL tutorial, you should build your tangent space matrix with the following code:

vec3 T = normalize(vec3(model * vec4(aTangent,   0.0)));
vec3 B = normalize(vec3(model * vec4(aBitangent, 0.0)));
vec3 N = normalize(vec3(model * vec4(aNormal,    0.0)));
mat3 TBN = mat3(T, B, N)

The first three lines are variations of:

vec3 worldSpaceVec = normalize(vec3(modelMatrix * vec4(modelSpaceVec, 0)));

Other implementations I've seen do:

vec3 worldSpaceVec = normalize(mat3(modelMatrix) * modelSpaceVec);

(by the way, I'm not sure I visualize why the LearnOpenGL tutorial doesn't use vec4(modelSpaceVec, 1); I'm not sure what either value would represent or how it would change the results. I assume this just a property of normals that we always represent them with a w of 0, but I don't get why it applies to the tangent and bitangent)

But more importantly, I don't understand the modelMatrix * ... part of the equation. According to this answer, normals are bivectors, a different type of data which is represented by a row matrix instead of a column matrix (that is, bivectors are $\begin{bmatrix} x & y & z \end{bmatrix}$ where vectors are$\begin{bmatrix} x \\ y \\ z \end{bmatrix}$).

If my understanding is correct, then normals are of a different type than regular vectors, just like kilograms and liters are similar but different types; and they need to be treated differently in regard to transformations. So why does the tutorial use:

vec3 worldSpaceVec = normalize(vec3(modelMatrix * vec4(modelSpaceVec, 0)));

and not

vec3 worldSpaceVec = inverse(transpose(modelMatrix)) * modelSpaceVec;

or even

vec3 worldSpaceVec = inverse(transpose(mvpMatrix)) * modelSpaceVec;

?

In other words, should the tangent and bitangent vectors of a vertex be converted from one space to another using the same transformation matrix as the vertex's normal vector, and if so, why?

In Normal Mapping, you want to go to "Tangent Space", that is the space where the center of the coordinate system is your Vertex (/Fragment). Just like you go from Model Space to World Space and from World Space to Eye Space, you can go to Tangent Space with a matrix multiplication.

In Tangent Space, the vector $n_T=\left(\begin{array}{c}0\\0\\1\end{array}\right)$ is always the Normal - since you read the normal $n_N$ out of the Normal Map in your Fragment Shader, you will obviously not use $n_T$ but $n_N$ for lighting calculations and this choice of $n_N$ rather than $n_T$ brings about the nice effect normal mapping has.

What's a bit special is that every vertex has its own Tangent Space, which is why you need the tangent, bitangent$^1$ and (geometry) normal of every vertex to construct your TBN matrix.

So much for theory - since you didn't really ask about that specifically, I assume you understand this (otherwise just edit your question).

  by the way, I'm not sure I visualize why the LearnOpenGL tutorial doesn't use vec4(modelSpaceVec, 1); I'm not sure what either value would represent or how it would change the results. I assume this just a property of normals that we always represent them with a w of 0, but I don't get why it applies to the tangent and bitangent

Vectors in general are represented with a $0$ in the $w$ coordinate. If you look into a translation matrix, you will notice that the last row is the one that moves points around. For vectors, this is not a wanted behaviour - a vector is the same, no matter where, since it is a direction. So to avoid "translating" a vector, you set the $w$ coordinate to $0$. Since your tangent and bitangent vectors are vectors, the $w$ coordinate is $0$ here as well.

  But more importantly, I don't understand the modelMatrix * ... part of the equation.

Using the the model matrix $M$ at this point is correct in most cases - but not all of them. The normal matrix $N$ is (as the answer you linked correctly said) the inverse transpose of the model matrix. But, since normals are vectors, you don't need a 4x4 matrix, a 3x3 matrix is just fine. And if you leave out that translation part, and you scale the same amount in all three coordinates, then the transpose of your model matrix is the same as the inverse of your model matrix. Therefore you get:

$
N = (M^{-1})^T = (M^T)^T = M
$

And that is why in most cases (i.e. everytime you use the same scaling in every coordinate) the normal matrix is the same as the model matrix.

  vec3 worldSpaceVec = inverse(transpose(modelMatrix)) * modelSpaceVec;

This is correct, however

  vec3 worldSpaceVec = inverse(transpose(mvpMatrix)) * modelSpaceVec;

this is not correct. Once you have transformed your normals into world space, you need to use the same view and projection matrix as before. Your view matrix brings your viewer into the origin (or rather it moves everything such that your viewer is the origin), inverse transposing this doesn't make sense (and similarly it doesn't make sense for the projection matrix. plus, the order in matrix multiplications is important, which is the next problem... don't do it ;-) )

Now, since you know that the model matrix may be the same as the normal matrix, you can (mathematically correctly) just go with the inverse transpose of the model matrix. Obviously this costs quite a lot of performance, especially inverting a matrix is costly. It is therefore more efficient to just go with the model matrix.

  In other words, should the tangent and bitangent vectors of a vertex be converted from one space to another using the same transformation matrix as the vertex's normal vector, and if so, why?

Unlike normals, tangent and bitangent vectors should be transformed with the model matrix, independently of the whether the normal matrix equals the modal matrix. A normal is always supposed to be perpendicular to a surface, and that's why simply taking the model matrix may be incorrect - in some cases the normal matrix would not be perpendicular to the surface anymore. Tangents however are always "lying" on the surface, so if you rotate a surface, the tangent must be rotated the same way to still "lie" on the surface.

$^1$ Sometimes this is called binormal, however Eric Lengyel explains why this is an incorrect term in Mathematics for 3D Game Programming and Computer Graphics.


How to build a triangle-voxel list from a triangle mesh? I have been playing with voxelization for a little time, I'm still very new to this technique.
I am trying to use it for ray tracing but I've been stuck with one problem. There are some codes out there in the web that already perform voxelization however, for ray tracing I need to check the intersection between a ray and a triangle.
Therefore, I've been thinking about what would be a good approach to create a list of triangles for each voxel in order to query the triangles once I know a ray hits a particular voxel? Should I try to modify the code of the voxelizer to benefit from the overlap-test or is it better to create a space-partitioning structure?

You could do this. Basically what you would do is generate a cube for each voxel (or actually nearby voxels could share the edges). Not exactly sure what benefit you would get form this, unless your intersection algorithm is somehow extremely finely tuned. It could be useful for real time rendering or some sort of sparse structure. While your at it why not dispense of the voxel concept altogether and go all FEM on the problem.

The thing is you want a more uniform sampling across the volumes density function. Because your integrating over a volume functions you want the step size to be somewhat equally spaced so you can control the integration better. Random or pseudo random solution converges often better, but mainly you do not do cube oriented sampling this way as it will make the structure obvious which is not your goal in most cases.

Image 1: Different sampling strategies. Images in RTL reading order samples along voxel grids,  Uniforms sampling and jittered ray length sampling.

Think of this as solving a ODE you sample the function uniformly and if your value changes too much you might do additional samples. This is probably the reason why you wouldn't really do collision to the voxels.


Rendering a volume that might be completely clipped in OpenGL Suppose that I have some world-space volume (which we may assume is a convex polyhedron with a small number of vertices), and I want to render its intersection with the view frustum, such that every relevant pixel gets rendered exactly once. My use-case is rendering light influence volumes for a deferred renderer, but I guess there may be other uses for this.
Everything is fine as long as the volume is completely behind the near clipping plane (i.e. isn't clipped by the near z-plane): rendering with back-face culling does the job, thanks to convexity of the volume. If, on the other hand, the volume intersects the near z-plane a bit, the intersection area gets clipped, but there are still parts of the volume behind this area that didn't get rendered. We could switch to front-face culling, but that breaks if the volume intersects the far z-plane. The volume could theoretically intersect both z-planes, so this problem cannot be solved simply by adjusting the culling mode. The volume could as well completely embody the view frustum, in which case nothing will be draw at all (all triangles outside the frustum), while in reality I'd like it to be rendered as if I'm rendering a fullscreen quad (since the intersection of the volume and the frustum equals the frustum).
Right now my solution is to check if any of the volume's vertices are in front of the near z-plane (i.e. are clipped by the near z-plane) and resort to a fullscreen quad in this case. It works fine, but I'm curious whether there is a better solution.
One solution that I don't particularly like is actually computing the mesh of the intersection of volume and the view frustum (both are convex polyhedrons, which should simplify the job a bit). It has the following disadvantages:

I have to either find a good enough library for mesh intersection, which means adding another dependency for the engine, or write the intersection code myself
More importantly, even if the intersection is computed with maximal possible precision, it still goes through the vertex shader, gets multiplied by the view-projection matrix, with numerical errors inevitably introduced. Then, a vertex at z=near may appear at z=near+/-error, and it might still be erroneously clipped.

First, regarding the specific problem of bits of the geometry being lost to the near and far planes, this can be solved by using depth clamping instead of depth clipping. This disables clipping against the near and far planes, in favor of clamping the output depth values to [0, 1]. (Geometry behind the camera will still be clipped though.) Given this, instead of a fullscreen quad you could render the light volume with inverted culling, depth testing inverted (to cull pixels behind the volume), and use depth clamping to prevent losing parts of the volume to the far clip plane.
(Depth clamping is also handy for shadow maps, as it lets you contract the shadow map depth bounds to bracket the view frustum, without losing geometry between the view frustum and the light source.)
In a broader sense, though, this doesn't really solve the problem of shading pixels within a given convex volume. The problem is that you need two depth tests: front of volume < pixel depth < back of volume. The approaches discussed so far can only do one of these tests at a time. You can switch which one you're doing by swapping culling and depth-testing modes, but you can't do both at once, which leaves performance on the table.
One approach to address this is to use the stencil buffer in concert with depth testing. Basically this works by testing scene depth against the front of the light volume first, storing that to the stencil buffer, then doing a second pass that tests scene depth against the back of the light volume, and combining that with the stencil result from the first pass. This has the advantage of getting tight culling (no pixels shaded outside the volume), but you need to draw each light volume twice, plus making a lot of state changes to switch back and forth, which can be inefficient for both CPU and GPU (though you can batch up to 8 lights together using 1 stencil bit per light).
Finally, using EXT_depth_bounds_test, if available, can be a good solution. This enables a form of depth test that checks whether the pixel already in the framebuffer (not the geometry being rasterized) is within a certain range. With this, you set the depth bounds to the min/max depth of your light volume, then draw the geometry with inverted culling, no depth test, and depth clamping enabled. This will shade all pixels within a sort of prism aligned with the view frustum, that bounds the actual volume reasonably tightly in most cases (though long narrow spotlights can be bad), and it does it in only one pass. However, this extension isn't universally available; it seems to be found on current NVIDIA and AMD GPUs, but not Intel ones (see OpenGL Hardware Database).
All of this difficulty is one reason that most game engines since the early 2010s have switched to compute-shader tiled culling instead of using the rasterizer to decide which lights to shade at which pixel.


Stretch the cube mesh but repeat the texture? I have a simple cube mesh, which I scale with modifying the model matrix.

Texture settings:

glGenTextures(1, &m_texture);
glBindTexture(GL_TEXTURE_2D, m_texture);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);

glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, data);

I set GL_REPEAT as wrap mode, but in my case it doesn't help, because the texture coordinates don't exceed [0...1].

For example from the unit cube I want to make a column whose dimensions are (1,10,1).

Just scaling this doesn't work because it textures the cube and then applies the scaling, which means that the texture will be stretched, instead of repeated along the y axis.

How can I implement this so I could set the length, width and depth of a cube dynamically and the texture will be repeated on it instead of stretched?

You need to set your texture coordinates appropriately. Let's look at one side of your cube. Let's say it's 100 x 100. If the user stretches the cube to be 100 x 150, you need to set the texture coordinates of the corner vertices to be:

lower left = <0.0, 0.0>
lower right = <1.5, 0.0>
upper right = <1.5, 1.0>
upper left = <0.0, 1.0>

That's the easiest way to handle it. Another way to handle it would be to pass either as scale factor or scale matrix into your fragment shader and apply that to your texture coordinates before doing any sampling.

To elaborate on the texture matrix idea, it's fairly simple. In your fragment shader, you'd have a uniform, say textureMatrix that's a mat4. It can contain any sort of transform you need - scale, rotation, translation, shear, etc. Before you sample any texture, you run the uv coordinates through the texture matrix, and then use the resulting values for your lookup. Something like this:

vec4 uv = gl_TexCoord[ 0 ]; // or whatever you called it coming from your vert. shader
vec4 newUV = textureMatrix * uv;
vec4 pixel = texture2D(texture, newUV.xy);

To do your non-uniform scaling, your matrix would be a scale matrix. For example if you wanted to scale the Y coordinate, you could use a matrix like this:

| 1 0 0 0 |
| 0 2 0 0 |
| 0 0 0 0 |
| 0 0 0 1 |


Importance Sampling in Unreal Engine Formula Derivation (Irradiance term) I am currently trying to understand IBL in Unreal Engine, and there are so much things that I don't understand about the formula. Unreal approximate the specular term of shading equation by doing split sum approximation.

This is how unreal pre-filter the environment map for specular-irradiance term.

float3 PrefilterEnvMap(float Roughness, float3 R )
{
    float3 N = R;
    float3 V = R;
    float3 PrefilteredColor = 0;
    const uint NumSamples = 1024;
    for(uint i = 0; i < NumSamples; i++ )
    {
        float2 Xi = Hammersley( i, NumSamples );
        float3 H = ImportanceSampleGGX( Xi, Roughness, N );
        float3 L = 2 * dot  ( V, H ) * H - V;
        float NoL =saturate( dot  ( N, L ) );
        if ( NoL > 0 )
        {
            PrefilteredColor += EnvMap.SampleLevel( EnvMapSampler, L, 0 ).rgb * NoL;
            TotalWeight += NoL;
        }
    }
    return PrefilteredColor / TotalWeight;
}

My question is why does the total weight is the sum of saturate(dot(N,L)). My understanding on importance sampling is that we should divide our sample with the pdf we use to sample. In this case, from what I read at here the pdf should be

$$p_i(wm, wo) = \frac{D(wm)(wm \cdot wg)}{4|wo \cdot wm|}$$

My reference on pbr in unreal engine is here and here.

I just read notes on moving frostbite to pbr and I found the derivation of the method above. So I will just show the derivation here and quote some of the explanation.

  One can notice an extra〈n·l〉in the LD term as well as a different
  weighting 1/(∑Ni〈n·l〉). These empirical terms have been introduce by
  Karis to allows to improve the reconstructed lighting integral which
  suffers from coarse hypothesis of separability of this integral.
  There is no mathematical derivation for these terms, goal was to have
  an exact match with a constant L(l).

So it turns out the pdf is weighted on the DFG term. As for dot(N,l), the term is introduced to minimize the error that is caused by split sum approximation. But I am actually still wondering what is the intuition on that empirical term.


Operations in the Rendering Pipeline I have a few short question regarding operations happening in the rendering pipeline (OpenGL, Vulkan, I'd expect for all questions to have similar answers for these two frameworks while DirectX might work a bit different).
Is the division of the coordinates x/w,y/w,z/w the last operation that happens on the GPU? Or are there other operations that happen afterwards? I wasn't really able to find any information on that, could anyone point me into the right direction?
Furthermore, concerning the depth of fragments, as far as I understand a quantization takes place to discrete units of depth, but I couldn't find anything detailed if these operations follow some kind of IEEE standard or if they have their own standards, maybe even depending on the manufacturer. Does anyone know more detail about this or could point me into the right direction?
In general are there some, more or less, complete information about the possible operations (besides of course programmable operations during the programmable shader stages), the error bounds, etc. for operations on the GPU?
Kind regards and thank you in advance.

Is the division of the coordinates x/w,y/w,z/w the last operation that happens on the GPU?

The division you are referring to is generally called "The Perspective Divide". You can google it and get information overload on helpful documentation. The division itself happens after the geometry shader before the fragment shader as part of rasterization. And is part of the process that creates the fragments that the pixel shader is run on. It is also an important part of vertex attribute interpolation. All of OpenGL, Vulkan, DirectX perform the perspective divide in the same place which is done by the GPU's hardware.
The most adopted floating point standard is IEEE 754 and yes virtually all vendors implement and follow these standards. I highly recommend (slowly) wrapping your mind around this standard to get a deeper understanding of how floats are processed. Here is a good first factoid to memorize (if you want): The standard put the sign bit for floating point numbers in the same place as the sign bit for integers to make conversion between and understanding both easier.
Quantization in this context refers to the accuracy of floating point numbers. There are many numbers that can not be fully represented in floating point math on the GPU, such as 1/3. Quantization is the process of representing that number as closely as possible within the limitations of the hardware. But minor difference in hardware implementations that are still standards compliant can and often will produce slightly different results.
Rather then focus on the limits of floating point numbers or attempting to write code that operates at the edge of these limits, coders generally use techniques that avoid the issue altogether like utilizing hardware offset capabilities to side step the issue completely is one method. (but is situation specific, like when applying decals)


File format for swept profile with changing normal I have to create a file describing a 3d surface with the following properties:

a circular arc cross section is swept along a closed 2d curve spine defined with equations
the normal direction of the arc is not always tangential to the spine, the direction is defined with an equation
the surface should be analytic (defined by equations), but a tesselated surface is also acceptable (with an external script I can derive the x,y,z coords of the vertexes for a tesselated surface)
the file will be modified by an external script (python), so it should be editable with text editor
the file should be opened by Creo(Proe), Catia and NX

The question:
Which is the best file format for making this feature?

What I tried:

step: buying the ISO standard is unavoidable, as I cannot figure out on my own how the definition works
vrml: only tesselated seems to be feasible
x3d: same as vrml, as it has a sweep (extrusion) feature, but the normal direction is the same as the local spine direction, and Proe does not seem to open it

The egg-shaped curve is the trajectory, the arc is the curve to be swept along it.
The plane of the arc is close to parallel to the plane "Top", but this angle varies between 0-30 degrees

Well realistically your choices are either IGES or STEP. IGES is slightly simpler but you will not successfully write either format without buying the standard (which in case of step is actually so many pages that by the time you have read them a year has passed, and lots of money is wasted on paper). You could also use one of the nonstandard kernel internal formats like Parasolid as its also commonly supported. Most other formats are polygon data so not suitable.

I would explore an alternative option to writing your own exporter, buy a commercial or take a free CAD kernel and let the kernel (open CASCADE) do the exporting for you.

On the other hand this is not hard to do inside your CAD application. Which you will need anyway to verify the result. Most CAD applications have a COM API's (don't know about NX most likely, but both creo and catia do have this) so you can edit, even do the generation from a text editor via python if you so wish. This is by far the least amount of effort (and it's usually less work to set up than using just the kernel).

Image 1: Swept with spine angle $60 \cos(4 t)+110$ and polar trajectory with $r = \cos(4 t)+3$ in Creo. All input from program injection.

So the approach depends on why you want to do what you do. And what your budget/time constraint is.

PS: NURBS (the underlying format of CAD data) solutions are not necessarily analytic, but parametric fits to analytical solutions. Sometimes these are exact sometimes not so exact as not all functions can be fitted to rational spline (though many can). Even when they can fitting is a more general solution.


Tone Mapping & Gamma Correction I've been looking up this question on the internet for a while, and I haven't been very successful, so I decided I'll ask it here. Should I apply tone mapping before gamma correction, gamma correction before tone mapping, or should I just use one or the other?

This depends on your calculations really, since you can do it either way. However, probably when you reach tone mapping you will be in linear space (i.e. no gamma correction has been done yet and you don't assume non-linear space for your calculations).
Assuming this, gamma correction should be applied after tone mapping, otherwise you have "linear" tone mapping on non-linear color values.

To quote from Real-Time Rendering 3rd by Möller, Haines and Hoffman, p. 145:

  It is important to apply the conversion at the final stage of rendering (when the values are written to the display buffer for the last time), and not before. If post-processing is applied after gamma correction, post-processing effects will be computed in nonlinear space, which is incorrect and will often cause visible artifacts. So operations such as tone mapping [...] can be applied to images to adjust luminance balance, but gamma corection should always be done last.


Rectangular prism parametric equation I have some problem in OSG doing my model, which need an rectangular prism, that I do not know the parametric equation.

I tried to search with the google, but no usable result.

Also tried to draw it by using this site: http://www.math.uri.edu/~bkaskosz/flashmo/tools/parsur/

Could somebody help to find the parametric equation for (x,y,z) cordinates

A prism is a polyhedron with a polygonal base, another parallel base of the same shape, and joining faces between corresponding edges of the two base faces.

The word "prism" is commonly used to describe a right prism with base faces perpendicular to the joining faces. In this case the joining faces are also rectangles, and so a rectangular right prism is simply a cuboid.

A cuboid can be described directly, but a simple way to generate one is to start with a cube and use directional scaling to give the edges the required lengths.

If the base faces are not perpendicular to the joining faces, this oblique prism can be achieved by applying a shear to a cuboid with the required edge lengths.

So all you need is a parametric description of a cube, and you can go from there to any rectangular prism required.

As joojaa points out, there isn't an easy way to describe a prism (including a cube) in just a single parametrisation. You will need to describe each face with a separate set of parametric equations. For example, using parameters u and v, you could describe one of the faces as follows.

$$  x=u  \\  y=v  \\  z=1  \\  0 \leq u \leq 1  \\  0 \leq v \leq 1  $$


Unable to pass custom Matrix4 to GLSL as a uniform I am fairly new to graphics programming and I am trying to pass a custom Matrix4 (non-glm) to my vertex shader by using a uniform.

    Matrix44 transform;  //creates a 4x4 identity matrix
    transform.SetTranslation(Vector3(0.5f, 0.3f, 0.0f));
    GLint transformLocation = glGetUniformLocation(ourShader.Program, "transformation");
    glUniformMatrix4fv(transformLocation, sizeof(transform), GL_FALSE, &transform.m[0][0]);

Above is the code which I am using to send the matrix to the shaders.

(the .m allows me to access the matrix as a 2D array of 4x4 floats);

This is how my vertex shader looks like:

#version 430 core

layout (location = 0) in vec3 position;

uniform mat4 transform;

void main()
{
   gl_Position = transform*vec4(position,1.0f);
}

When I run it, I don't get anything drawn on my screen, except the background color.

Below is the matrix which I send to the shader. I am using column major, so as far as I've read this should be the translation part of the 4x4. I've also tried transposing the matrix with no luck whatsoever.

The last clue I have is that the values which I am sending are 0s. I came to this conclusion because I tried playing with the fragment shader and I set the color to be equal to the last column from the uniform mat4:

#version 430 core
out vec4 color;

uniform mat4 transform;

void main()
{
 color = vec4(transform[0][3],transform[1][3], transform[2][3], transform[3][3]);
}

Which gives me a black triangle i.e. the color vec4 is filled with 0s.

I apologize in advance if the question is too noob-ish but I've been struggling with this for several days and I could not find a way to run it :(

Any advise would be appreciated!

glUniformMatrix4fv(transformLocation, sizeof(transform), GL_FALSE, &transform.m[0][0]);

The second parameter to glUniformMatrix is not the number of bytes in the data. OpenGL can figure that out from the 4fv part of the command (a 4x4 matrix of GLfloats). The second parameter is the number of matrices you're sending to that uniform, in case it is an array of mat4s.

So that value ought to be 1. You almost certainly got a GL_INVALID_OPERATION error from calling it with sizeof(transform).


Dealing with wasted space when using SIMD on CPU for vector maths I've been getting interested in SIMD programming on CPU using SSE recently, but I'm a complete noob on the subject. I found this article describing how to make an efficient float3 type using the recent __vectorcall convention, and it made me wonder how best to deal with the wasted space inherent in using a 128-bit type to represent vectors smaller than 4.

For a typical use case, you might want to transform a bunch of positions and normals on the CPU in order to do a batched draw call. When transmitting the data over to the GPU, though, ideally you'd want this as tightly packed arrays of float3, either interleaved or in separate buffers.

It seems like there are two options here:

A - store the data tightly packed, and unpack and repack to and from SSE-friendly format when you need to manipulate it, taking the CPU performance hit, or

B - store the data loosely packed on 16-byte boundaries, SSE it directly, and take the bandwidth and VRAM hit when sending to GPU.

Which of these options would be preferred in a real-world 3D engine, or under what circumstances would you prefer one to the other?

There's a third option you've neglected. Because of how graphics APIs work, typically you have to copy the data between the CPU and GPU anyway - even if you're on a mobile SoC where they share the same physical memory. This copy is the ideal time to take the hit of changing format, since you have to touch all of the memory anyway. So the copy in user memory, owned by the CPU, can be stored padded for SIMD alignment. Then, when you need to send that data to GPU memory, instead of doing a memcpy, just read in the 4-vectors and write out 3-vectors. It's less memory bandwidth than a normal copy (you're reading just as much, but writing less) and negligible CPU work.

If you've got that fourth float doing nothing, you might as well use it and make your co-ordinates homogeneous instead of just xyz. Depending on what compute you're doing on the CPU, that might make some things easier (transforms are the obvious example, but also reducing divisions if you were renormalising your normals a lot before).

This is similar but not identical to a trick I used when implementing OpenGL texture upload. On ARM, most Neon instructions operate at memory bandwidth rate, so if you're already doing a memcpy, it's free to do a pipeline-worth of instructions at the same time. You can use those free instructions to do pixel format conversion and reordering to Z-order or U-order.


How to perform vertex selection using unproject (Qt/openGL) I am struggling trying to implement vertex selection by mouse clicking. I load in some set of vertices and display them onto the screen. Then I do the following to try and retrieve a line that passes through the selected point in world coordinates and through the eye position.

  setFocus();
  float xRatio, yRatio, xScene, yScene;
  xRatio = (float)event->x() / width();
  yRatio = (float)event->y() / height();

  xScene = (1-xRatio)*-1 + xRatio*1;
  yScene = yRatio*-1 + (1-yRatio)*1;
  // mouse click now transformed to [-1,1]^2

  // Import the viewPort
      GLint *params = new GLint(4);
      glGetIntegerv(GL_VIEWPORT, params);
      QRect vp = QRect(*params, *(params + 1), *(params + 2), *(params + 3));

      QVector3D dir = QVector3D(xScene, yScene, 0.2); // distance to near pane = 0.2
      QVector3D origin = QVector3D(0.0, 0.0, 0.0);

  // Now I have the origin and point in near clipping pane of the line (I think)
  // Unproject them to world coordinates.
      dir = dir.unproject(modelViewMatrix, projectionMatrix, vp);
      origin = origin.unproject(modelViewMatrix, projectionMatrix, vp);

  float minDist = 1000.0;
  int minIndex = 0;
  float dist;

  for (int i = 0; i < vertexCoords.size(); ++i){
      dist = vertexCoords[i].distanceToLine(origin, dir);
      if (dist < minDist){
          minDist = dist;
          minIndex = i;
      }
  }

I keep getting wrong result with this method. It will almost always select the same point and report a large distance of about 20 to that point. So something is going terribly wrong. Can anyone spot what is going wrong or what I am missing?

Additionally, here is how the view and projection matrices look:

  modelViewMatrix.setToIdentity();
  modelViewMatrix.translate(QVector3D(0.0, 0.0, -3.0));
  modelViewMatrix.scale(QVector3D(1.0, 1.0, 1.0));
  modelViewMatrix.rotate(rotX, 0.0, 1.0, 0.0);
  modelViewMatrix.rotate(rotY, 1.0, 0.0, 0.0);

  projectionMatrix.setToIdentity();
  projectionMatrix.perspective(FoV, dispRatio, 0.2, 8.0);

Sorry for the long question, I have found quite some resources online but none of them help me figure out what is going wrong. Also, if there is a better method I would gladly hear it! Thanks

You're basically mixing up your spaces.

If you look at the documentation for QVector3D::unproject, you see that it actually expects coordinates in window space. And this makes sense, since you explicitly query the viewport (i.e. basically the window's pixel dimensions) and give that to the function.

But with your ratio computation above you already transformed your coordinates from that space into normalized device space, i.e. the $[-1,1]$ square. So this whole computation isn't necessary at all and is already done for you by unproject. Likewise are the near and far distances from world/eye space not the same in window space, which uses the normalized $[0,1]$ depth range. If we assume that your viewport is your whole window, then all you need to do is put those pixel coordinates you clicked on into unproject (still accounting for the different Y-convention, though):

dir = QVector3D(event.x, height()-event.y, 0.);    //in window space 0 = near, 1 = far
dir = dir.unproject(modelViewMatrix, projectionMatrix, vp);

A problem is that you don't easily know the window space coordinates of the camera position, since it is in front of the near plane, but the transformation from $[0,1]$ to $[near,far]$ is non-linear. One way to overcome this would be to just unproject a point on the near plane (depth = 0) as origin and the difference to the unproject of that same point on the far plane (depth = 1) as the direction.

QVector3D near(QVector3D(event.x, height()-event.y, 0.);
QVector3D far( QVector3D(event.x, height()-event.y, 1.);
origin = near.unproject(modelViewMatrix, projectionMatrix, vp);
dir = far.unproject(modelViewMatrix, projectionMatrix, vp) - origin;

Or you compute dir just the same and don't unproject the origin at all, rather than just transforming the eye-space origin (which is conveniently $\mathbf{0}$) back into model space (assuming your modelview matrix isn't projective, which it really shouldn't be, otherwise don't forget the W-division):

origin = QVector3D(modelViewMatrix.inverted() * QVector4D(0., 0., 0., 1.));
//origin = QVector3D(modelViewMatrix.inverted().column(3));    //would work the same

If your viewport is not the same as your window dimensions however, you need to fiddle with the mouse coordinates a little more and transform them from the entire window into your viewport rect before unprojecting them. But the point is that

unproject expects window space (i.e. pixel) coordinates.


How physically-based is the diffuse and specular distinction? The classical way of shading surfaces in real-time computer graphics is a combination of a (Lambertian) diffuse term and a specular term, most likely Phong or Blinn-Phong.

Now with the trend going towards physically-based rendering and thus material models in engines such as Frostbite, Unreal Engine or Unity 3D these BRDFs have changed. For example (a pretty universal one at that), the latest Unreal Engine still uses Lambertian diffuse, but in combination with the Cook-Torrance microfacet model for specular reflection (specifically using GGX/Trowbridge-Reitz and a modified Slick approximation for the Fresnel term). Furthermore, a 'Metalness' value is being used to distinguish between conductor and dielectric.

For dielectrics, diffuse is colored using the albedo of the material, while specular is always colorless. For metals, diffuse is not used and the specular term is multiplied with the albedo of the material.

Regarding real-world physical materials, does the strict separation between diffuse and specular exist and if so, where does it come from? Why is one colored while the other is not? Why do conductors behave differently?

To start, I highly suggest reading Naty Hoffman's Siggraph presentation covering the physics of rendering. That said, I will try to answer your specific questions, borrowing images from his presentation.

Looking at a single light particle hitting a point on the surface of a material, it can do 2 things: reflect, or refract. Reflected light will bounce away from the surface, similar to a mirror. Refracted light bounces around inside the material, and may exit the material some distance away from where it entered. Finally, every time the light interacts with the molecules of the material, it loses some energy. If it loses enough of its energy, we consider it to be fully absorbed.

To quote Naty, "Light is composed of electromagnetic waves. So the optical properties of a substance are closely linked to its electric properties." This is why we group materials as metals or non-metals.

Non metals will exhibit both reflection and refraction.

Metallic materials only have reflection. All refracted light is absorbed.

It would be prohibitively expensive to try to model the light particle's interaction with the molecules of the material. We instead, make some assumptions and simplifications.

If the pixel size or shading area is large compared to the entry-exit distances, we can make the assumption that the distances are effectively zero.
For convenience, we split the light interactions into two different terms. We call the surface reflection term "specular" and the term resulting from refraction, absorption, scattering, and re-refraction we call "diffuse".

However, this is a pretty large assumption. For most opaque materials, this assumption is ok and doesn't differ too much from real-life. However, for materials with any kind of transparency, the assumption fails. For example, milk, skin, soap, etc.

A material's observed color is the light that is not absorbed. This is a combination of both the reflected light, as well as any refracted light that exits the material. For example, a pure green material will absorb all light that is not green, so the only light to reach our eyes is the green light.

Therefore an artist models the color of a material by giving us the attenuation function for the material, ie how the light will be absorbed by the material. In our simplified diffuse/specular model, this can be represented by two colors, the diffuse color, and the specular color. Back before physically-based materials were used, the artist would arbitrarily choose each of these colors. However, it should seem obvious that these two colors should be related. This is where the albedo color comes in. For example, in UE4, they calculate diffuse and specular color as follows:

DiffuseColor = AlbedoColor - AlbedoColor * Metallic;
SpecColor = lerp(0.08 * Specular.xxx, AlbedoColor, Metallic)

where Metallic is 0 for non-metals and 1 for metals. The 'Specular' parameter controls the specularity of an object (but it's usually a constant 0.5 for 99% of materials)


Fast clipping without clearing stencil buffer I'm writing an OpenGL application that runs on a Raspberry Pi, i.e., a quite resource-limited system. Essentially, a few quads with RGBA textures are overlayed/alpha blended on top of each other with individual translation/rotation applied. They are clipped against an unrotated rectangle.

For this purpose, I currently use the stencil buffer. I.e., when drawing a texture which requires clipping, I clear the stencil buffer, render the clipping quad to the stencil buffer and enable the stencil testing before rendering the translated/rotated object.

On a Pi, clearing the stencil buffer is a painfully slow process in the order of 20ms, completely ruining performace. I thought about pre-rending the clipping rects to different bits of the stencil buffer, but that would lead to a maximum of 8 clipping regions (I need more).

Therefore, is there a different way of achieving clipping against a simple rectangle that does not require an expensive operation such as flushing the stencil buffer?

It sounds like what you need is a scissor test. It's specifically designed for clipping against an unrotated rectangle, and should be faster than messing about with the stencil buffer.


Strange sphere reflection effects in ray tracing project. I cannot figure out what causes them I started a ray tracing project in Python. I made a CUDA kernel and a trace() function for each pixel. It works perfectly when calculating intersections, shadows and lambert shading, however when I introduce a reflection from the surface of a sphere I get a 'band' around the edge of the projected sphere disk and I cannot figure out why.

Image results: https://imgur.com/a/LoXOyl5

My reflection algorithm:

Find point P of the intersection.
Find normal vector N at the intersection with C being the sphere center:

    N = P - C
    N = N / norm(N)

Find reflection vector R using (where D is the original ray direction:

    R = D - 2 * (D * N ) * N
    R = R / norm(R)

Shift point P along R a little bit (to avoid reflection acne, adding this didn't solve the problem)

    P = P + 1e-4 * R

Call the ray trace function (only one more time) again as:

    color_refl = trace(ray_origin=P, ray_direction=R)

Add the reflected color using:

    color = color + color_refl * reflect_intensity

Notes:

Between step 2 and step 3 I calculate the Lambert shading which works quite well.
Reflections that bounce off a plane work perfectly.
I use a CUDA kernel which does not support recursive function calls, so I made another trace function with a different name that doesn't calculate reflections. (that's why there is only one reflection call)

Here is the source code for the trace and reflection trace functions:
https://github.com/Speterius/ray_tracing/blob/master/cuda_ray_tracing.py

The issue was caused by an incorrect calculation of the reflection direction vector.
With D ray direction and N the normal vector:

R = D - 2 * dot(D, N) * N

The issue was caused by calculating the components of R as follows:

R[i] = D[i] - 2 * (D[i] * N[i]) * N[i]

It took me a while to find the mistake because this produced a correct reflection with the horizontal plane.


How can I get a signed distance (SDF) from a mesh? I am trying to use constructive solid geometry (CSG) and boolean operators to combine various shapes and get the outer (possibly concave) hull. This seems to work okay when using primitive shapes like cubes and spheres but I cannot figure out how to use other shapes.
Can I get a signed distance to a mesh? How would I calculate it?
I found https://www.iquilezles.org/www/articles/sdfbounding/sdfbounding.htm which seems like it might be what I want, specifically the section where they use an sdMesh method, but I do not understand where the sdTriangles method comes from. The only sdTriangle method that I found is on the 2D Distance Functions page. Do I use that somehow? Will that give a negative number if the point is "behind" the triangle in 3D space (i.e. inside the mesh)?
I am working in Unity / C#, but can translate from other languages.

Generally it boils down to finding the nearest element. Computing distance to it. Checking the normal to tell if inside or out. There are a number of edge cases. Also you usually compute the "angle weighted psuedonormal" as its called.
Here's the usual reference "Signed Distance Computation using the Angle
Weighted Pseudo-normal" by J. Bærentzen.
There are many such libraries with the implementation if you don't want to write it yourself. I personally use ImplicitPolyDataDistance function in VTK. Also a fan of LibIGLs. But there are many lighter weight ones to do it as well, https://github.com/Magnus2/MeshSdf.
It's also worth mentioning that you can compute SDFs in other less accurate ways for more performance. Namely, another popular approach is to voxelize/binarize a shape and compute the SDF of the binary image instead. The distances at the boundary of the shape won't be exact but it is a good estimate and often faster.
EDIT: Also worth mentioning narrow bands and hierarchical SDFs. Those are very popular as well. Instead of computing SDF values for every element. Say in an image. We can compute SDF values only near the boundary of the shape. With an octree you can compute it at each vertex and interpolate to get inbetween estimates. There are special methods for computing these as well and libraries such as OpenVDB.


How to render the wireframe of a cube without the diagonals on the faces? I'm developing with OpenGL 4.1 and have good understanding of the modern pipeline. I was thinking on doing it with GL_LINE_LOOP for each face, but I think this will require several draw calls which certainly is not optimal. What could be a better approach?

Things like that are usually done using an index buffer. The idea is that you have you have two buffers: One for all the vertices and one that determines the topology of what to draw. Then you can draw all the lines at once with a single glDrawElements call, using GL_LINES as mode.


Is there a term for mapping between two arbitrary quadrilaterals without using homography / perspective projection? When warping the contents of an arbitrary quadrilateral to another by extrapolating equal points along both the "lengths" and "widths", thus ignoring all perspective effects / foreshortening, we are not using a "homography" or "affine" mapping.

Is there a correct name to describe the type of mapping we are doing?

Yes, this is called bilinear interpolation or a bilinear mapping. The term "bilinear" crops up more commonly in the context of image resizing or resampling, but it's the same basic idea for transforming quadrilaterals—it's just that the interpolation is applied to the vertex positions of the quad, instead of the colors of pixels in an image.

One paper that briefly discusses bilinear mappings of quadrilaterals is Fundamentals of Texture Mapping and Image Warping by Paul Heckbert; see section 2.2.2.