Loading compressed image to GPU and reading back decompressed on CPU I have some images that I know will be stored with either PVR or ETC compression. I would like to be able to convert these images into uncompressed RGB. However, because of binary size constraints, I would like to do so without adding a dedicated library for the conversion.

Does there exist any ways to do this? I know functions like glCompressedTexImage2D will let me load the compressed image to GPU. However, I'm having issues reading the decoded version out. For extra context, I am trying to do this with OpenGL ES 2.0.

Thanks!

Desktop OpenGL requires implementations to convert pixel data between the internal format of the image and the format you specify in the pixel transfer command.

OpenGL ES does not. Indeed, it is so serious about not allowing this that it actually changes the meaning of the parameters to functions like glTexImage*d. In ES, you cannot use sized internal formats (well, outside of texture storage calls). Instead, the internal format is kinda there. What really defines the internal format is the pixel format and type parameters.

That is, when you tell OpenGL ES that the pixel data you're providing looks like X, it requires the implementation to create a texture whose real internal format exactly matches that. And if you're using glTexSubImage*d or glGetTexImage, you must provide pixel transfer parameters that match those you provided to the glTexImage*d call.

So OpenGL ES doesn't allow any of that pixel conversion stuff.


Fill the plane with pentagons as tightly as possible in a regular way I wrote a small python script with recursion to create a "lattice" of non-overlapping pentagons. Below one can see the first stages of recursion.

One can see that 5 small pentagons are missing in the second stage of recursion. The code in python to generate these figures is this:

# fill the plane with pentagons as tightly as possible in a regular way
#!/usr/bin/python
import numpy as np
import matplotlib.pyplot as plt
N = 5
phi = (1.0+np.sqrt(5.0))/2
def pentagon (X, Y, n):
    if n == 0:
        rx = np.append(X, X[0])
        ry = np.append(Y, Y[0])
        plt.plot(rx, ry, 'r-')
        return
    for i in range(N):
        Xn = np.array([X[i], X[i], X[i], X[i], X[i]])
        Yn = np.array([Y[i], Y[i], Y[i], Y[i], Y[i]])
    U = Xn/phi+X*(1-1.0/phi)
    V = Yn/phi+Y*(1-1.0/phi)
    pentagon (U, V, n-1)
Xn = X/phi +np.roll(X, 2)*(1-1.0/phi)
Yn = Y/phi +np.roll(Y, 2)*(1-1.0/phi)
# center pentagon
pentagon (Xn, Yn, n-1)
Ind = np.arange(N)
theta =np.pi/10
X = np.cos(Ind*2*np.pi/N+theta)
Y = np.sin(Ind*2*np.pi/N+theta)
plt.figure()
plt.axes().set_aspect('equal')
n_iter = 2 # number of iterations
pentagon (X, Y, n_iter)
figure = "pentagons%d.png" % n_iter
plt.axis('off')
plt.savefig(figure)
plt.show()

Any suggestions on how to fill in the missing pentagons in a simple way (e.g. keeping the recursive nature of the algorithm)?

If I understand the general idea correctly, you are going to close the gaps which are appearing after further structure growth. I cannot do it algorithmically but geometrically it is quite easy to do. I use Illustrator here and manually cover the gaps with two additional shapes.

So in addition to the pentagon and flat rhombus, I will need a 5-fold star and another rhombus:

So after first steps we get such a structure and the filling will be:

(Initial center is marked red)

I don't fill all holes and I leave out the obvious ones just to make arrangements more visible.

Then next step will give this:

Here I do only halves of crack fillings, those are symmetrical. As seen those cracks indeed can be covered using same shapes without gaps and have self-similar patterns. How this can be achieved algorithmically, I don't know unfortunatelly. But this is interesting question, probably there are solutions somewhere, IDK.

Theoretically if one analyse several crack fillings for each step, one can guess a relatively
simple algorithm.


Mipmap sampler in Metal compute kernel (not vertex or fragment shader) I have a source texture (480x480) that was created with mipmapped set to true (error checking removed to simply this post), and a dest texture (100x100):

// source texture
var textureDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: MTLPixelFormat.r8Unorm, width: Int(480), height: Int(480), mipmapped: true)
textureDescriptor.usage = .unknown // .shaderWrite .shaderRead
srcTexture = metalDevice!.makeTexture(descriptor: textureDescriptor)
// Dest texture
textureDescriptor = MTLTextureDescriptor.texture2DDescriptor(pixelFormat: MTLPixelFormat.r8Unorm, width: Int(100), height: Int(100), mipmapped: false)
textureDescriptor.usage = .shaderWrite
destTexture = metalDevice!.makeTexture(descriptor: textureDescriptor)

The sampler is defined:

let samplerDescriptor = MTLSamplerDescriptor()
samplerDescriptor.magFilter = .linear
samplerDescriptor.minFilter = .linear
samplerDescriptor.rAddressMode = .clampToZero
samplerDescriptor.sAddressMode = .clampToZero
samplerDescriptor.tAddressMode = .clampToZero
samplerDescriptor.normalizedCoordinates = true
textureSampler = metalDevice!.makeSamplerState(descriptor: samplerDescriptor)

I populated the src texture with an image.

Then generated the mipmaps:

let blitEncoder = metalCommandBuffer!.makeBlitCommandEncoder()
blitEncoder!.pushDebugGroup("Dispatch mipmap kernel")
blitEncoder!.generateMipmaps(for: srcTexture!);
blitEncoder!.popDebugGroup()
blitEncoder!.endEncoding()

And in the same command buffer, ran the resize kernel:

let computeEncoder = metalCommandBuffer!.makeComputeCommandEncoder()
computeEncoder!.pushDebugGroup("Dispatch resize image kernel")
computeEncoder!.setComputePipelineState(resizeImagePipeline)
computeEncoder!.setTexture(srcTexture, index: 0)
computeEncoder!.setTexture(destTexture, index: 1)
computeEncoder!.setSamplerState(textureSampler, index: 0)
let threadGroupCount = MTLSizeMake(20, 10, 1)
let threadGroups = MTLSizeMake(destTexture!.width / threadGroupCount.width, destTexture!.height / threadGroupCount.height, 1)
computeEncoder!.dispatchThreadgroups(threadGroups, threadsPerThreadgroup: threadGroupCount)
computeEncoder!.popDebugGroup()
computeEncoder!.endEncoding()

The compute kernel is (remember, this is not a fragment shader which would automatically know how to set the mipmap level of detail):

kernel void resizeImage(
texture2d<half, access::sample> sourceTexture [[texture(0)]],
texture2d<half, access::write> destTexture [[texture(1)]],
sampler samp [[sampler(0)]],
uint2 gridPosition [[thread_position_in_grid]])
{
float2 srcSize = float2(sourceTexture.get_width(0),
sourceTexture.get_height(0));
float2 destSize = float2(destTexture.get_width(0),
destTexture.get_height(0));
float2 sourceCoords = float2(gridPosition) / destSize;
/*+ The following attempts all produced a pixelated image
(no edges smoothed out like a fragment shader would)
half4 color = sourceTexture.sample(samp, sourceCoords);
float lod = srcSize.x / destSize.x;
float lod = 0.0;
float lod = 1.0;
float lod = 2.0;
float lod = 3.0;
float lod = 4.0;
float lod = 4.5;
float lod = 5.0;
float lod = 5.5;
*/
float lod = 6.0;
half4 color = sourceTexture.sample(samp, sourceCoords, level(lod));
destTexture.write(color, gridPosition);
}

No matter what lod is set to, I get the same exact pixelated results. Why won't the mipmapping work?
Thanks for any help you can provide.

You havent' set the mipFilter property for your samplerDescriptor.  The default value for mipFilter is notMipped which means the sampler will never access anything other than level 0. Set it to nearest or linear.


PBR missing rim darkening on non metallic rough surfaces If you have a dielectric (non metallic) sphere with a large roughness value (say, 0.95), and put it in a lighting environment that has a constant onmidirectional lighting value (such as 0.5) Cook-Torrance ggx PBR renderers seem to be missing a darkening at glancing angles.

I thought this was a bug in my PBR implementation at first when comparing vs a mitsuba ggx render, but I've found the same issue in google filament (https://github.com/google/filament) as well as in substance.

For instance, below is the iray version from substance compared to the opengl version from substance. 0 metalness, 1 roughness, omnidirectional ibl of 0.5, albedo of (1,1,1).

Is this a known limitation of cook-torrance ggx based PBR? Can anyone explain why this limitation is here, and the reason for the difference?

I've noticed that it's much closer to the path traced "ground truth" at lower roughness values (like up to around 0.5-0.75), and that the most commonly interesting roughness values are near the lower end of roughness. 0.5 even looks almost "completely rough" for instance.  Is the issue that the approximations used don't really try to make these less important higher roughness values look correct?

The problem is in the diffuse term, which can be seen by making the specular portion of IBL not be added into the result.  The diffuse only render will not have the darkening, and of course, the specular can't make the diffuse darker. (Note, fresnel vs no fresnel is not the issue here).

The core problem is that lambertian diffuse is being used, which is view independent. There are other view independent diffuse terms which give results closer to the ground truth.

Two example diffuse terms are Burley (Disney) and Oren-Nayar.

Burley is talked about here: https://google.github.io/filament/Filament.md.html#toc4.5

An approach used in Titanfall is described here:
https://www.gdcvault.com/play/1024478/PBR-Diffuse-Lighting-for-GGX


OpenGL pipeline stages, when is the primitive assembly executed? The question is simple, I have this diagram from the Khronos page:

And this diagram from the Learn OpenGL page

In the first picture the Primitive assembly is executed for example after the Geometry shader, in the second the execution happens before.
For the latter picture I took the relevant bit from the tutorial:

The primitive assembly stage takes as input all the vertices (or vertex if GL_POINTS is chosen) from the vertex shader that form a primitive and assembles all the point(s) in the primitive shape given; in this case a triangle.
The output of the primitive assembly stage is passed to the geometry shader. The geometry shader takes as input a collection of vertices that form a primitive and has the ability to generate other shapes by emitting new vertices to form new (or other) primitive(s). In this example case, it generates a second triangle out of the given shape.

I'm a bit confused cause I can't figure what that module and the Geometry shader takes as input, I'd be more prone to say the first diagram is correct. But maybe there's an interpretation I'm missing here.
When is the Primitive assembly executed then?

For a diagram to be comprehensive, it'd have to show way too much detail. It's just a simple tool.
Primitive assembly is the process of taking a sequence of vertices for a particular rendering primitive type, and generating a sequence of base primitives (points, lines, triangles) from them.
This process does indeed happen after vertex post-processing, as the input to the rasterizer is a sequence of base primitives. After all, you don't rasterize a triangle strip; you rasterize a triangle.
However, the input to a geometry shader invocation is a single base primitive, taken from the sequence of vertices provided up to that point. As such, a form of primitive assembly also must happen before the GS, to convert the vertex sequence into a sequence of base primitives.
This is also one reason why GS's are slow. If there is no GS, then this primitive assembly doesn't happen.
The output of the geometry shader is a bunch of vertex sequences, which still later undergo primitive assembly based on the output primitive type of the GS. A GS can output triangle strips, so there has to be primitive assembly to turn those strips into individual triangles.


Transformations about a Plane I need to find the transformation matrix
(homogeneous coordinates) that flips an
object about a plane whose normal is
directed towards (-2 2 0), and intersects the
y-axis at y=2.

I know how to do that when I have 3 points on the plane (A,B,C): translate to origin, find the orthonormal basis of the plane (u,v,w axes) and then rotate the plane to match XYZ coordinate system (with the axes I found), perform the reflection, rotate back and translate back.

However in this case I have a normal and another data which I don't know how to use.

Any ideas?

This is the same problem as the rotation about an axis. What you need is a matrix (any matrix) that satisfies the following corner constraint: one axis must point toward $(-2, 2, 0)$ in the case of a rotation this matrix must be a rotation matrix, in the case of a planar mirror it just has to span the 3D space. But since we know how to make an arbitrary rotation matrix we use the same approach.

Pick any vector (u) that's not parallel to the normal (n)
Take cross product of u and n for b
Take cross product of n and b for a

You now have an arbitrary rotation matrix with one axis defined using vectors n, b and a. Otherwise same as the last question.


Applying correct light physics to gaussian blur formulas for glow I am trying to make a glow shader using separable gaussian blurring.

I have recently been inspired by the short youtube video "computer color is broken" and I have messed with it with color interpolation and boy his suggestion is beautiful!

A big thing the video talks about is that this principal should be applied to blurring, however I am pretty confused.  I don't really know what to square when and what to sqrt when when values are being added. My current theory is each texture sample for the gaussian blur gets raised by the power of two weighted with a bell curve and added to a sum like usual. At the end the sum is square rooted, but i'm not sure if that is correct. Could someone please confirm? Would this make an appreciable difference that made things worth it?

Yes, your theory is correct. A gamma-correct blur entails converting the input pixels to linear color space, performing the blur weighting and accumulation in that space, and then converting back to gamma space at the end.

As noted in the comments, the actual transform is not literally squaring and square-rooting, that's just an approximation (and not that good of one). For the true sRGB gamma transform, see the equations in this Wikipedia article (look down the page for the equations involving $C_\text{srgb}$ and $C_\text{linear}$).

By the way, some visual comparisons of gamma-correct and gamma-incorrect blurs can be found on this page by Elle Stone, which shows why this whole thing matters.


Cohen-Sutherland Clipping Specify individually the translation and scaling matrices required to transform a 2D
window of [Xmin=-234, Ymin=156] and [Xmax=66, Ymax=456] to a display viewport
of [Umin=45, Vmin=35] and [Umax=245, Vmax=185].

Ignore the question above since I solved the matrix, the information is just relevant for the question I'm stuck on

I was asked to compute the view-port positions (U1,V1) and (U2,V2) for two points A (-100,300)
and B (30,-40) in a 2D window and determine if these two points are inside the view-port.

Based on the transformation Matrix I found (U1,V1) to be (403/3, 407) and (U2,v2) to be (221, -103). It turns out that both these points are outside our view-port but part of the line between them is inside.

Now I'm confused about this part below:

Apply a 2D clipping method to the line segment between the two points A and B as
given in above.

My Attempt

delta x = 221-(403/3) = 260/3

delta y = -103 - 406 = -510

m = delta y / delta x = -5.88

I started with U1,V1 since it is above our viewport:

Y = 185

X = 403/3 + (185-407)*(delta x/ delta y)

X = 279.48

Point 1 - (172, 185)

Is this correct? Since the point is now within the view-port. Do I then do the same for the second point?

Taken from Wikipedia's article on Cohen-Sutherland:

  Both endpoints are in different regions: in case of this nontrivial situation the algorithm finds one of the two points that is outside the viewport region (there will be at least one point outside). The intersection of the outpoint and extended viewport border is then calculated (i.e. with the parametric equation for the line), and this new point replaces the outpoint. The algorithm repeats until a trivial accept or reject occurs.

So yes, since your new first point is inside the viewport, but your second point is outside, the situation is non-trivial (i.e. no "go render it all" or "go discard it all"), it will repeat with the second point. Afterwards, both points will be inside the viewport, therefore it is a trivial draw and no further action is needed.


In Ray-Sphere intersection is $b=2*(O-C) \cdot dirv$? In Ray-Sphere intersection is $b=2*(O-C) \cdot dirv$?

Where $dirv$ is the Ray direction vector. $O$ is origin and $C$ is center of the sphere.

I've also seen this without the $2$, i.e. $b=(O-C)\cdot dirv$.

Particularly here it's calculated like:

// Compute:
//    d = L - E ; // Direction vector of ray, from start to end
//    f = E - C ; // Vector from center sphere to ray start
const Vector &d = ray.direction ;
Vector f = ray.startPos - this->pos ;
real a = d • d ;  // a is non-negative, this is actually always 1.0
real b = 2*(f • d) ;
real c = f • f - radius*radius ;

real discriminant = b*b-4*a*c;

TL;DR $b = 2(f \cdot d)$

I'll try to derive it and then we can see what $b$ should be. If the sphere is centered at point $a$ with radius $r$, its equation is $|p - a| = r$, where $p$ is a point on the sphere. $|p - a|$ is the distance from a point $p$ to the centre of the sphere, and if that is equal to the radius, then it's a point on the sphere.

We'll assume the centre of the sphere is at the origin, so $a = 0$ and we have $|\vec{p}| = r$. Squaring both sides gives $|\vec{p}|^2 = r^2$ which I'm going to write as $$\vec{p}^2 = r^2 \tag{1}$$

Ray has equation $$x + td\tag{2}$$ where $x$ is the endpoint, $d$ is the direction, and $t$ is the parameter.

If the ray doesn't intersect the sphere, we will not be able to find a $t$ satisfying the ray equation. In the quadratic equation the discriminant (the bit under the square root) will be $< 0$. If it grazes the sphere (i.e. tangent) then we'll have a single point of intersection and a single $t$ and the discriminant will be $0$. If it goes through the sphere, we'll get a $t$ for the sphere entry, and a distinct $t$ for the sphere exit and the discriminant will be $> 0$. We need to plug $(2)$ into $(1)$ and solve for $t$:

$$
\begin{alignat}{2}
  &&(x + td)^2  &= r^2\\
  &\Rightarrow\quad
  &(x + td)(x + td)  &= r^2\\
  &\Rightarrow\quad
  &x \cdot x + 2t(x \cdot d) + t^2(d \cdot d)  &= r^2\\
  &\Rightarrow\quad
  &(d \cdot d)t^2 + 2(x \cdot d)t + (x \cdot x - r^2) &= 0 \tag{3}\\
\end{alignat}$$

Equation $3$ is a quadratic expression in $t$ (i.e. $at^2 + bt + c = 0$) where $$a = (d \cdot d)$$ $$b = 2(x \cdot d)$$ $$c = (x \cdot x - r^2)$$

This matches the code you posted, but the code is allowing spheres to be centered at an arbitrary point C. The code then has this line:

// f = E - C ; // Vector from center sphere to ray start

which effectively translates the endpoint of the ray E to a new endpoint f so that we can now consider the sphere to be centered at the origin. The ray direction is unchanged.


How do you scale an image across planes while retaining perfect tiling? Scaling up or down a texture is quite simple, modify the UVs.

In my particular scenario though, I have multiple "quads" (it is a patch tessellation shader) each with a tiling texture. Does anyone know of a way to change the texture scale, while retaining the tiling in the shader?

Is precomputing/scaling the uvs on the CPU the only option?

Example of (traditional) tiling problem:

One solution would be to use repeating texture wrapping, and then use a texture transformation matrix. So let's say you normally set the texture coordinates of your GL_TEXTURE_2D to be 0-1 in u and v. Instead, ensure that your GL_TEXTURE_WRAP_S and GL_TEXTURE_WRAP_T are set to GL_REPEAT or GL_MIRRORED_REPEAT and set the texture coordinates to be the tile number to tile number + 1. So the first one would still be 0 to 1. The second one would be 1 to 2, the third would be 2 to 3, etc.

Next, your vertex shader would have a mat4 uniform called textureMatrix or something like that. The texture matrix you pass in could have scaling in it (and probably other transformations, too). You just multiply the texture coordinates for the polygons by the texture matrix in the vertex shader, and it should apply appropriately.


What are forms of affine transformations? Please help me tackle the below question:

  Explain THREE forms of affine transformations using relevant examples for each case.

The answers I want to see are scaling, rotation and reflection.

Note: I have answered before the edit from trichoplax and I thought you were searching for other transformations other than the one you mentioned. The informations below are still useful so I will keep the answer here, but it does not directly answer your question.

Affine transformations (surprise!) map affine spaces to affine spaces. An affine space is substantially a vector space where you can establish an origin and define points as tuples of their coordinates. This is far from a formal definition, if you are interested in one I can edit the answer.

Now an affine transformation $T$ transform points into points and must preserve affine combinations:

$T(\lambda_1 P_1 + \lambda_2 P_2 + ...  +  \lambda_n P_n) = \lambda_1 T(P_1) + \lambda_2 T(P_2) + ... + \lambda_n T(P_n)  $

Given $\sum_{i}^{n} \lambda_i = 1 $

However these transformations can't be arbitrary as the following must be preserved:

Parallelism is preserved. This means that if you transform parallel lines they remain parallel after the affine transformation.
Relative ratios are preserved. This means that if you have $R = (1-\beta) P + \beta Q$  then $T(R) = (1 - \beta) T(P) + \beta T(Q)$
Addition between vector and points are preserved. Meaning that $T( P + \vec{v} ) = T(P) + T(\vec{v})$

With these properties in mind you can come up with a very big number of affine transforms yourself. A couple of obvious ones other the ones you mentioned are:

Translation

$$T_vp =
\begin{bmatrix}
1&0&0&v_x\\0&1&0&v_y\\0&0&1&v_z\\0&0&0&1\end{bmatrix}
\begin{bmatrix}
p_x+v_x\\p_x+v_y\\p_z+v_z\\1\end{bmatrix}=p+v$$

That moves a point into a specific direction by a specific amount specified by a displacement vector.

Shearing

image from wikipedia

That is a transform that displace all points in a given direction by an amount that is proportional to their perpendicular distance to a line parallel to that direction.

For example the transform matrix for an horizontal shear in a 2D space is given by:

$$
\begin{bmatrix}x'\\y'\end{bmatrix} = \begin{bmatrix}x+my\\y\end{bmatrix} =
\begin{bmatrix}1&m\\0&1\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}$$

Again, these two are just example, the important information you should really keep is the definition (and properties) of an affine transform; with that in mind it shouldn't be too hard to recognize an affine transform.  Also note that combining affine transforms will give you an affine transform!


Opengl black screen for a simple translation i am a beginner to opengl following an online guide,
I am trying to make a simple translation to my triangle.
The triangle did render correctly when i made a float uniform to translate it, but when i multiply a mat4 to translate it, i get a black screen.
Opengl outputs no error, i can't tell where is the problem.
Here is the code:
https://pastebin.com/en2zpPTh

Here's my drawing loop:

//Loop until window close
while (!glfwWindowShouldClose(mainWindow))
{
    //Get + Handle user input events
    glfwPollEvents();

    if (direction)
    {
        triOffset += triIncrement;
    }
    else
    {
        triOffset -= triIncrement;
    }

    if (abs(triOffset >= triMaxOffset))
    {
        direction = !direction;
    }

    //Clear window
    glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
    glClear(GL_COLOR_BUFFER_BIT);

    //Draw
    glUseProgram(shader);

    glm::mat4 model;
    /*
    [1,0,0,0]
    [0,1,0,0]
    [0,0,1,0]
    [0,0,0,1]
    */
    model = glm::translate(model, glm::vec3(triOffset, 0.0f, 0.0f));

    glUniformMatrix4fv(uniformModel, 1, GL_FALSE, glm::value_ptr(model));

    glBindVertexArray(VAO);
    glDrawArrays(GL_TRIANGLES, 0, 3);

    glBindVertexArray(0);
    glUseProgram(0);

    glfwSwapBuffers(mainWindow);

    return 0;
}

and here's my vertex shader:

// Vertex Shader
static const char* vShader = " \n"
"#version 330                  \n"
"layout (location = 0) in vec3 pos;\n"
"uniform mat4 model;                        \n"
"void main()\n"
"{\n"
"   gl_Position = model * vec4(pos, 1.0);\n"
"}\n";

And fragment shader:

//Fragment shader
static const char* fShader =  " \n"
"#version 330                  \n"
"out vec4 color;               \n"
"void main()\n"
"{\n"
"   color = vec4(1.0, 1.0, 0.5, 1.0);\n"
"}\n";

We fixed it ! it was glm::mat4(); no initializing to a identity matrix, glm::mat4(1.0f); works fine, thank you for helping me !


Which graphics APIs texture formats are HDR texture formats? I understand that HDR files are images encoded with a higher precision in order to keep more details.
When loading such images in real-time rendering applications using common Graphics APIs (DirectX, OpenGL, Vulkan), which are the "HDR texture formats" that ensure details will be preserved?

Also consider these alternatives for reducing bandwidth in comparison to R16G16B16A16.

DXGI_FORMAT_BC6H_UF16 is block compressed fp16 (half) RGB.

  https://docs.microsoft.com/en-us/windows/desktop/direct3d11/bc6h-format

DXGI_FORMAT_R11G11B10 unsigned small float

  https://bartwronski.com/2017/04/02/small-float-formats-r11g11b10f-precision/

Common image file formats for HDR are exrs or dds. They're pretty trivially encoded as 32-bit or 16-bit per channel floats.


Anisotropic GGX BRDF implementation. How is it related to isotropic GGX BRDF? Introduction

I am implementing anisotropic GGX BRDF and have encountered strange behaviour of my implementation. I thought that if I compare the microfacet distribution function I have with the one of an isotropic GGX, then they should be equal when the anisotropy parameter is 0. I haven't been able to do that, though.

Here is the GGX formula I have used:

Here is the anisotropic GGX formula I have used:

If anisotropy is 0, then  is equal to , therefore I can get the following from the second formula:

Problem 1

The problem now is that the third formula can never match the first one because of the negative 1 in the first one.

Problem 2

The specific issue I have when rendering using my anisotropic GGX is that the normals seem to be ignored in the result.

Here is a visualization of the distribution function of isotropic GGX on a flat material patch with normal mapping:

And here is the anisotropic one:

Notation

To complete the explanation of my solution, I use  for the half-vector,  for the normal,  for roughness,  for the tangent and  for the bi-tangent (in my case these are simply aligned with the x and y axes respectively).  and  represent roughness in the corresponding directions.

Questions

Is there a mistake in my reasoning? Is my anisotropic GGX correct?
What is the relationship between the two? Is there a simple explanation for the extension from GGX to anisotropic GGX?
Do you have any general tips for verifying the correctness of one's BRDF implementation?

References

Here are my main references:

Anisotropic GGX implementation
BRDF formulas

After coming back to the problem now with a fresh pair of eyes, I have managed to find my mistake.

Ad Question 1

Yes, the formula is correct. The problem is hidden in the tangent  and bi-tangent . When performing normal mapping, the tangent frame has to be perpendicular to the mapped normal, otherwise the term
 will not carry any information about the normal mapping.

Another issue here might be that once the normal is mapped, the tangent frame is not uniquely determined. Some more info on that can be found here: http://www.thetenthplanet.de/archives/1180. I have decided to solve this the same way as Blender does (look for make_orthonormals_tangent() in their repo) because I need to be compatible with it. Here is my algorithm for that:

bitangent_ortho = norm(cross(normal, tangent))
tangent_ortho = cross(bitangent_ortho, normal)

(Assuming that the normal is mapped and has unit length and the tangent was a valid tangent before the normal was mapped.)

Ad Question 2

This was indeed well answered by user @lightxbulb in a comment to my question. My only problem with the "Physically based shading at Disney" course notes was that the variable symbols weren't very clearly explained. I have since learned that the convention is the following:

 is the angle between the normal and
 is the azimuthal angle between the tangent and

Ad Question 3

The only pieces of advice that were useful for me here were the following:

Always come up with the simplest toy examples possible
Verify your assumptions
Visualize everything

Conclusion

I hope this is helpful for somebody else than just me. Good luck implementing BRDFs!


An algorithm to find the area of intersection between a convex polygon and a 3D polyhedron? Suppose I have a set of points in 3D which are all co-planar, and which describe the vertices of a convex polygon. I know the coordinates of all of these vertices, I know the unit normal to the plane of the polygon, and I know the order (in a right-hand sense with respect to the normal) in which the vertices go around the polygon.

I also have a polyhedron, which is in general non-convex, and the surface of which is made of triangles. It is a triangle mesh like the one shown here. It describes a completely enclosed volume (no holes in the surface). I know the coordinates of all the vertices of all the triangles that make up the polyhedron. For a given vertex, I know the list of which triangles it is a vertex of.

One other thing I know, which may or may not be useful, is that the polygon lies "in the vicinity of" a particular pair of vertices of the polyhedron, say the ones with indices $i$ and $j$. So if there were to some kind of a search of the surface of the polyhedron, that would be the place to start.

My task is to find the area of the intersection between the polygon and the polyhedron. I believe this is a well-defined quantity. I need an algorithm to calculate this, hopefully efficiently but at this point I'll take anything.

My thoughts:

I can classify each polygon vertex as either inside or outside the polyhedron. I've written a ray-casting algorithm that computes this fairly accurately and efficiently. That means that each edge of the polygon can be classified as either "both vertices inside", "both vertices outside", or "one in, one out." I can calculate the point of intersection between a given polygon edge and the polyhedron by checking a bunch of nearby triangles.

But that's not enough. For instance, just because both vertices of an edge are inside doesn't mean the whole edge is inside. A "finger" of the polyhedron might intersect an edge without touching either of its endpoints. Furthermore, the polyhedron might intersect the polygon in such a way that it doesn't touch any of the edges at all. Clearly, one must also calculate the intersections between the triangles' edges and the polygon as well.

It gets a bit too complicated for me at that point, and I'm wondering if there is some established algorithm for this?

Perhaps something like: Project the polyhedron onto the plane containing the polygon and then... I don't know.

There are different numerical approximations you could use:

A simple solution is to use brute-force Monte Carlo integration. Distribute $N$ random points on the polygon and calculate the number of points inside the 3D polyhedron $N_i$ using ray tracing. The area inside the polyhedron is $A_i=A\frac{N_i}{N}$, where $A$ is the area of the polygon.

To improve convergence of the above algorithm, you can distribute the points on grid on the polygon and jitter the sample positions within each grid cell. This is called stratified sampling.

Further improvement would be to perform the MC integration in 1D instead of 2D. Pick a sample line $L$ that's on the polygon plane and a sample direction $D$ that's on the plane perpendicular to $L$. Select sample line segment $L_s$ that bounds the polygon within $D$-directed lines (min & max of the polygon projection on $L$ for tight fit). For each sample on $L_s$ first calculate the line segment $S$ that intersects the polygon in direction $D$. Next calculate the length of $S$ that's inside the polyhedron $S_i$. You can calculate $S_i$ using ray tracing, by starting from one end of $S$ and marching along $S$ to intersection points of the segment with the polyhedron, summing up the segments of $S$ inside the polyhedron. More specifically, you need to first check if the start point on $S$ is within polyhedron and set the "inside" state accordingly. Then for each intersection flip the state and add the length from previous intersection to $S_i$ if the state changes from inside to outside. The approximation of the polygon area inside the 3D polyhedron is:
$$A_i=\frac{|L_s|}{N}\sum_{i=1}^N S_i$$


Moving a vertex through the cursor There is a way to move a selected point in a mesh with a cursor(assuming a camera that doesn;t change between frames).
The way I remember the algorithm (but seems to be wrong) is:

Unproject the cursor position onto the world position
Use this and the camera position to create a ray $R$
for every vertex in the mesh, see if $R$ intersects a small sphere centered at the vertex.
If you find a valid vertex, mark it as selected
project the selected vertex onto the screen using the view_projection transformation of the camera
store the resulting depth (z component) into a variable.
Now when you move the mouse, you create a fabricated position consisting of the cursor position and the stored projected depth, then you unproject that.

The resulting point should be constrained on a plane in 3D that is parallel to the screen. I have coded the procedure but it seems I am mathematically wrong. So I am not entirely sure if I forgot a step.
To be clear, the objective is to drag the mouse around and have the selected point move in 3D such that it is rendered at the same position as the cursor is on the screen.

Figured it out, I did skip a step, you are supposed to multiply by the homogeneous coordinate.
Basically the actually projected point is (courtesy of GLM):
        vec<4, T, Q> tmp = vec<4, T, Q>(obj, static_cast<T>(1));
        tmp = model * tmp;
        tmp = proj * tmp;

        tmp /= tmp.w;

Kinda obvious in hindsight.


Confused about prefiltering environment maps (Manson 16) I'm working my way through this paper about GGX-filtering environment maps:

http://www.ppsloan.org/publications/ggx_filtering.pdf

I find myself stuck on the basics.  In particular, I am confused by two statements in section 1.1, Related Work:

  Most recent reflection models assume
  dielectric materials, and in the case of a metal would simply multiply
  the specular color by an auxiliary specular color map.

This sentence immediately follows an overview of old-school mirrored torus environment mapping, which it refers to as "metal".  However, I'm not sure what the author means by specular color here -- incident light or material color.  Likewise, what is this secondary map that is referenced.  Is that the metalness map from the currently popular PBR texturing workflow?  Or is "auxillary specular color map" a cubemap?

  Importance
  sampling is inefficient for texture filtering because raster images
  are band-limited, which results in the peak of the function being
  oversampled

I don't understand this at all.  I am still learning importance sampling via Peter Shirley's weekend book, so I kind of get the difference between sampling a function with infinite resolution and sampling a down sampled raster grid... but I don't know what the author means by band-limited and how that produces the peak oversampling effect.

Sorry if these questions are overly naive!  I'm going to try to read (Kautz 03) now to hopefully fill in some gaps.  My ultimate goal is to write the simplest possible dynamic environment mapping demo to show an after school class the connection between offline global illumination and high-end realtime rendering.

A side goal is to synthesize for myself a history of these techniques from the end of this article until today:

http://www.pauldebevec.com/ReflectionMapping/

Your first quote is referring to "Split-sum approximation" presented in "Real Shading in Unreal Engine 4" by Brian Karis, and also referred in the paper [Kar13]:

$$\frac{1}{N}\sum_{k=1}^N \frac{L_i(l_k)f(l_k,v)cos\theta_{l_k}}{p(l_k,v)}\approx \bigg(\frac{1}{N}\sum_{k=1}^N L_i(l_k)\bigg)\bigg(\frac{1}{N}\sum_{k=1}^N \frac{f(l_k,v)cos\theta_{l_k}}{p(l_k,v)}\bigg)$$

In split-sum approximation the incident radiance (cubemap) is pre-convolved with GGX kernel using for example importance sampling, and normal-incident specular reflectance ($F_0$ or "specular color" you refer to) is multiplied with another pre-convolved 2D look-up-texture (referred as "an auxiliary specular color map") shown below.

Next about your second question of importance sampling being inefficient due to band-limited signal - It simply means that the incident radiance is sampled with finite resolution (cubemap resolution) and thus higher frequencies have been filtered out. But because importance sampling biases samples to important regions of the function (peak of the GGX BRDF) this region may be sampled in unnecessarily high frequency thus wasting samples.


How to pass a 2D array to a function How do you allow a function to accept a reference of a double array ? For example:
MyFunc(uint x, uint y, float2[][] myArray) {}

I get unexpected [ error does it allow reference passes like this?
I am trying to pass this to my function:
float2 _switchBuffer[2][128]
float2 _switchBuffer[2][256]`

This is declared as one of my properties that the entire shader has access to so i need the kernel to pass the relevant switchbuffer to the functions.
So i have something like:
void FFT128 (uint3 id : SV_DispatchThreadID)  { Test(id.xy, 3, _switchBuffer128); }
void FFT256 (uint3 id : SV_DispatchThreadID)  { FFT1D(id.xy, 3, _swotchBuffer256); }

Unsized arrays aren't implemented in HLSL. You can pass fixed-size arrays into functions like
void foo(float2 x[2][4])
{
    ...
}

(Note the brackets go after the parameter name, not after the type—just like declaring a variable.)
However, having different array sizes that can be passed into the same function won't work in HLSL the way it would in C/C++. For variable-size data, better to use a buffer resource, such as a constant buffer, structured buffer, etc (depending on what you're doing with this data). Or if it's in groupshared memory, you'll have to specify a maximum size and allocate the array for that.


Path tracer isn't converging So I just completed a naive path tracer that uses explicit direct light sampling.
Problem is the path tracer isn't converging. It's like a raytracer, showing a single image on each pass. More info on how am I doing this is provided below. I am using OpenCL 1.1.

  __kernel void render(__write_only image2d_t outputImage, __read_only image2d_t inputImage, int reset)
{
    float total_color;  // initalized to black
    for(int i = 0; i < samples_x; i++)
    {
        for(int j = 0; j < samples_y; j++)
        {
            float4 direct_color, indirect_color; initialized black.
            Shoot ray through pixel.

            HitInfo new_info, old_info;
            If( !intersects(ray, old_info) )
            {
                total_color += Background_color
                continue;
            }
            else
            {
                direct_color = computeDirectLighting(old_info);
                float4 color_arr[BOUNCES];
                float cosine_falloff[BOUNCES];
                for(int i = 0; i < BOUNCES; i++)
                {
                    new_ray = sampleHemisphere();

                    if( !intersects(new_ray, new_info) )
                    {
                        color_arr[i] = sky_color;
                        cosine_falloff[i] = max(dot(old_info.normal, new_ray.dir), 0.0f);
                        break;
                    }
                    else if ( intersects light source)
                    {
                        i--;
                        continue;
                        (Sample again)
                    }

                    color_arr[i] = computeDirectLighting(new_info);
                    cosine_falloff[i] = max(dot(old_info.normal, new_ray.dir), 0.0f);
                    old_info = new_info;
                }

                for(i = color_arr.length() - 1; i >= 0; i--)
                {
                    indirect_color += color_arr[i];
                    indirect_color *= cosine_falloff[i];
                }
            }

            total_color += direct_color + indirect_color;

        }
    }

    }
    color /= (samples_x * samples_y);

    if ( reset == 1 )
    {
        write_imagef(outputImage, pixel, color);
    }
    else
    {
        float4 prev_color = read_imagef(outputImage, sampler, pixel);
        color += prev_color;
        color /= 2.0f;
        write_imagef(outputImage, pixel, color);
    }
}

So this is the main outline. When the kernel runs the first time, reset = 1. Thus it writes the color directly into the output image. At each iteration, the buffers of outputImage and inputImageget swapped. After iteration 1, reset = 0.

This means at iteration 2, the kernel reads from the image it wrote to, in iteration 1. It then averages the new color and the previous color.

I thought taking 64 samples 2 times, and averaging them would be analogous to taking 128 samples once. If this is true then my image should converge, but it isn't. It shows only this image constantly.

EDIT:-
I think I should clear how I am calculating GI. After the first intersection. I send 1 ray directly to the light source (explicit direct light sampling). Then uniformly sample the hemisphere w.r.t area, and send another ray in this sampled direction. If the new ray hits light source, we sample again. This goes on bounces times. Each bounce the program stores the direct color due to explicit direct light sampling at each hit point. The indirect color is the sum of all these direct colors. Added a simple algo of what I did to the code snippet.

The cosine_falloff array holds all the weakening factors that should be multiplied to the respective indirect colors brought by secondary GI rays.

I can find two possible reasons for the image not converging.

#1. Every sample is the same

For every sample, you generate random rays. You do that when you shoot the ray through a pixel (for anti-aliasing and DoF) and when you sample the hemisphere (for a new indirect bounce). The problem would be if for every sample, it would generate the same direction, then every sample is the exact same colour and you would not be doing any converging.

Most of the times when generating random numbers on the GPU, we generate them in advance and save them to a texture. When we need a number, we just sample the texture. Of course, for every pixel you want a different random value, and you would then use something like the thread id or the pixel coordinate to sample the texture. Then, every frame must also be different, so you would also use the frame counter to offset the thread id. But, notice how there is nothing to make sure that every sample also has a new random number. If for every sample you generate the same random direction, then every sample is the exact same colour and you would not be doing any converging.

This can often easily be fixed by keeping track of how many random numbers you
already have generated and use that to offset every lookup. Basically, have a struct that holds the u and v values for the texture lookup. You initialize it with the thread id or pixel coordinate together with either the frame count or just a random number you generate on the CPU and pass it to the kernel as a global offset to the random values. Make sure that the struct is initialized in the very beginning of the kernel (where you initialize total_color). Then when you need a random number, you pass a reference to that struct to the method that generates a random number, it then looks up the random number in the texture and then it offsets the u and v values in the struct so that the next time you will get a different number.

Of course, I have not seen how you generate the rays (both on shooting a ray through a pixel and sampling a hemisphere), so I cannot say for sure if this is the problem, but it does seem very much like it.

#2. Not properly averaging multiple frames

  I thought taking 64 samples 2 times, and averaging them would be analogous to taking 128 samples once.

This is actually true. You average samples to get an image.
$$IMG1 = \frac{sample1 + sample2 + sample3 + samples4}{4}$$
Which you then do again for the second image.
$$IMG2 = \frac{sample5 + sample6 + sample7 + samples8}{4}$$
Which you then combine.
$$RESULT = \frac{IMG1 + IMG2}{2}=$$
$$\frac{\frac{sample1 + sample2 + sample3 + samples4}{4}+ \frac{sample5 + sample6 + sample7 + samples8}{4}}{2}=$$
$$\frac{\frac{sample1 + sample2 + sample3 + samples4 + sample5 + sample6 + sample7 + samples8}{4}}{2}=$$
$$\frac{sample1 + sample2 + sample3 + samples4 + sample5 + sample6 + sample7 + samples8}{8}$$
As you can see, it would indeed be like sampling twice as much.

However, your piece of code does not properly combine two images.

float4 prev_color = read_imagef(outputImage, sampler, pixel);
color += prev_color;
color /= 2.0f;
write_imagef(outputImage, pixel, color);

When we look at what you are doing.
$$outColor = \frac{newColor + oldColor}{2}$$
Where $outColor$ is the colour saved in the image.
$newColor$ is the colour you just calculated.
$oldColor$ is the color of the previous frame.

We see that it does what it is supposed to do, it averages both frames.

Of course, when the third frame comes around, $outColor$ becomes $oldColor$ as you swap the buffers of outputImage and inputImage. Thus, we can substitute it.
$$outColor = \frac{newColor + \frac{prevColor + oldColor}{2}}{2}$$
Where $prevColor$ was the color calculated the previous frame.

Although, this does not seem that good.
$newColor$ contributes 50%, while $prevColor$ and $oldColor$ only contribute 25%. This is a problem. You want every frame to contribute equally because then you are just adding up the samples. This means that you need to compensate for this.

There is an extremely simple solution to this. Multiply $oldColor$ by the amount of frames excluding the current frame and to then divide by the amount of frames including the current frame. you would then get this.
$$outColor = \frac{newColor + 2*\frac{prevColor + oldColor}{2}}{2 + 1}$$ Which would cancel out the divide and give you.
$$outColor = \frac{newColor + prevColor + oldColor}{3}$$ (The $+1$ is because we multiplied by 2 at the top, basically making it a total of 3 parts, and thus we need to divide by 3.)

If you would be calculating the fifth frame you would do it like this.

float4 prev_color = read_imagef(outputImage, sampler, pixel);
color += prev_color * 4.0f;
color /= 5.0f;
write_imagef(outputImage, pixel, color);

If you would be calculating the tenth frame you would do it like this.

float4 prev_color = read_imagef(outputImage, sampler, pixel);
color += prev_color * 9.0f;
color /= 10.0f;
write_imagef(outputImage, pixel, color);

This does mean that you need to pass through a counter which tells you how many frames you already have calculated. Most progressive or interactive path tracers actually only do one sample per frame (make sure that the ray through the pixel is randomly jittered), since it would take less time to calculate one frame, so that you see changes much earlier and often it is fine if you edit your scene using only one sample since you still get enough information.


Equivalent of Fresnel law for determining transmitted/diffuse split I understand that Fresnel tells us what percentage of light reflects vs refracts, but what law tells me the  percentage of refracted light that is transmitted versus the percentage of refracted light that is scattered back out as diffuse?

Physically, the origin of diffuse light is subsurface scattering, which happens continuously as light travels through a material. So, the proportion of transmitted light depends on the thickness of the object.

There's no precise equivalent to the Fresnel law, but maybe the closest thing is the Beer–Lambert law. It states that the transmitted light falls off exponentially with distance, proportional to $e^{-\sigma x}$, where $x$ is the distance the light has traveled through the material and $\sigma$ is the extinction coefficient (with units of inverse length). The non-transmitted portion of the light may be partly absorbed, and partly scattered through the material.

You get the appearance of "diffuse" when typical length scale of scattering is smaller than you can see ($1/\sigma \ll  1\text{ pixel}$). Then the light gets scattered so many times in such a short distance that by the time it comes back out, its distribution looks isotropic (Lambertian). The apparent diffuse color is controlled by the wavelength-dependence of the scattering and attenuation coefficients.

When the scattering length scale is larger, you get a soft appearance to the material (as in wax, marble, human skin, etc), or partial translucency.


Is the laplacian operator for meshes just the sum of the differences of neighbouring vertices? I am starting to learn about the laplacian operator $\Delta = \nabla \cdot \nabla\phi(p)$
Which can be described as the divergence of the gradient of a scalar function $\phi$. This is equivalent to the sum of the second order partial derivatives of the function.
However, it seems its graph equivalent is just the sum of the differences of adjacent vertices. Am I reading that right? In my head that should be equivalent to the first order partial derivatives, not the second order ones.
(I didn't know whether to ask here or in math, I decided here is more relevant because this is in the context of geometry processing)

You can see why the sum of differences is giving a second derivative if you look at the derivative of a one-dimensional function $f$. The derivative of $f$ at $x_0$ is
$$ \frac{df}{dx}(x_0) \approx \frac{f(x_0 + dx) - f(x_0)}{dx} $$
The second derivative is then
\begin{eqnarray*}
\frac{d^2f}{dx^2}(x_0) &\approx&
  \frac{df/dx(x_0) - df/dx(x_0-dx)}{dx} \\
  &\approx& \frac{\left(f(x_0+dx) - f(x_0)\right) -
                  \left(f(x_0) - f(x_0-dx)\right)}{dx^2} \\
  &=& \frac{\left(f(x_0+dx) - f(x_0)\right) +
                  \left(f(x_0-dx) - f(x_0)\right)}{dx^2},
\end{eqnarray*}
i.e. the sum of the local differences, scaled by a distance factor $dx^2$. In a graph Laplacian, you don't have any distances, just graph connections, so it's just the sum of the local differences.
When working on meshes, though, we typically want to take the geometry of the mesh into account. There are many ways to do this, but the most popular seems to be the cotangent Laplacian, where the local difference for each edge is weighted by the sum of the cotangents of the angles opposite that edge.


Making low discrepancy sequence noise textures, not LDS sample positions When making sampling locations, there is a lot of information out there on how to do white noise sampling, uniform sampling, uniform+jittered sampling, low discrepancy sequence sampling (eg halton) and blue noise sampling (eg mitchell's best candidate algorithm Mitchell's Best Candidate Algorithm).

For instance these, where each point is a sample location in a 2d quad. (source: https://blog.demofox.org/2017/05/29/when-random-numbers-are-too-random-low-discrepancy-sequences/)

White Noise:

Uniform:

Uniform + Jitter:

Halton (2,3):

Blue Noise:

While those are good for sampling (a 2d quad in this case), here is a blue noise texture which is useful for dithering or stippling (src: http://momentsingraphics.de/?p=127)

Similarly, here is a white noise (random) one:

I've been trying to figure out in this case (a dither / stippling noise texture), if there is an equivalent to uniform sampling, uniform + jitter, or low discrepancy, and if so, what those textures might look like.

One obvious difference between the first set of the textures and the second is that the first textures take a single scalar value as input (an index) and give a 2d vector as output (a location on the 2d quad).  The second set of textures takes an (x,y) 2d vector location as input, and gives out a scalar value as output.

It almost looks though like in the blue noise case that maybe you could start with the blue noise sample points, and then color every other pixel based on it's distance from the closest sample point, and that you might come up with the same image.  If that's correct, could you do the same for the other flavors of sample distributions to make the other noise textures?  It seems like white noise (random) wouldn't work for this though, so that doesn't seem correct.

Lastly, Jorge Jiminez has some "Interleaved Gradient Noise" that looks sort of blue noise-ish but kind of not... so seems a bit low discrepancy sequence. (http://www.iryoku.com/next-generation-post-processing-in-call-of-duty-advanced-warfare  and also an image and the math mentioned here https://bartwronski.com/2016/10/30/dithering-part-three-real-world-2d-quantization-dithering/)

It makes me wonder if something about that noise shows the secret on how to generalize (low discrepancy) sample points to 2d?

Does anyone know whether there is a way to translate these sample concepts to noise textures for dithering and stippling?

In theory, it is possible to stuff every sample distribution into a texture to "pre-bake" it for fast access. The question is whether any of the results might be useful. For blue noise, this makes a lot of sense, as blue noise distributions have global influence and are hard to evaluate at runtime. Uniform random sampling, on the other hand, is so simple precisely because it is not global, but has no influence whatsoever from neighboring samples. So the best you can achieve, as you noticed yourself, is writing uncorrelated white noise to the texture, which gains you very little.

I think the key property of the blue noise textures that you failed to mention is that they are progressive. If you take, for example, Blue Noise Masks, Forced Random Dithering, or Relaxation Dart Throwing to generate a matrix/texture, you will not get grayscale values, but unique, ascending integer values corresponding to the order of insertion of the samples/texels. This means that you can also re-arrange such textures to a 1D list of width * height elements, where each element is a 2D position within the matrix. I.e., the first element of this list will be the (x, y) of the first sample inserted into the texture upon creation. The second element will be the (x2, y2) of the second sample, which will be quite far away from the first one, because the blue noise texture algorithms try to maximize distances between similar values.

The list representation is nothing but a sequence accomplishing what 2D low-discrepancy sequences do: giving you a 2D sample position (albeit discrete and in a grid) for a scalar index. You do not lose any information while converting. So applying the reverse conversion to low-discrepancy sequences, it should be possible to write their element indices into a texture for fast lookup. What I don't know is how fast this will degenerate, because sooner or later higher values will end up at the same discretized locations as previously written ones. Low-discrepancy sequences guarantee spatial uniformity on average, but if you want to fill an n² texture and create exactly n² samples, I bet that you will get some of these overlaps while also having some holes. These conflicts can, by design, not occur in blue noise textures, but here you would have to find a strategy. Just discarding the higher values? I don't know how this influences the distribution properties. I assume that results vary between different low-discrepancy sequences. I have only ever worked with Halton and Hammersley, which don't seem like the best fit in this case. Maybe something based on a grid/lattice would help you to reduce/eliminate the overlaps. Not my field of expertise, unfortunately.

This is the key problem, that low-discrepancy sequences (at least the ones I know) and in general sampling distributions used in computer graphics output nD values in the continuous domain rather than in a discrete grid. The reason blue noise textures work so well is because the discretization is already part of the generation. What you want is a permutation of n² integer values such that the result has special properties. Maximizing distances between similar values would be blue noise. Having no correlation between values would be white noise. And some other magic property could be pink noise, green noise, or a perfect interleaved gradient noise as in your provided picture. Would this be a reasonable approach, to understand your problem as the problem of permuting given values? Instead of continuous sampling? Maybe it simplifies the solution.

Another point to consider for dithering and related effects is that blue noise textures can usually be tiled seemlessly because the sample domain is taken to be toroidal. You will lose this feature unless you find a way to make your low-discrepancy sequence toroidal, which I think is pretty tough, but would be a great scientific contribution!


computing derivatives of sampled data I am trying to calculate the derivatives of a sampled data set by following an example in this book. The data is a set of 2d quads(green area in below) and their vertices(4 each). The data is in the following example is sampled from $z=f(x,y)=e^{-(x^2+y^2)}$ but, in general, the underlying function might not be known.

The book states that the derivatives of any points are calculated by:

 are the quadrilateral basis functions:

and

I can calculate df/dx and df/dy.

But where I struggle is calculating df/dz. I assume

and therefore df/dz = 0. But this cannot be correct since df/dz should be 1(by making the above analytic function implicit and differentiating it for z).

Rather than fit bilinear patches to your sample points, which are only C0 continuous$^\spadesuit$ may I instead suggest fitting the surface using Catmull-Rom splines? These will give you C2 continuity.
Catmull-Rom is an interpolating spline which means it will pass through your sample points, as did the bilinear surface, but will do so smoothly from patch to patch.

$^\spadesuit$ because the first derivatives won't generally match at the boundaries of your patches

Background/Assumption

You have a function $$F:\Re^2\to\Re^3$$ of the form  $$F(s,t)=[s,t,f(s,t)]$$ where $f:\Re^2\to\Re$. The function, $f$, can be considered a 'black box' in the sense that we might not be given it explicitly.

$F$ has been regularly sampled as $$S_{i,j}=F(x_{scale}.i, y_{scale}.j, f(x_{scale}.i,y_{scale}.j))$$ where $i,j\in\mathbb{Z}$.  You then want interpolate between these sample points using piecewise bicubic Catmull-Rom patches.

Constructing patches

To form a particular bicubic Catmull-Rom patch, $C_{i,j}$, which has a given 2x2 set of samples $\{S_{i,j},  S_{i+1,j}, S_{i,j+1},  S_{i+1,j+1}\}$ at the corners, you need the set of 4x4 samples formed from the 2x2 at the centre and the surrounding 12 samples, i.e.$\{S_{i-1,j-1}$ through $S_{i+2,j+2}\}$,  as shown below.
(My apologies that this is 'hand placed' so is far from ideal)

If we just consider the behaviour of the surface at sample, $S_{i,j}$, then the partial derivative in the $i$ direction is given by
$$\frac{\partial C_{ij}}{\partial i}=\frac{S_{i+1,j}-S_{i-1,j}}{2}$$
and similarly, in the $j$ direction by
$$\frac{\partial C_{ij}}{\partial j}=\frac{S_{i,j+1}-S_{i,j-1}}{2}$$

Remembering these are vectors, you can use the cross product to generate a surface normal at any of the $S_{i,j}$ points$^\clubsuit$. (Though you may need an extra set of boundary samples)

  Ali: You mentioned that both partial derivatives are vectors(in i,j directions). could you write each vector components plz

Certainly: If we say $S_{i,j}=[x_{i,j},y_{i,j},z_{i,j}]^T$ then $$\frac{\partial C_{ij}}{\partial i}=\frac{1}{2}.[(x_{i+1,j}-x_{i-1,j}),(y_{i+1,j}-y_{i-1,j}),(z_{i+1,j}-z_{i-1,j})]^T$$
$$\frac{\partial C_{ij}}{\partial j}=\frac{1}{2}.[(x_{i,j+1}-x_{i,j-1}),(y_{i,j+1}-y_{i,j-1}),(z_{i,j+1}-z_{i,j-1})]^T$$

If you've sampled the surface 'finely' enough you could probably choose to render the surface by just using 2 or 4 triangles per original patch.

Alternatively, you can recursively subdivide your quad mesh to get a finer set of triangles. To double the number of points in the 'i' direction, simply insert a new point between each $S_{i,j}$ and $S_{i+1,j}$ that equals ...
$$\frac{9(S_{i,j}+S_{i+1,j}) - (S_{i-1,j}+S_{i+2,j})}{16}$$
 Once you have the finer grid in 'i' simply repeat with 'j' with the appropriate equivalent set of points. You can repeat this ad nauseam to get whatever triangle resolution you desire ;-)

$^\clubsuit$ (Copying some text from the comments below): I've made a couple of valid assumptions here.  Noting that if a given first (partial) derivative is non-zero (which is guaranteed in your case because of the way you construct the surface), then it also is locally tangential to that surface. Further, we know your two partial derivates aren't parallel, so the cross product will construct a valid normal.
FWIW For some arrangements of control points, i.e. in more general use cases, you can get so-called 'degenerate' surfaces where you can't naively apply the above rules, but you're safe here.


Environment map importance sampling (beginner) beginner here.
I am implementing importance sampling for environment maps in Scotty3D, a CMU CS project (but I am not a student).
The task can be found here
I have a problem with the sample's pdf, it is unusually small and that results in the object being rendered white even if its albedo is not white.
Here is how I construct the CDF:
// Calculate Lsintheta across the image. Accumulate to normalize the pdf to 1 later.
for (size_t j = 0; j < h; j++) {
    for (size_t i = 0; i < w; i++) {
        // 0.5f at the pixel center. Else we never sample top and bottom rows
        float Lsintheta = image.at(i, j).luma() * sin(PI_F*(j + 0.5f)/h);
        total += Lsintheta;
        cdf.push_back(total); // Joint Cdf. P(theta, phi)
    }
}

So total would have all the pixels' luminosities. cdf would actually be the cumulative luminosity.
Next is the sampling and calculating the actual pdf.
float Xi = RNG::unit(); // [0, 1]
auto lower = std::lower_bound(cdf.begin(), cdf.end(), Xi*total); // cdf is not normalized yet. Scale the RV
auto index = std::distance(cdf.begin(), lower);
out_pdf = *lower;
if (index > 0) out_pdf -= cdf[index - 1]; // pdf[index] = cdf[index] - cdf[index - 1]
out_pdf /= total;

Xi is a uniformly sampled real number from 0 to 1. Then it is multiplied by the total luminosity in the image to get the sampled accumulated luminosity. Use std::lower_bound to get the nearest accumulated luminosity greater than or equal to it. Find the index of this nearest accumulated luminosity using std::distance. Then minus with the previous accumulated luminosity to get the actual luminosity in this pixel. Divide by total to get the pdf.
This doesn't work, but if I just do out_pdf = lower/total it can at least show the right color.
What am I doing/understanding wrong?

From all other implementations I have seen, you generally sample twice, once in the Y dimension to select a row and then once in the X dimension to selct the pixel from this row.
I am not sure if this is necissary, maybe you're one dimensional sampling is fine.
I think you also need to multiply your pdf so that it is relative to solid angle (so it's total sums to 4pi).
You should be able to do this by dividing by (2 * Pi * Pi * sinTheta).
PBRT is probably the best textbook to help you. You can find it online here:
http://www.pbr-book.org/
The source code is also available. Here is the code for the environment light:
https://github.com/mmp/pbrt-v3/blob/aaa552a4b9cbf9dccb71450f47b268e0ed6370e2/src/lights/infinite.cpp
and the code for the Distribution2D sampler:
https://github.com/mmp/pbrt-v3/blob/aaa552a4b9cbf9dccb71450f47b268e0ed6370e2/src/core/sampling.h


Guidance with Sphere-Map calculation in GLSL I'm trying to achieve the same effect in OpenGL/GLSL as what Blender does when you add a texture, set to "Reflection"

This ultimately is a way to fake specularity, or "metallicness" if I get it right.

I followed this tutorial (Mid-page, "Sphere Mapping" section):
http://www.ozone3d.net/tutorials/glsl_texturing_p04.php

Here's the problem(some serious artifacts):
https://www.youtube.com/watch?v=hx66_xWhVu4&feature=youtu.be

Everything's happening at the bottom of the Vertex Sahder:

#version 330 core

layout(location = 0) in vec3 vertexPosition_modelspace;
layout(location = 1) in vec2 vertexUV;
layout(location = 2) in vec3 vertexNormal_modelspace;
layout(location = 3) in vec3 vertexTangent_modelspace;

out vec2 uv;
out vec3 toLightVector[4];
out vec3 toCameraVector;
out float fog_Visibility;

uniform mat4 FTM;
uniform mat4 P;
uniform mat4 V;
uniform vec3 LP[4];

const float fog_Density = 0.2;
const float fog_Gradient = 0.9;

void main(){

    vec4 worldPosition = FTM * vec4(vertexPosition_modelspace, 1.0);
    uv = vertexUV;

    vec3 surfaceNormal = (FTM * vec4(vertexNormal_modelspace, 0.0)).xyz;

    vec3 norm = normalize(surfaceNormal);
    vec3 tang = normalize((FTM * vec4(vertexTangent_modelspace, 0.0)).xyz);
    vec3 bitang = normalize(cross(norm, tang));

    mat3 toTangentSpace = mat3(tang.x, bitang.x, norm.x, tang.y, bitang.y, norm.y, tang.z, bitang.z, norm.z);

    for (int i = 0; i < 4; i++) {
        toLightVector[i] = toTangentSpace * (LP[i] - worldPosition.xyz);
    }

    toCameraVector = toTangentSpace * ((inverse(V) * vec4(0.0, 0.0, 0.0, 1.0)).xyz - worldPosition.xyz);

    vec4 positionRelativeToCam = V * worldPosition;
    gl_Position = P * positionRelativeToCam;

    // Fog
    float fog_Distance = length(positionRelativeToCam.xyz);
    fog_Visibility = exp(-pow((fog_Distance * fog_Density), fog_Gradient));
    fog_Visibility = clamp(fog_Visibility, 0.0, 1.0);

    // Reflection Map
    vec3 u = normalize(toCameraVector);
    vec3 r = reflect(u, norm);
    float m = 2.0 * sqrt(r.x*r.x + r.y*r.y + (r.z + 1.0)*(r.z + 1.0));

    gl_TexCoord[0].s = r.x / m + 0.5;
    gl_TexCoord[0].t = r.y / m + 0.5;
}

And in the Fragment Shader, all I do is:

out_color = texture(reflectionSampler, gl_TexCoord[0].st);

The reflection map seems to do it's thing similarly to the way it's seen in Blender, but not quite.

I need to be able to do this without calculating lights. Thats how Blender does it, it works Shadeless mode so all it should take is the camera's vector...

You probably need to calculate the reflection vector per pixel, not per vertex. The calculation is non-linear, so doing it per vertex and interpolating will not give good results unless the model is finely tesselated.

Send norm and toCameraVector down to the pixel shader, normalize them both and move all your calculations under // Reflection Map to the pixel shader. That should give you some better results.


Realistic rendering: which processes of the human eye and brain do I need to consider? Photorealistic rendering has the goal of rendering an image as a real camera would capture it. While this is already an ambitious goal, for certain scenarios you might want to take it further: render an image as the human eye would capture it or even as the human being would perceive it. You could call it visiorealistic or perceptiorealistic rendering, but if anyone could come up with a catchier term (or tell me that there is one already in existence) I'd appreciate that.

Here are some examples to make my point clear. When you take a picture with a camera at a low illumination level, you either have a good lens or get a noisy image. For a human observer, scotopic vision kicks in and gives rise to the Purkinje effect (colors are shifted towards blue). This effect depends on the HDR luminance information, which are lost when I display the image on a LDR display. In addition, the human brain may use depth information to 'filter' the perceived image - information which are lost in a final (non-stereo) rendering.

Assembling an exhaustive list is probably an elusive goal. Could you suggest some of the effects of the eye and the brain that I would need to consider?

you might want to take it further: render an image as the human eye would capture it or even as the human being would perceive it.

There are two ways to interpret this.  I'll do both.

Interpretation 1: Render an image that looks perceptually realistic.

At the end of the day, your image still needs to be displayed somewhere.  Here's the key: you want to render your image in such a way that when you *display* that image on a particular display device, it will produce the same sensation the original radiometric image would have produced.

Here's how to unpack that idea.

In the real world, radiometric spectra (i.e., real distributions of light) enter your eye and stimulate approximately1 four light receptors.  The stimulations of the receptors produce the sensations of color we associate with images.

In rendering, we don't have arbitrary control over the spectra we produce.  Fortunately, since we (usually) have only three cones, each of which produces only a scalar value, color vision can be reproduced by using exactly three primaries.  The bottom line is you can produce any color sensation by using a linear combination of three wavelengths only (up to a few colors that might have to be negative, in which case, you just use different primaries).

You don't have a choice of primaries.  Almost all color display devices use the sRGB standard, which provides three primaries (which actually usually don't have a single wavelength).  That's fine because it turns out it's all abstracted and you don't have to care.

To clarify the mess that is perceptually accurate rendering, here's the algorithm:

Render your image using correct radiometric calculations.  You trace individual wavelengths of light or buckets of wavelengths.  Whatever.  In the end, you have an image that has a representation of the spectrum received at every point.
At each pixel, you take the spectrum you rendered, and convert it to the CIE XYZ color space.  This works out to be integrating the product of the spectrum with the standard observer functions (see CIE XYZ definition).
This produces three scalar values, which are the CIE XYZ colors.
Use a matrix transform to convert this to linear RGB, and then from there use a linear/power transform to convert linear RGB to sRGB.
Convert from floating point to uint8 and save, clamping values out of range (your monitor can't represent them).
Send the uint8 pixels to the framebuffer.
The display takes the sRGB colors, does the inverse transform to produce three primaries of particular intensities.  Each scales the output of whatever picture element it is responsible for.  The picture elements light up, producing a spectrum.  This spectrum will be (hopefully) a metamer for the original spectrum you rendered.
You perceive the spectrum as you would have perceived the rendered spectrum.

Interpretation 2: Attempt to simulate the end data the human eye might receive for visualization purposes or compensation for LDR displays.

This one has a less useful meaning, I think.  Essentially, you're trying to produce an image that tweaks the way the brain perceives it for fun/profit.

For example, there was a paper at SIGGRAPH this year where they simulated afterimages and color reduction to make images appear perceptually different.  Of course, the only reason they do this at all is because the displays we're working with are all low-dynamic range (LDR).  The point is to simulate the effects someone might see if exposed to a real high-dynamic range (HDR) display as actual image data.

In practice, this turns out to not work very well.  For afterimages, for example, we see afterimages because of a very bright stimulus exhausting color cells.  If you instead try to stimulate the effect with a fake afterimage, it might look kindof similar--but since it's a completely different mechanism, it's not very convincing.

This sort of graphics is actually underexplored in the literature if you want to make a go at it.  The mentioned paper is an example of more-or-less the most state-of-the-art approaches we have.  I think the current consensus, though, is that it is not really worth trying to simulate (at least at this time), since at best you'd only be approximating real vision effects by substituting different ones, and that this doesn't really work.

1Rod+3*cones, the usual case.  Approximate because humans may have as few as zero functional light receptors up to a conjectured maximum of seven (with the highest ever observed being five).


Local Coordinates of Deformed Lattice I'm making a lattice based volume FFD and I'm having trouble with gaps in my data resulting from doing a typical deformer. I think I need to go through the voxels in the deformed lattice and then find the corresponding undeformed coordinate, that is, go from having the deformed position to getting the local (s,t,u) coordinate that was deformed to that position and then write the interpolated value of the original voxel. This is obviously the opposite way of doing a lattice based FFD with vector valued trivariate Bernstein polynomials.

I am struggling to figure out a way to get the local coordinates of a point in the deformed volume. If anyone has any advice it would be much appreciated.

Thank you!

Maybe something quick (to set up, no guarantees of actual speed) and dirty.

Whatever your method was, it probably boiled down to a vector function
\begin{align}
\begin{bmatrix} x,\, y,\, z\end{bmatrix} = F\Big([s,\, t,\, u] \,\, \Big| \,\, V_{ijk} \, : i,j,k = 1..N\Big)
\end{align}  where $V_{ijk} \in \mathbb{R}^3$are the vertices of the three dimensional lattice.

Basically, you currently have the new coordinates $[x,\, y,\, z]$ of a point, and you want to find the original ones $[s,\, t,\, u]$. I hope you still have the coordinates of the lattice vertices $V_{ijk}$. To retrieve the initial coordinates $[s,\, t,\, u]$ you have to solve for $[s,\, t,\, u]$ the system of equations
\begin{align}
\begin{bmatrix} x,\, y,\, z\end{bmatrix} = F\Big([s,\, t,\, u] \,\, \Big| \,\, V_{ijk} \, : i,j,k = 1..N\Big)
\end{align} where $[x,\, y,\, z]$ is given and the points $V_{ijk}$ are given
For simplicity, I am going to write
$$\begin{bmatrix} x,\, y,\, z\end{bmatrix} = F\big([s,\, t,\, u] \,\, \big| \,\, V \,\big)$$
To solve it, use a type of Newton's method.
Set a small enough number $h$.

Set your initial $s_0=x, t_0=y, u_0=z$ and set up your error equal to $err = 1$ (or whatever).

While $err \geq h^2$ execute the following iteration:

Assume you have a current approximation $[s_n,\, t_n,\, u_n]$;
Calculate the following three three-vector (as rows)
\begin{align}
&\Delta_s F_n = \frac{1}{h} \Big( \, F\big([s_n+h,\, t_n,\, u_n] \,\, \big| \,\, V \,\big) - F\big([s_n,\, t_n,\, u_n] \,\, \Big| \,\, V \,\big) \, \Big)\\
&\Delta_t F_n = \frac{1}{h} \Big( \, F\big([s_n,\, t_n+h,\, u_n] \,\, \big| \,\, V \,\big) - F\big([s_n,\, t_n,\, u_n] \,\, \Big| \,\, V \,\big) \, \Big)\\
&\Delta_u F_n = \frac{1}{h} \Big( \, F\big([s_n,\, t_n,\, u_n+h] \,\, \big| \,\, V \,\big) - F\big([s_n,\, t_n,\, u_n] \,\, \Big| \,\, V \,\big) \, \Big)
\end{align}
Set the following three by three matrix
$$M_n = \begin{bmatrix} \Delta_s F_n \\  \Delta_t F_n \\
   \Delta_u F_n\end{bmatrix}$$ (the vectors are the ones computed above but transposed and are therefore vector-columns);
Calculate the inverse matrix $M_n^{-1}$;
Calculate the update
$$[s_{n+1},\, t_{n+1},\, u_{n+1}] = [s_n,\, t_n,\, u_n] + \Big(\,[x,\, y,\, z] - F\big([s_n,\, t_n,\, u_n] \,\, \Big| \,\, V \,\big)\,\Big)\, M_n^{-1}$$
(all of this is written as vector matrix multiplication, where you are working with $1 \times 3$ row-vectors and multiplying such$1 \times 3$ row-vectors with the $3 \times 3$ matrix $M_n^{-1}$ on the right)
Calculate the error $$err = \Big\|\, [x,\, y,\, z] - F\big([s_{n+1},\, t_{n+1},\, u_{n+1}] \,\, \Big| \,\, V \,\big)\, \Big\|^2$$
Go back to the beginning of the iteration to check whether $err \geq h^2$

The result of this iteration is an approximation of the coordinates $[s,\, t,\, u]$. I hope this works.


How do I create my own bitmap using a text editor? I'm a Computer Science student who's been getting into bitmaps & pixmaps recently. I clearly understand what pixmaps do. A pixmap, as I understand, defines each pixel individually. So, maybe I could write a pixmap filelike this:

255,0,0 0,255,0 0,0,255

which would give me a red pixel, a green pixel, and a blue pixel.

Seems pretty straightforward.

However, I fail to understand exactly how bitmaps work. I mean, I understand that a bitmap simply maps a bit to some color at some point on the screen. I watched a YouTube video on how to create a bitmap file, but the guy's explanation of what a bitmap file is went over my head.

Edit: I feel I now have a better conceptual grasp of what a bitmap is. Well, maybe not. Each bit in a bitmap corresponds to a certain color that is defined somewhere in the file. But how does the computer know which bit corresponds to which color?

In addition to the pixel values, the width and height of the image are also required, and the colour depth (how many colours are available per pixel). Some formats also specify an alpha value, for transparency, which may be defined per pixel in addition to the red, green and blue components.

There are many different image file formats but a simple one to experiment with to gain understanding is ppm, one of the Netpbm formats.

Netpbm defines the following file types:

PBM: portable bitmap (pixels are either on or off)
PGM: portable graymap (grayscale pixels)
PPM: portable pixmap (full colour pixels)

These have the advantages of being simple to understand, and easy to generate in a text editor for working with simple examples.

For example, a PPM file is simply the characters "P3" to identify it as PPM, followed by the width, height, maximum colour value and the pixel values in English reading order (left to right then top to bottom). Each pixel colour is a red value, a green value, and a blue value. All these values are space separated and human readable.

P3                         # Indicates an ASCII PPM file
2 2                        # Width and height of the image
255                        # Maximum colour value
255 0 0     0   255 0      # A red pixel (255 0 0) followed by a green pixel
0   0 255   255 255 255    # A blue pixel followed by a white pixel

While this makes the format very easy to work with for small examples, it will also make the file size much larger than other formats which use bytes to represent colour components instead of writing out the human readable numbers. Many formats also include compression (either lossless or lossy).

Once you are accustomed to writing PPM files in ASCII characters, indicated by the file starting with "P3", you can move on to PPM files using bytes, indicated by the file starting with "P6". This is a similar format to what many programs use internally, but is no longer human readable in a text editor as not every byte will correspond to a printable ASCII character.

You also mention that a pixmap can have pixel values that refer to colours stored elsewhere in the file. I have avoided this format as it is not necessary for understanding how a pixmap works. This is known as using a palette (a list of predefined colours so each pixel is represented by an index of a colour in the palette). This is necessary for formats that only allow a small number of distinct colours, but for full colour images there is no need for a palette - each pixel is defined as red, green and blue components without having to refer to anything else.


Why do GPUs still have rasterizers? Despite advancements modern GPUs still have fixed rasterizers. Highly customizable, with programmable shaders but nevertheless not fully programmable.

Why is that?

Why can't GPUs be simply massively parallel devices with universal computing units where rasterizer is just a software for that device provided by the user?

Is having fixed function hardware so beneficial performance-wise that such approach is unfeasible?

In short, performance reasons are why they aren't programmable.

History and Market

In the past, there used to be separate cores for vertex and fragment processors to avoid bloated FPU designs. There were some mathematical operations you could only do in fragment shader code for instance (because they were mostly only relevant for fragment shaders). This would produce severe hardware bottlenecks for applications that didn't max out the potential of each type of core.

As programmable shaders became more popular, universal units were introduced. More and more stages of the graphics pipeline were implemented in hardware to help with scaling. During this time, GPGPU also became more popular, so vendors had to incorporate some of this functionality. It's still important to note though that the majority of the income from GPUs were still video games, so this couldn't interfere with performance.

Eventually a big player, Intel, decided to invest in programmable rasterizers with their Larrabee architecture. This project was supposed to be groundbreaking, but the performance was apparently less than desired. It was shut down, and parts of it were salvaged for Xeon Phi processors. It's worth noting though that the other vendors haven't implemented this.

Attempts at Software Rasterizers

There have been some attempts at rasterization through software, but they all seem to have issues with performance.

One notable effort was an attempt by Nvidia in 2011 in this paper. This was released close to when Larrabee was terminated, so it's very possible that this was a response to that. Regardless, there are some performance figures in this, and most of them show performance multiple times slower than hardware rasterizers.

Technical Issues with Software Rasterization

There are many issues that were faced in the Nvidia paper. Here are some of the most important issues with software rasterizers though:

Major Issues

Interpolation:
The hardware implementation generates interpolation equations in specialized hardware. This is slow for the software renderer since it had to be done in the fragment shader.
Anti-aliasing:
There were also performance issues with anti-aliasing (specifically with memory). Information regarding the sub-pixel samples must be stored on-chip memory, which isn't enough to hold this. Julien Guertault pointed out that the texture cache/cache may be slower with software. MSAA certainly has issues here because it overflows the cache (the non-texture caches) and goes into memory off of the chip. Rasterizers compress data that is stored in that memory, which also helps with performance here.
Power Consumption:
Simon F pointed out that power consumption would be lower. The paper did mention that custom ALUs are in rasterizers (which would reduce power consumption), and this would make sense since fragment and vertex processing units in the past used to have custom instruction sets (so likely custom ALUs as well). It certainly would be a bottleneck in many systems (e.g., mobile), although this has implications beyond performance.

Summary

TL;DR: there are too many inefficiencies that software rendering can't past, and these things add up. There are also many larger limitations, especially when you are dealing with VRAM bandwidth, synchronization problems, and extra computations.


Vertices of a regular polygon given the incircle radius How can I find the vertices of a polygon (specifically hexagon) given the radius of the incircle? I want to calculate the vertices of a polygon given the face-to-face distance (inscribed circle diameter?)

I've below code to calculate the vertices from the Circum-circle of a polygon

(x,y) - Center of the circumcircle

r - Radius of the circumcircle

for (var i = 1; i <= sides; i++)
{
    var X = x + r * Math.Cos(2 * Math.PI * i / sides);
    var Y = y + r * Math.Sin(2 * Math.PI * i / sides);

    vertices.Add(new Tuple(X,Y));
}

Alternatively, what is the relation between radius of the in-circle and circum-circle of a polygon? Could you please share a relevant resource or a code snippet (any language is fine).

Thanks!

what is the relation between radius of the in-circle and circum-circle of a polygon?

That is $cos ( \frac{2\pi}{n}*\frac{1}{2} ) = cos ( \frac{\pi}{n})$

The triangle with edges from the center to the middle of a side, from the center to an adjacent corner and half the side connecting them is a right angle triangle. The angle of the point at the center $ =\frac\pi 2$.

Then it's simply the cosine to get the ratio between adjacent and hypotenuse.


OpenGL shader introspection/interface queries for images I've been looking at Interface queries in order to prevent common mistakes that are time consuming to debug, such as forgetting to bind a buffer, binding the wrong resource type, etc.
However, I wasn't able to find what I wanted for images. I'd like to make sure that the type of the texture, the memory qualifier and the texture format are the same in the c++ code and in the shader :
glBindImageTexture(index, texture, 0, true, 0, GL_READ_ONLY, GL_RGBA32F);

layout(binding = 0, rgba32f) uniform writeonly iimage2D _heightmap;

Using glGetProgramResourceiv with GL_UNIFORM and GL_TYPE, I was able to get the type of the image, iimage2D. But nothing for the memory qualifier and the image format.
Is there a way to do that, other than parsing the shader?

Those are not accessible via introspection. It should also be pointed out that some of them don't have to "match".
The shader-defined format doesn't have to be the same as the image-bound format. The image-bound format defines how writing from the image interprets the data provided by the shader. The shader-defined format determines how reading from the image interprets the data in the image. So the two values are fundamentally doing different things and need not match either each other or the texture's internal format. There are rules for how conversion works between these and the image's actual format.
For the access type, you could just pass GL_READ_WRITE for everything and let the shader decide how it's going to access the storage. Given that Vulkan has no equivalent to this value, it probably doesn't matter performance-wise. Also, note that the multibind function glBindImageTextures, which binds multiple images in one call, always uses GL_READ_WRITE (and doesn't have a specified format). Since the whole point of that function is performance, it almost certainly doesn't matter if you always use read-write from the binding side.


Do I need separate Graphics and Transfer commands? I was following this tutorial on Vulkan https://vulkan-tutorial.com/Vertex_buffers/Staging_buffer and I decided to do this bit:

The buffer copy command requires a queue family that supports transfer operations, which is indicated using VK_QUEUE_TRANSFER_BIT. The good news is that any queue family with VK_QUEUE_GRAPHICS_BIT or VK_QUEUE_COMPUTE_BIT capabilities already implicitly support VK_QUEUE_TRANSFER_BIT operations.
The implementation is not required to explicitly list it in queueFlags in those cases.
If you like a challenge, then you can still try to use a different queue family specifically for transfer operations. It will require you to make the following modifications to your program:

Modify QueueFamilyIndices and findQueueFamilies to explicitly look for a queue family with the VK_QUEUE_TRANSFER bit, but not the VK_QUEUE_GRAPHICS_BIT.
Modify createLogicalDevice to request a handle to the transfer queue
Create a second command pool for command buffers that are submitted on the transfer queue family
Change the sharingMode of resources to be VK_SHARING_MODE_CONCURRENT and specify both the graphics and transfer queue families
Submit any transfer commands like vkCmdCopyBuffer (which we'll be using in this chapter) to the transfer queue instead of the graphics queue

It's a bit of work, but it'll teach you a lot about how resources are shared between queue families.

Are there any benefits of using a second queue only for transfer operations ?

Your question seems... very confused. The "challenge" you tried to do specifically stated to "use a different queue family specifically for transfer operations". But that's not what your code is trying to do; you used the same queue family.

The point of creating a "transfer queue" is to allow you to perform transfer operations without having to submit them as part of another batch. vkQueueSubmit cannot be called on the same queue from multiple threads. So if one thread builds some transfer work, but the other thread is doing some graphics work, if you only have one queue then the two threads must have some communication mechanism to submit that work.

If you have a transfer queue, that inter-thread communication is no longer necessary. Or at least, it is no longer necessary for the basic submission action. Obviously whichever batch of work that uses the results of the transfer commands will have to receive a semaphore to wait on from the thread that submitted the transfer work. But that is usually a less contentious inter-thread communication path, which can easily be done with a lockless queue of some kind.

The challenge in question is specifically talking about a queue family that only has the VK_QUEUE_TRANSFER_BIT flag set (or more specifically, does not have graphics or compute capabilities). Such transfer-only queues are usually tied to specialized hardware that (potentially) perform transfers faster than a graphics queue, but likely with extra limitations (as defined in VkQueueFamilyProperties::minImageTransferGranularity and other properties).

So basically, you're doing the challenge wrong.

Do keep in mind that Vulkan does not require implementations to provide a transfer-only queue. So if you want to take advantage of the presence of such a queue, you also have to take into account the possibility that no such queue exists.

Now, here's the thing: the term "queue family" is used because you can potentially get multiple queues from the same family. VkQueueFamilyProperties::queueCount tells you how many queues a particular family allows you to get.

So if a GPU does not provide a dedicated transfer-only queue, but you still want the asynchronous benefits of submitting transfer operations outside of non-transfer work, you can create a second graphics queue.

Of course, Vulkan also doesn't require implementations to provide more than one graphics or compute queue. So again, you have to have a synchronous fallback plan in case the hardware only provides you with one queue.


Naive Path tracer produces much darker images than Explicit light sampling So I just thought of comparing the results from a Naive Path tracer and one using Next Event Estimation aka Explicit Direct Light Sampling. However the results from the Naive PT are very dark.

Is this expected? The one with next event estimation produces brighter images like in my previous question which was related to MIS

I thought that a Naive PT would produce images with the same brightness but more noise and it would take longer to converge. The main reason for dark images I think is because I'm averaging the colors obtained from the previous run with the next run. On each run I shoot 1 sample per pixel.

This means if I take 20 samples (20 passes) and only 2 samples ($S_1$ and $S_2$) are useful (hit the light in the end) and 18 return 0 since they don't hit the light, I'd be doing,

$\displaystyle{\frac{S_1 + S_2 + 0*18}{20}}$

The brighter colors will surely get averaged out. Moreover they won't converge or in other words they won't get brighter since the number of useless samples surely outweigh the number of useful samples. However in every implementation I could find over the net, people averaged it simply like I did. So is that actually a problem or is it expected?

I stumbled upon this question which is pretty much the same thing OP is asking but in the end they find problem with the raytracer and not the path tracer. I don't understand.

If you guys are interested in the code, here is my kernel. The main functions to look for are shading() which contains the whole bouncing around portion. EvaluateBRDF simply evaluates the blinn phong model. The averaging out is done in the main kernel evaluatePixelRadiance in the end.

UPDATE:- So I corrected what Stefan pointed out and removed color clamping. I was using an 8bit depth default framebuffer and RBO of 32 bit floating points. So I guess the data gets clamped automatically when blitting to the default framebuffer. And it seems to have solved the issue. I'll update the results shortly.

I think this solved my MIS issue as well, will share results on MIS post soon. Here is the one for naive-PT, 5000 samples in around 10 seconds. And it seems I was not getting fireflies since i was clamping radiance as expected. Good to learn new things :)

It looks like you're clamping all your samples to 0-1 in line 215. Apply clamping only when displaying the image, not when accumulating samples.


Understanding camera() settings in Processing I have trouble understanding the way a camera is set in Processing (according to the reference this shoudn't be very different from OpenGl). For starters, when I set the eye position of the camera I have to set three coordinates. I'm assuming that the X coordinate is the horizontal border of the display and the Y coordinate would be the vertical border. I think the Z coordinate to be behind or in front my computer.

So when I do:

camera(width/2, height/2, depth, 250, 250, 0, 0, 1, 0);

I'm setting the camera in the middle of the display and putting it some depth in the axis that comes into my eyes.

I think this part is clear enough. However, the two other parameters are not as clear. In particular in don't see the difference between the center of the scene and the last parameter which is the axis that is facing upward.

For the center of the scene I'm assuming that it gives me the direction the camera is looking to but what is the axis that is facing upward?

As the documentation describes it, it looks like each group of three parameters is a vector in 3D space. The first three are the position of the “eye”, the next three are the position it’s looking at, and the last three are the “up vector”, i.e. the direction of an arrow pointing out of the top (…sort of—more detail below) of the eye. The positions aren’t related to the size of your display window—the view of your 3D geometry from the camera at that point will fill the display.

In other words, if you have a box centered at (0, 0, 0) that’s 1×1×1 units in size, then your camera should be positioned a few units away—say (2, 3, 4)—and looking roughly at the center of the box, i.e. (0, 0, 0). If the box is 100×100×100 instead, then the eye position should be on the same order of magnitude in distance, e.g. (200, 300, 400). The behavior of the up vector is a little more complicated to explain, but leaving it at (0,1,0) is fine for most purposes—it’s mostly there to help you avoid gimbal lock.