rendering a triangle has no color I wrote a program to render a simple triangle with colors, the triangle is there but there's no color.
GLfloat vertices[] = {
        0.6f,0.4f , 0.3f,0.5f, 0.1f,0.3f, 0.8f,0.2f, 0.1f,-0.4f, 0.9f,0.3f
    };

    unsigned int program;
    if (!loadShader(program))
        return -1;

    unsigned int VAO;
    glGenVertexArrays(1, &VAO);

    unsigned int buffer;
    /* GENERATE THE BUFFER (1 BUUFER) */
    glGenBuffers(1, &buffer);
    glBindVertexArray(VAO);
    /* SELECT THAT BUFFER TO WORK WITH */
    glBindBuffer(GL_ARRAY_BUFFER, buffer);

    //glBufferData(GL_ARRAY_BUFFER, RealPositions.size()*sizeof(float), RealPositions.data(), GL_STATIC_DRAW);
    glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), vertices, GL_STATIC_DRAW);

    glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 4 * sizeof(float), (void*)0);
    glEnableVertexAttribArray(0);
    glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, 4 * sizeof(float), (void*)(2 * sizeof(float)));
    glEnableVertexAttribArray(1);

    glBindVertexArray(0);
    /* Loop until the user closes the window */
    while (!glfwWindowShouldClose(window))
    {
        /* Render here */
        glClear(GL_COLOR_BUFFER_BIT);

        glUseProgram(program);

        glBindVertexArray(VAO);
        glDrawArrays(GL_TRIANGLES, 0, 3);

        /* Swap front and back buffers */
        glfwSwapBuffers(window);

        /* Poll for and process events */
        glfwPollEvents();
}

and this the vertex shader:
#version 440

layout (location = 0) in vec3 vertices;
layout (location = 1) in vec3 vertex_color;
out vec3 vs_color;
void main()
{
    gl_Position=vec4(vertices, 1.f);
    vs_color=vertex_color;
}

and this is the fragment shader:
#version 440 core

in vec4 vs_pos;
in vec3 vs_color;
out vec4 fs_color;

void main()
{
    fs_color=vec4(vs_color, 1.f);
}

what am i doing wrong?

Your inputs and outputs still don't line up.
#version 440 core

// Comment out the line below
// in vec4 vs_pos;    <----- No corresponding output in the vertex shader
in vec3 vs_color;

out vec4 fs_color;

void main()
{
    fs_color=vec4(vs_color, 1.f);
}

The gl_Position output is consumed and does not need a specific input in the fragment shader. Only variable declared as "out" in the vertex shader need a corresponding "in".
The next issue is the vertices need to have colors in them. The way it is setup now all the components are consumed before even a single triangle is drawn.
The vertex buffer needs to be setup:
vertex,color, vertex,color, vertex,color
where each vertex and color has 3 floating point values.
GLfloat vertices[] = {

//    vertex           color
-0.5f, -0.5f, 0.0f,  1.0f,0.0f,0.0f
 0.5f, -0.5f, 0.0f,  0.0f,1.0f,0.0f
 0.0f,  0.5f, 0.0f,  0.0f,0.0f,1.0f
};

Something like the above. Is what you need for the vertices.


What transformations should I apply to a model to produce this 3D effect? I saw a video on YouTube where a person uses a transparant, knotted pyramid to display "holograms" using their smartphone. (I have not been able to reproduce this yet, so I can't tell for sure if it works).

I was wondering, given a model, which transformations I should apply to get the different projections needed to produce this effect?

Assuming your optimal viewing angle is parallel to the surface of the display and the pyramid is made from faces that are 45 degrees to its virtual (non-existant) base, it's actually just a simple non-transformed image (besides the reflection).

1:1 projection. No transformations. No scaling.


FPS counter for a simple Vulkan app What measurement/monitoring tools could I use to get the FPS values for a simple Vulkan app?

Thanks in advance

You might want to try the VK_LAYER_LUNARG_monitor layer from the LunarG Vulkan SDK.

Download the SDK from the LunarXchange site.

After SDK installation, you can activate the layer by setting this environment variable:

VK_INSTANCE_LAYERS=VK_LAYER_LUNARG_monitor

The monitor layer displays the FPS value in the application window's title bar.


Should the alpha channel be gamma corrected We know that in PNG,BMP,etc... the pixel value stored is not in the linear RGB space. But I found no document saying anything about the alpha channel. Is the alpha channel stored in image files in linear space or not?

We know that in PNG,BMP,etc... the pixel value stored is not in the linear RGB space.

This is not necessarily true. You can store whatever color space you want into an image, it doesn't even need to be colors (such as normal maps).

The alpha channel is generally linear. The alpha channel doesn't get displayed, but it is generally a non-color term used for transparency (or whatever else). Because they don't need to display on a monitor, there's no reason to store in in gamma space. If you did, you would unnecessarily lose precision at the lower end of the alpha values. Normals maps follow a similar line of reasoning, as explained very well by Julien Guertault.


What's the simplest, open, animation format? I have an obj parser that handles triangle meshes specifically.
I want to extend it (and the rest of my system) to handle animations as well. However, I don't think I am ready to tackle the glTF format. Because it seems to encompass much more than what I want.
I just need a simple file format that describes a mesh, some bones, and the skinning weights of the mesh vertices with respect to the bones. Plus a series of poses for the skeleton.
What file format could I look into?

For what it's worth, the glTF Tutorials contain a section on Skinning that shows how the raw vertex and joint weight data feed into a vertex shader to distort the mesh.
If you do design your own format to hold this, you'll need tooling to export it from 3D content creation software, as well as software to read the new format into your application.  Using an open format like glTF can grant access to existing tools, such as the Blender glTF exporter, that can write all this data into the file, in a form that's ready to load into vertex attributes and shaders.


Determining shortest vector between a ray and an axis-aligned 3D bounding box (correcting a near-miss) I have a situation where (due to floating point error) a ray that is known to hit a bounding box fails due to floating point error. (1e-8 or so)

Expanding the box is not an acceptable solution in my specific use case. However, if I could "snap" the ray to the box, that would work out well.

In order to do so, I need to find the shortest vector from a given ray to a bounding box. In 2 dimensions, that would look like this:

If I can get that red vector, I can add it to the origin and the ray will be shifted to the bounding box.

In 2D this is apparently relatively easy, as you can get distance between two lines like here, then find the smallest of the 4 lines and use that.

But in 3D I am not aware of how to do this. Does anyone know if there is a way?

If you know the rounding error, you could subtract the error from your vectors so that rays would overlap the bounding box. This could be done in your intersection tests.

In 3D, computing shortest distance between a vector and face of a bounding box would be another way to shift the vector to overlap the bounding box. But I doubt this approach would work well as it essentially tries to fix a rounding problem.


Fresnel equation with incident angle, n and k as input I've seen a Fresnel equation for calculating the reflection amount at incidence angle with n and k values defined. This formula usually comes up in relation to computer graphics, but I can't find this equation anywhere in a science context.

Here is an example written in Python, taken from the website
Fresnel Formula in python:

def IOR(n,k):
theta_deg = 0

n = n
k = k
fresnel = []

while theta_deg <= 90:
theta = math.radians(theta_deg)

a = math.sqrt((math.sqrt((n**2-k**2-(math.sin(theta))**2)**2 + ((4 * n**2) * k**2)) + (n**2 - k**2 - (math.sin(theta))**2))/2)

b = math.sqrt((math.sqrt((n**2-k**2-(math.sin(theta))**2)**2 + ((4 * n**2) * k**2)) - (n**2 - k**2 - (math.sin(theta))**2))/2)

Fs = (a**2+b**2-(2 * a * math.cos(theta))+(math.cos(theta))**2)/(a**2+b**2+(2 * a * math.cos(theta))+(math.cos(theta))**2)
Fp = Fs * ((a**2+b**2-(2 * a * math.sin(theta) * math.tan(theta))+(math.sin(theta))**2*(math.tan(theta))**2)/(a**2+b**2+(2 * a * math.sin(theta) * math.tan(theta))+(math.sin(theta))**2*(math.tan(theta))**2))
R = (Fs + Fp)/2
fresnel.append(R)

theta_deg += 1
return fresnel

I can't find any references online for this formula, the Wikipedia article on Fresnel equations contains completely different formulas, and I can't see the connection between them. I've searched high and low, but can't find any reference to this particular formula, why is that? Can you help me find it, or explain to me why I can't find it online? Or could you show me how to derive it from the Fresnel formulas from the Wikipedia article?

Yes, I realize that I already have the formula right there in the code sample. But I'm baffled over why I can't find this formula anywhere, I would like to have more sources to cite then just this code example.

@PaulHK's answer is correct I'm sure, here's a bit of a check to show that the IOR() function is calculating the reflection coefficients for $s$ and $p$ polarizations then averaging the two assuming unpolarized incident light.

The reflection is for a single interface, and at least at normal incidence the results reduce to the simpler $(n_2-n_1)^2/(n_2+n_1)^2$. With this script you can check against other on-line calculators using non-zero incident angle and nonzero values for $k$.

I cleaned up the python a bit (indents) and added the plot. You can see when exiting from glass to air, the reflectivity goes to 100% at the critical angle for total internal reflection:

def IOR(n,k):
    theta_deg = 0

    fresnel = []

    while theta_deg <= 90:
        theta = math.radians(theta_deg)

        a = math.sqrt((math.sqrt((n**2-k**2-(math.sin(theta))**2)**2 +
                                 ((4 * n**2) * k**2)) + (n**2 - k**2 -
                                            (math.sin(theta))**2))/2)

        b = math.sqrt((math.sqrt((n**2-k**2-(math.sin(theta))**2)**2 +
                                 ((4 * n**2) * k**2)) - (n**2 - k**2 -
                                            (math.sin(theta))**2))/2)

        Fs = (a**2+b**2-(2 * a * math.cos(theta))+
              (math.cos(theta))**2)/(a**2+b**2 +
                            (2 * a * math.cos(theta))+(math.cos(theta))**2)

        Fp = Fs * ((a**2+b**2 -
                    (2 * a * math.sin(theta) * math.tan(theta)) +
                    (math.sin(theta))**2*(math.tan(theta))**2)/(a**2+b**2 +
                    (2 * a * math.sin(theta) * math.tan(theta)) +
                    (math.sin(theta))**2*(math.tan(theta))**2))

        R = (Fs + Fp)/2

        fresnel.append((R, Fs, Fp))

        theta_deg += 1
    return fresnel

import math
import matplotlib.pyplot as plt

n1, n2  = 1.00, 1.45  # into glass, one side only
n2ovrn1 = n2/n1

Rin, Fsin, Fpin = zip(*IOR(n2ovrn1, 0))

print Rin[0]
print (n2ovrn1-1.0)**2 / (n2ovrn1+1.0)**2  # check

n1, n2  = 1.45, 1.00  # outof glass, one side only
n2ovrn1 = n2/n1

Rout, Fsout, Fpout = zip(*IOR(n2ovrn1, 0))

print Rout[0]
print (n2ovrn1-1.0)**2 / (n2ovrn1+1.0)**2  # check

plt.figure()
plt.plot(range(91), Rin, '-k', linewidth=2)
plt.plot(range(91), Rout, '-k', linewidth=2)
plt.plot(range(91), Fsin, '-k')
plt.plot(range(91), Fsout, '-k')
plt.plot(range(91), Fpin, '--k')
plt.plot(range(91), Fpout, '--k')
plt.show()


OpenTK - Vertex attribute layout issue I use C# and OpenTK as my OpenGL implementation.
My Vertex layout consists of 6 Doubles and one Float:

public struct Vertex
{
    //Positions
    public double X;
    public double Y;
    public double Z;
    //ContourEdge
    public float IsContourEdge;
    //Normals
    public double NX;
    public double NY;
    public double NZ;
    public Vertex(float x, float y, float z) : this()
    {
        X = x;
        Y = y;
        Z = z;
        IsContourEdge = 0f;
    }

    public Vertex(float x, float y, float z, float nX, float nY, float nZ) : this()
    {
        X = x;
        Y = y;
        Z = z;
        NX = nX;
        NY = nY;
        NZ = nZ;
        IsContourEdge = 0f;
    }
    public static int IsContourEdgeOffset()
    {
        return sizeof (double) * 3;
    }
    public static int NormalOffset()
    {
        return sizeof(double) * 4;
    }
}

And I render my vertices like this:

//Bind vao, vbo and ebo
GL.BindVertexArray(bufferObject.Vao);

GL.BindBuffer(BufferTarget.ArrayBuffer, bufferObject.Vbo);
GL.BufferData(BufferTarget.ArrayBuffer, (IntPtr)(vertices.Length * Marshal.SizeOf(typeof(Vertex))), vertices, BufferUsage);

// Vertex Positions
GL.EnableVertexAttribArray(0);
GL.VertexAttribPointer(0, 3, VertexAttribPointerType.Double, false, Marshal.SizeOf(typeof(Vertex)), IntPtr.Zero);
// Is Contour Edge
GL.EnableVertexAttribArray(1);
GL.VertexAttribPointer(1, 1, VertexAttribPointerType.Float, false, Marshal.SizeOf(typeof(Vertex)), Vertex.IsContourEdgeOffset());
// Vertex Normals
if (hasNormals)
{
    GL.EnableVertexAttribArray(2);
    GL.VertexAttribPointer(2, 3, VertexAttribPointerType.Double, false, Marshal.SizeOf(typeof(Vertex)), Vertex.NormalOffset());
}

//Unlink buffers
GL.BindBuffer(BufferTarget.ArrayBuffer, 0);
GL.BindVertexArray(0);
GL.BindBuffer(BufferTarget.ElementArrayBuffer, 0);

Everything works fine here.
But the offset of the normals doesn't make sense for me. In my Vertex struct I said:

public static int NormalOffset()
{
    return sizeof(double) * 4;
}

But shouldn't it be

public static int NormalOffset()
{
    return 3* sizeof(double) * sizeof(float);
}

since IsContourEdge is a float and a I also pass it as a float (VertexAttribPointerType.Float) to my vertex shader?

Anyway, if I use this NormalOffset instead of the previous one, the normals don't work anymore.
The only explanation I have is that OpenGL expands the size of a Float to the size of a Double. Is that right?

Just for completeness, here is the vertex shader code:

#version 330 core
layout (location = 0) in vec3 position; //Vertices and normals are in world coordinate space
layout (location = 1) in float isContourEdge;
layout (location = 2) in vec3 normal;

out VS_OUT {
    vec3 Normal;
    vec3 FragPos;
    vec3 VertWorldNormal;
    vec3 VertWorldPos;
    float ExcludeEdge;
} vs_out;

uniform mat4 view;
uniform mat4 proj;

void main()
{
    gl_Position = proj * view *  vec4(position, 1.0f);
    vs_out.FragPos = vec3(view * vec4(position, 1.0f));
    vs_out.Normal = vec3(view * vec4(normal, 0));
    vs_out.ExcludeEdge = isContourEdge;

    vs_out.VertWorldNormal = normal;
    vs_out.VertWorldPos = position;
}

welcome to the world of padding. Doubles are 8 byte aligned so when a 4 byte float gets inserted between 2 doubles some padding gets inserted to keep the next double aligned to a 8 byte boundary.

You can make sure the offset is always correct (even when alignment changes in future versions) by using the Marshal.OffsetOf(Type, String) method.


ping pong between same framebuffer two color attachments In order to optimize I'm trying to set up a setup which ping pong between same framebuffer two color attachments to apply some postprocess effects. It seems to work (effects are applied) but concurently I'm getting a lot of strange artifacts like:

blinking squares
borders on depth discontinuities
color spilling on screen (window resize removes it)

Which change or disappear partly when I for example remove one step (one post process effect) from "ping ponging".

If I do this ordinary way using different framebuffers seems to remove them but I'm not 100% sure if it is not a mess in the shaders or something but they seem fine.

In each frame I'm doing ping pong like that:

First step - render to current color attach using other fb texture

fbDeferredLighting->bindFramebuffer(); //fb which has two color attachments
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
effect1->setConstPerFrameUniforms();
//get 0 attachment from fb scene prepass to apply effect and draw on fbDeferredLighting current color attach
effect1->setTextureUniforms(fbScene, 0);
quad->draw();

And then repeating some number of times: swap step and drawing, using same fb swapped attachment:

fbDeferredLighting->bindFramebuffer();
//after swap - now current=1, other=0 or current=0, other=1
std::swap(colorAttachment0Current, colorAttachment0Other);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texturesDiffuse[colorAttachment0Current]->getTextureID(), 0);
glClear(GL_DEPTH_BUFFER_BIT); //only clear depth
effectX->setConstPerFrameUniforms();
//get other color attach and use it to apply effect on current
effectX->setTextureUniforms(fbDeferredLighting , colorAttachment0Other);
quad->draw();

Is this correct?

EDIT:
I wonder, maybe rewrite it to compute shaders, that would simplify things.

Actually, GL_COLOR_BUFFER_BIT also need to be cleared

After creation of the framebuffer, draw buffer need to be set to:

glDrawBuffer(GL_COLOR_ATTACHMENT0);

and then after each swap:

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

instead of only

glClear(GL_DEPTH_BUFFER_BIT);

To clear current attachment 0.
I don't really understand these clearing bits extensively but this seems to fix blinking "chess board" artifact.

Borders seems to be connected with motion blur shader.


generate multiple border contour Once a 3D model is sliced against a plane and projected onto it, we get a contour curve at certain height. The contour curve I am generating in this process is made of several line segments in closed loop. Now I want to generate another contour curve with the following characteristics:

Each of the line segment of the newly generated inner contour curve must be parallel to the corresponding line segments at the outer contour curve.
Each of the parallel line segments on both outer and inner contour curve has an adjacent distance in between and defined by a constant value as shown in the attached picture (green mark)

Image Link

I need some suggested algorithms to generate it . I hope that I have explained the issue properly. Please do let me know if it is not clear enough.

Thanks

Polygon offsetting is certainly possible to do yourself. It is not nearly as involved as Bezier or B-spline offsetting is. First you want to decide how to deal with:

Areas with no info, in essence how to deal with the joins. Do you just extend the line to the common intersection (miter join) or do you just connect the endpoints (bevel). You could also use round, parabolic or other join styles if you wish. Options here are unlimited.

For the purposes of this discussion I will assume you will want a miter or bevel join.
How do you want to deal with sidedness. Is an open curve's offset closed or just on the other side. Other than that the convention is usually in 2D that outside is on the clockwise side of the path

So lets look at what happens when 2 straight lines join and they are offset.

Image 1: Situation in 2D.

For a beveled corner join vector, from any point to its offset is magnitude of $a$. Take the direction of the segment rotated counterclockwise 90 degrees then multiply that with the length of offset. Or in the case of the picture in mathematical notation

$$
 A' = A + a\bigg(\big|B-A\big|   \begin{bmatrix}
    0 & 1\\
    -1 & 0
  \end{bmatrix}\bigg)
$$

Where $a$ is the amount of offset. This assumes lines go from $A$ to $B$ to $C$. Connecting segments must be processed on both sides separately. The same formula can be used for all the points. Then just connect $B'^1$ to $B'^2$ with a line. In case you are matrix illiterate, the matrix says swap x and y coordinate and invert the x value.

For a miter join the following applies:

Image 2: Situation of miter.

$$
 B^{B-B'^{1}} = a\bigg(\big|B-A\big| \begin{bmatrix}
    0 & 1\\
    -1 & 0
  \end{bmatrix}\bigg)
$$
$$
B' = B+\dfrac{ B^{B-B'^{1}}}{B^{B-B'^{1}}  \cdot (|B-A|-|C-B|)}
$$

The above can be heavily simplified but I'll leave that to the reader.

Now the reason to use a library is that when the curves start to self intersect it gets a bit tedious to manage this. So just find a library that does this for you and you'll save a lot of time.


Transfer the texture between two different sets of uv Say I have a mesh with two sets of uvs. And I have a texture made for the first set of uvs. I would like to create a second texture that has the same relationship between its pixels and the second set of uvs, as the first texture does with the first set of uvs. Such that if we change between the two different sets of uvs and apply the corresponding texture we see identical texturing on the model. And the only difference would be the texture density in different regions because of the differences in distortion and the area of the uv faces between the two uv sets.
What technique can we use do this? Can it be done on the GPU?
Would we possibly need to transform each uv face of the first uv set into the corresponding face of second uvs set? Not sure where we would go from there.

Are the two sets well correlated? This would boil down to having a good set of vertex attributes for each set of uv's. If each set of uv's in mapping 1 has a know set of mappings in set 2 then it should be relatively straight forward to read from the first set of uv's and write to the second set.
If this is as simple as vertex 1 has a mapping in uv set 1 and a different mapping for uv set 2 then read from set one then resample the texture for set 2.
On the other hand if the two sets are completely uncorrelated then it gets more complex:
A fully generic brute force solution would be to read 1 of the vertex attributes of the known mapping. Then search the entire set of uv's for set 2 filtering out those vertex positions that overlap with the current uv under consideration. Then use that list to read, resample and write the new texture.
The first case is easier to implement on the gpu. The second case is more difficult and may be better to implement on the cpu for debugging then move to the gpu as performance requirements dictate.
Edit for the first case:
This is the easy case, Use the uv coords from set 1 on the gpu just like you would normally to read the texture. Then use the second set as actual vertex positions adding a z value that is some constant. Don't use a MVP matrix just go straight to clip space. The gpu will worry about the resampling. In the end you will have a fullscreen quad with the final mappings. Then just read the image back from the gpu.
More detail:
The vertex attributes for this would be:
Vertex Position = vec2{u_set2, v_set2}
uv coords = vec2{u_set1, v_set1}

The vertex shader:
vec2 remapping = vertex_position * 2.0 - 1.0; // remap x,y to -1 to 1
gl_position = vec4( remapping, 0.5, 1 );
out texcoord = uv_coords_in;

The fragment shader:
gl_color = texture( image_sampler, tex_coords);


Jos Stam's "Stable Fluids" boundary conditions In the paper Stable Fluids by Jos Stam, I am confused about the boundaries in the set_bnd function in the code. I don't know what kind of boundaries is this. Is it Neumann boundary condition or Dirichlet boundaries?

Here is the set_bnd function; the full code can be found here.

void set_bnd ( int N, int b, float * x )
{
    int i;

    for ( i=1 ; i<=N ; i++ ) {
        x[IX(0  ,i)] = b==1 ? -x[IX(1,i)] : x[IX(1,i)];
        x[IX(N+1,i)] = b==1 ? -x[IX(N,i)] : x[IX(N,i)];
        x[IX(i,0  )] = b==2 ? -x[IX(i,1)] : x[IX(i,1)];
        x[IX(i,N+1)] = b==2 ? -x[IX(i,N)] : x[IX(i,N)];
    }
    x[IX(0  ,0  )] = 0.5f*(x[IX(1,0  )]+x[IX(0  ,1)]);
    x[IX(0  ,N+1)] = 0.5f*(x[IX(1,N+1)]+x[IX(0  ,N)]);
    x[IX(N+1,0  )] = 0.5f*(x[IX(N,0  )]+x[IX(N+1,1)]);
    x[IX(N+1,N+1)] = 0.5f*(x[IX(N,N+1)]+x[IX(N+1,N)]);
}

It's Dirichlet boundary condition. The quantities at domain boundaries will take either the same value as its inner neighbor or the negated value, depending on the condition $\mathbf{b}$. Take velocity field for instance, velocity of fluid will either gets reflected or not change at domain boundary. The condition $\mathbf{b}$ is a user option, $\mathbf{b}==1$ is used to set reflected values at the horizonal boundaries, and $\mathbf{b}==2$ is to set reflected values at vertical boundaries. To be more clear:

$\mathbf{b}==1$: field values at horizonal boundaries get reflected, field values at vertical boundaries do not change.
$\mathbf{b}==2$: field values at vertical boundaries get reflected, field values at horizonal boundaries do not change.
$\mathbf{b}== other\ value$: field values at neither vertical nor horizonal boundaries do not change.

set_bnd could be called multiple times to set desired boundary values.


Average intensity of an gray image Suppose i have a 400*300 gray scale image.Now I want to scan each line horizontally and make the average of that line 128,such that the values get scaled as per their difference from mean(128) which will not cause any distortion to image.

Here's an example on how to do this with Python, Numpy and Matplotlib.
Image on the right has been averaged (row by row), while the left is the original.

EDITED: As suggested by @Nathan Reed, I avoided flipping colors when row average is 0. Rows with an average of 0 now get a value of 0.5 (128). The dark image encountered previously was due to values higher than 1 (255) due to scaling, that were represented as white in matplotlib's image renderer. I fixed this by setting the max value in any pixel to be 1.

import numpy as np
from matplotlib import image as mpimg
from matplotlib import pyplot as plt

def rgb2gray(rgb):
    return np.dot( rgb[...,:3], [0.299, 0.587, 0.144] )

imgPath = "C:/myImg.jpg"

im = mpimg.imread( imgPath )
gs = rgb2gray( im )

averaged = gs.copy()

for row in averaged:
    rowAvg = sum( row ) / len( row )
    if rowAvg == 0:
        row = 0.5
    else:
        # Scale each row by ratio between 128 and current row avg
        row *= 0.5 / rowAvg

    # Make sure that no pixel has a higher value than 1
    if rowAvg > 0: row[row>1] = 1

plt.subplot( 1, 2, 1 ), plt.imshow( gs,       'gray' )
plt.subplot( 1, 2, 2 ), plt.imshow( averaged, 'gray' )

plt.show()

Here's another test with a simple gradient image (left is the original):


How to use multithreading in 3D software renderer to speed up fetching texture values The biggest bottleneck of my code is fetching texture RGB values from memory.

My code looks something like this:

game loop{
    for every mesh{
        perform clipping;
        for every 3 verticies in mesh {
            draw triangle {
                interpolate texture coordinates;
                fetch texture RGB and place value in backbuffer;
            }
        }
    }
}

Fetching the texture RGB value from memory takes up so much time!!

I am not familiar with multithreading but I'm assuming there is some way to apply it here. Let me know what suggestions you have and how I should go about doing this!

It could be that you have to overcome a different bottleneck first.

Have you ever read Jim Blinn's "The Truth About Texture Mapping"? (I had a quick search to see if I could find a non-paywalled version but you may have better luck than me. Alternatively you might find a dead tree version of "Jim Blinn's Corner" in a library). Though this article is old and describes paging of texture data, it is still very relevant today.

Essentially, if your textures are large (i.e. too large to fit in the cache), in scan order, and they have been rotated when displayed on the polygons, you are very likely to be thrashing your cache and, as memory is an order or two of magnitude slower than the CPU, this will hurt performance.

To avoid the cache thrashing, textures are often stored in twiddled-order (that's what we called it in the early 90s but, more correctly, it'll be some variant of Morton order) or in a block order, which is what Blinn describes. This then makes texel/memory accesses far more coherent and the cache more effective.


What is Importance Sampling? What is importance sampling? Every article I read about it mentions 'PDF' what is that as well?

From what I gather, importance sampling is a technique to only sample areas on a hemisphere that matter more than others. So, ideally, I should sample rays towards light sources to reduce noise and increase speed. Also, some BRDF's at grazing angles have little no difference in the calculation so using importance sampling to avoid that is good?

If I were to implement importance sampling for a Cook-Torrance BRDF how could I do this?

Short answer:

Importance sampling is a method to reduce variance in Monte Carlo Integration by choosing an estimator close to the shape of the actual function.

PDF is an abbreviation for Probability Density Function. A $pdf(x)$ gives the probability of a random sample generated being $x$.

Long Answer:

To start, let's review what Monte Carlo Integration is, and what it looks like mathematically.

Monte Carlo Integration is an technique to estimate the value of an integral. It's typically used when there isn't a closed form solution to the integral. It looks like this:

$$\int f(x) \, dx \approx \frac{1}{N} \sum_{i=1}^N \frac{f(x_i)}{pdf(x_{i})}$$

In english, this says that you can approximate an integral by averaging successive random samples from the function. As $N$ gets large, the approximation gets closer and closer to the solution. $pdf(x_{i})$ represents the probability density function of each random sample.

Let's do an example: Calculate the value of the integral $I$.

$$I = \int_{0}^{2\pi} e^{-x} \sin(x) dx$$

Let's use Monte Carlo Integration:

$$I \approx \frac{1}{N} \sum_{i=1}^N \frac{e^{-x} \sin(x_i)}{pdf(x_{i})}$$

A simple python program to calculate this is:

import random
import math

N = 200000
TwoPi = 2.0 * math.pi

sum = 0.0

for i in range(N):
    x = random.uniform(0, TwoPi)

    fx = math.exp(-x) * math.sin(x)
    pdf = 1 / (TwoPi - 0.0)

    sum += fx / pdf

I = (1 / N) * sum
print(I)

If we run the program we get $I = 0.4986941$

Using separation by parts, we can get the exact solution:

$$I = \frac{1}{2} (1 − e−2π) = 0.4990663$$

You'll notice that the Monte Carlo Solution is not quite correct. This is because it is an estimate. That said, as $N$ goes to infinity, the estimate should get closer and closer to the correct answer. Already at $N = 2000$ some runs are almost identical to the correct answer.

A note about the PDF: In this simple example, we always take a uniform random sample. A uniform random sample means every sample has the exact same probability of being chosen. We sample in the range $[0, 2\pi]$ so, $pdf(x) = 1 / (2\pi - 0)$

Importance sampling works by not uniformly sampling. Instead we try to choose more samples that contribute a lot to the result (important), and less samples that only contribute a little to the result (less important). Hence the name, importance sampling.

If you choose a sampling function whose pdf very closely matches the shape of $f$, you can greatly reduce the variance, which means you can take less samples. However, if you choose a sampling function whose value is very different from $f$, you can increase the variance. See the picture below:

Image from Wojciech Jarosz's Dissertation Appendix A

One example of importance sampling in Path Tracing is how to choose the direction of a ray after it hits a surface. If the surface is not perfectly specular (ie. a mirror or glass), the outgoing ray can be anywhere in the hemisphere.

We could uniformly sample the hemisphere to generate the new ray. However, we can exploit the fact that the rendering equation has a cosine factor in it:

$$L_{\text{o}}(p, \omega_{\text{o}}) =  L_{e}(p, \omega_{\text{o}}) + \int_{\Omega} f(p, \omega_{\text{i}}, \omega_{\text{o}}) L_{\text{i}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}}$$

Specifically, we know that any rays at the horizon will be heavily attenuated (specifically, $\cos(x)$ ). So, rays generated near the horizon will not contribute very much to the final value.

To combat this, we use importance sampling. If we generate rays according to a cosine weighted hemisphere, we ensure that more rays are generated well above the horizon, and less near the horizon. This will lower variance and reduce noise.

In your case, you specified that you will be using a Cook-Torrance, microfacet-based BRDF. The common form being:

$$f(p, \omega_{\text{i}}, \omega_{\text{o}}) = \frac{F(\omega_{\text{i}}, h) G(\omega_{\text{i}}, \omega_{\text{o}}, h) D(h)}{4 \cos(\theta_{i}) \cos(\theta_{o})}$$

where

$$F(\omega_{\text{i}}, h) = \text{Fresnel function} \\
G(\omega_{\text{i}}, \omega_{\text{o}}, h) = \text{Geometry Masking and Shadowing function} \\
D(h) = \text{Normal Distribution Function}$$

The blog "A Graphic's Guy's Note" has an excellent write up on how to sample Cook-Torrance BRDFs. I will refer you to his blog post. That said, I will try to create a brief overview below:

The NDF is generally the dominant portion of the Cook-Torrance BRDF, so if we are going to importance sample, the we should sample based on the NDF.

Cook-Torrance doesn't specify a specific NDF to use; we are free to choose whichever one suits our fancy. That said, there are a few popular NDFs:

GGX
Beckmann
Blinn

Each NDF has it's own formula, thus each must be sampled differently. I am only going to show the final sampling function for each. If you would like to see how the formula is derived, see the blog post.

GGX is defined as:

$$D_{GGX}(m) = \frac{\alpha^2}{\pi((\alpha^2-1) \cos^2(\theta) + 1)^2}$$

To sample the spherical coordinates angle $\theta$, we can use the formula:

$$\theta = \arccos \left( \sqrt{\frac{\alpha^2}{\xi_1 (\alpha^2 - 1) + 1}} \right)$$

where $\xi$ is a uniform random variable.

We assume that the NDF is isotropic, so we can sample $\phi$ uniformly:

$$\phi = \xi_{2}$$

Beckmann is defined as:

$$D_{Beckmann}(m) = \frac{1}{\pi \alpha^2\cos^4(\theta)} e^{-\frac{\tan^2(\theta)}{\alpha^2}}$$

Which can be sampled with:

$$\theta = \arccos \left(\sqrt{\frac{1}{1 = \alpha^2 \ln(1 - \xi_1)}} \right) \\
\phi = \xi_2$$

Lastly, Blinn is defined as:

$$D_{Blinn}(m) = \frac{\alpha + 2}{2 \pi} (\cos(\theta))^{\alpha}$$

Which can be sampled with:

$$\theta = \arccos \left(\frac{1}{\xi_{1}^{\alpha + 1}} \right) \\
\phi = \xi_2$$

Putting it in Practice

Let's look at a basic backwards path tracer:

void RenderPixel(uint x, uint y, UniformSampler *sampler) {
    Ray ray = m_scene->Camera.CalculateRayFromPixel(x, y, sampler);

    float3 color(0.0f);
    float3 throughput(1.0f);

    // Bounce the ray around the scene
    for (uint bounces = 0; bounces < 10; ++bounces) {
        m_scene->Intersect(ray);

        // The ray missed. Return the background color
        if (ray.geomID == RTC_INVALID_GEOMETRY_ID) {
            color += throughput * float3(0.846f, 0.933f, 0.949f);
            break;
        }

        // We hit an object

        // Fetch the material
        Material *material = m_scene->GetMaterial(ray.geomID);
        // The object might be emissive. If so, it will have a corresponding light
        // Otherwise, GetLight will return nullptr
        Light *light = m_scene->GetLight(ray.geomID);

        // If we hit a light, add the emmisive light
        if (light != nullptr) {
            color += throughput * light->Le();
        }

        float3 normal = normalize(ray.Ng);
        float3 wo = normalize(-ray.dir);
        float3 surfacePos = ray.org + ray.dir * ray.tfar;

        // Get the new ray direction
        // Choose the direction based on the material
        float3 wi = material->Sample(wo, normal, sampler);
        float pdf = material->Pdf(wi, normal);

        // Accumulate the brdf attenuation
        throughput = throughput * material->Eval(wi, wo, normal) / pdf;

        // Shoot a new ray

        // Set the origin at the intersection point
        ray.org = surfacePos;

        // Reset the other ray properties
        ray.dir = wi;
        ray.tnear = 0.001f;
        ray.tfar = embree::inf;
        ray.geomID = RTC_INVALID_GEOMETRY_ID;
        ray.primID = RTC_INVALID_GEOMETRY_ID;
        ray.instID = RTC_INVALID_GEOMETRY_ID;
        ray.mask = 0xFFFFFFFF;
        ray.time = 0.0f;
    }

    m_scene->Camera.FrameBuffer.SplatPixel(x, y, color);
}

IE. we bounce around the scene, accumulating color and light attenuation as we go. At each bounce, we have to choose a new direction for the ray. As mentioned above, we could uniformly sample the hemisphere to generate the new ray. However, the code is smarter; it importance samples the new direction based on the BRDF. (Note: This is the input direction, because we are a backwards path tracer)

// Get the new ray direction
// Choose the direction based on the material
float3 wi = material->Sample(wo, normal, sampler);
float pdf = material->Pdf(wi, normal);

Which could be implemented as:

void LambertBRDF::Sample(float3 outputDirection, float3 normal, UniformSampler *sampler) {
    float rand = sampler->NextFloat();
    float r = std::sqrtf(rand);
    float theta = sampler->NextFloat() * 2.0f * M_PI;

    float x = r * std::cosf(theta);
    float y = r * std::sinf(theta);

    // Project z up to the unit hemisphere
    float z = std::sqrtf(1.0f - x * x - y * y);

    return normalize(TransformToWorld(x, y, z, normal));
}

float3a TransformToWorld(float x, float y, float z, float3a &normal) {
    // Find an axis that is not parallel to normal
    float3a majorAxis;
    if (abs(normal.x) < 0.57735026919f /* 1 / sqrt(3) */) {
        majorAxis = float3a(1, 0, 0);
    } else if (abs(normal.y) < 0.57735026919f /* 1 / sqrt(3) */) {
        majorAxis = float3a(0, 1, 0);
    } else {
        majorAxis = float3a(0, 0, 1);
    }

    // Use majorAxis to create a coordinate system relative to world space
    float3a u = normalize(cross(normal, majorAxis));
    float3a v = cross(normal, u);
    float3a w = normal;

    // Transform from local coordinates to world coordinates
    return u * x +
           v * y +
           w * z;
}

float LambertBRDF::Pdf(float3 inputDirection, float3 normal) {
    return dot(inputDirection, normal) * M_1_PI;
}

After we sample the inputDirection ('wi' in the code), we use that to calculate the value of the BRDF. And then we divide by the pdf as per the Monte Carlo formula:

// Accumulate the brdf attenuation
throughput = throughput * material->Eval(wi, wo, normal) / pdf;

Where Eval() is just the BRDF function itself (Lambert, Blinn-Phong, Cook-Torrance, etc.):

float3 LambertBRDF::Eval(float3 inputDirection, float3 outputDirection, float3 normal) const override {
    return m_albedo * M_1_PI * dot(inputDirection, normal);
}


How to interpolate vertex normals for Phong lighting when barycentric coordinates are negative? I'm interpolating normals for Phong lighting models. Since I am calculating in the light of each pixel, I'm doing the interpolating after the viewport transform has been applied. So I save a version of the vertices in eye coordinate in order to do barycentric coordinate on them.

I get all these weird negative barycentric coordinates though. For example one is (-14928, 18491, -3562). What am I doing wrong? This very barycentric coordinate I also use for my texture mapping, and since it gets weird big negative result, it broke the index of my texture map.

for (int Y = minY; Y <= maxY; Y++) {
    for (int X = minX; X <= maxX; X++) {
        float lambda1, lambda2, lambda3;
        getBarycentricCoordinate(v1, v2, v3, X, Y, &lambda1, &lambda2, &lambda3);
        if ((0.0f <= lambda1 && 0.0f <= lambda2 && 0.0f <= lambda3)) {
            float zValue = lambda1 * zA + lambda2 * zB + lambda3 * zC;
            if (abs(zValue) < abs(raster.zBuffer[Y * rasterWidth + X])) {// If a point is closer to the eye, which is at (0, 0, 0) then draw. zBuffer check
                float lamb1, lamb2, lamb3;
                getBarycentricCoordinate(eyeCoorV1, eyeCoorV2, eyeCoorV3, X, Y, &lamb1, &lamb2, &lamb3);

                Vector4f incidentPoint = lamb1 * eyeCoorV1 + lamb2 * eyeCoorV2 + lamb3 * eyeCoorV3;
                Vector4f l = lightInEyeCoor - incidentPoint;

                //transforming normals to eye coordinates
                Vector4f normal1 = modelViewMatrix * objects[k].vertexNormal[vn1Index];
                Vector4f normal2 = modelViewMatrix * objects[k].vertexNormal[vn2Index];
                Vector4f normal3 = modelViewMatrix * objects[k].vertexNormal[vn3Index];
                Vector4f normal = lamb1 * normal1 + lamb2 * normal2 + lamb3 * normal3; // lamb1,lamb2 lamb3 are getting weird results

                l.normalize();
                normal.normalize();

                Colorf color = pointLight.LdLs * objects[k].KaKd * fmaxf(l*normal, 0.0f); // add diffuse color
                color = color + ambientLight * objects[k].KaKd; // add ambient color
                Vector4f r = 2 * (normal * l) * normal - l;
                Vector4f v = -incidentPoint; // because eye is at (0, 0, 0) in eye coordinate, view vector is just negative incidentPoint
                r.normalize();
                v.normalize();
                //specular lighting
                color = color + pointLight.LdLs * objects[k].Ks * powf(fmax(r*v, 0.0f), objects[k].Ks.A);
                raster.pixels[Y * rasterWidth + X] = floatColorToIntColor(color);
                raster.zBuffer[Y * rasterWidth + X] = zValue;
            }
        }
    }
}

Below is the code for my barycentric coordinate

void getBarycentricCoordinate2D(Vector3f & v1, Vector3f & v2, Vector3f & v3, int X, int Y, float * l1, float * l2, float * l3) {
    float denonimator = (float)((v2.y - v3.y) * (v1.x - v3.x) + (v3.x - v2.x) * (v1.y - v3.y));
    float lambda1 = ((v2.y - v3.y)*(X - v3.x) + (v3.x - v2.x)*(Y - v3.y)) / denonimator;
    float lambda2 = ((v3.y - v1.y)*(X - v3.x) + (v1.x - v3.x)*(Y - v3.y)) / denonimator;
    *l1 = lambda1;
    *l2 = lambda2;
    *l3 = 1 - lambda1 - lambda2;
}

I have figured out the solution to this, and want to share it. What I am doing is I calculate different barycentric coordinate for each of the step I need to do: z-buffer, interpolation of normal, etc.

The correct way is to use the barycentric coordinate that is calculated when checking z-buffer to test whether a pixel inside a triangle. Because that barycentric coordinate is guaranteed to be inside the [0, 1] range (otherwise the pixel wouldnt have been drawn), it is the correct barycentric coordinate to use.


How does temporal reprojection work? Temporal anti aliasing (and other temporal algorithms) work by matching pixels this frame with pixels from the last frame and then using that information.

I get that you can use the last and current frame matrices along with motion vector information to match the pixels between frames.

What I don't get though is how do you know whether the reprojected pixel is valid or not?  For instance the old pixel may now be hidden behind a different object.

Is it by color only? If so, how are animated textures or changing light conditions handled?

One strategy mentioned in Brian Karis's talk about TAA is neighborhood clamping. The general idea is that, for the previous frame's pixel to be valid, its color should be in the color range found in the neighborhood (say 3x3 pixels) of the current pixel this frame.

This rejects history from changing light conditions, which is probably what you want anyway if you don't want moving shadows to produce ghosting.

(Animated textures, depending on the speed of the animation, could also be handled with a motion vector, if you have a predictable UV mapping or can guess reasonably well.)


Sobel edge detection line thickness My goal is to implement an edge detection algorithmus that is capable to find edges of arbitrary 3d meshes. I want to find the edges by detecting normal discontinuities. Furthermore, I want the edges to be one pixel wide.

I render the color values of my scene and the corresponding normal vectors to two different render buffer targets, i.e. to two different textures (screenTexture and normalTexture).
I use the sobel edge detection algorithmus in my fragment shader:

#version 330 core

out vec4 color;

uniform sampler2D screenTexture;
uniform sampler2D normalTexture;
uniform sampler2D depthTexture;

mat3 sx = mat3(
    1.0, 2.0, 1.0,
    0.0, 0.0, 0.0,
   -1.0, -2.0, -1.0
);
mat3 sy = mat3(
   1.0, 0.0, -1.0,
   2.0, 0.0, -2.0,
   1.0, 0.0, -1.0
);

void main()
{
    vec4 diffuse = texelFetch(screenTexture, ivec2(gl_FragCoord), 0);
    vec3 normal = texelFetch(normalTexture, ivec2(gl_FragCoord), 0 ).xyz;
    vec3 I[3];
    for (int i=0; i<3; i++) {
        float sampleValLeft  = dot(normal, texelFetch(normalTexture, ivec2(gl_FragCoord) + ivec2(i-1,-1), 0 ).rgb);
        float sampleValMiddle  = dot(normal, texelFetch(normalTexture, ivec2(gl_FragCoord) + ivec2(i-1,0), 0 ).rgb);
        float sampleValRight  = dot(normal, texelFetch(normalTexture, ivec2(gl_FragCoord) + ivec2(i-1,1), 0 ).rgb);
        I[i] = vec3(sampleValLeft, sampleValMiddle, sampleValRight);
    }

    float gx = dot(sx[0], I[0]) + dot(sx[1], I[1]) + dot(sx[2], I[2]);
    float gy = dot(sy[0], I[0]) + dot(sy[1], I[1]) + dot(sy[2], I[2]);

    float g = sqrt(pow(gx, 2.0)+pow(gy, 2.0));
    g = smoothstep(0.4,0.8, g);

    if(g > 0.2) {
        color = vec4(0., 0., 0.0, 1);
    } else {
        color = diffuse;
    }
  }

The shader produces the following result:

As you can see, the line thickness of the solids are not consistent:

Edges visible between the turquois background and the solid are 1px wide like
intended.
Edges within the object are 2px wide which is too bride.

The reason why 1. works is because I initalized normalTexture with vec3(0,0,0) normal vectors which produces a dot-product of 0 as soon as the current pixel is a scene pixel and the adjacent right or left pixel is an edge pixel of the solid.

But edges within the object are detected two times by sobel because of a gx or gy which is one time positive and one time negative. In other words, gx or gy is both times not 0 and therefore g is greater than 0.2 both of the time.

I'm totally stuck what I can do to produce consistent wide lines.

The edge between the sphere and the background is actually the one that is incorrect; you need to initialize your normal texture with a unit normal to get correct results.

The two pixels thickness is a limitation of the Sobel based edge detection and other 3x3 convolution filters: you can only detect edges twice as big as your pixels. The Robert operator uses a 2x2 convolution filter with weights making a cross, so it yields thinner edges but they are still thicker than one pixel.

If you absolutely want one pixel width, you could use filtered texture sampling and sample every half pixel when applying the Sobel operator, but the detection will probably miss some features.


How to prevent moire artifacts in this light casting algorithm? I am writing a 2D light casting algorithm with OpenGL compute shaders. The algorithm is simple:
For each light source shoot out light rays from it, the ray begins at the light coordinates, it ends at a pixel in a circle around the light source (the light distance). When calculating the ray sample the pixel underneath it to model light colour/alpha changes. The algorithm works well, but a few pixels are missed, so they stay black causing Moire-like artifacts. The lightrays are calculated in a compute shader parallelly, one thread for each pixel in the light border circle. This means for example for a light with 256 radius, there are 2*256*Pi=1608 threads on the GPU, each calculating a single ray.

Heres the relevant compute shader code:

#define LOCAL_WG_SIZE 128u
const float PI = 3.1415926535897932384626433832795;
const float RAY_LENGTH = 256.0f;

// get the number of the current thread (0...1608)
uint renderNodeNum = local_coords;
// get the endpoint of the ray, it will be on a circle
endPoint.x = int(RAY_LENGTH * sin(float(renderNodeNum) * PI / 1024.0f)) + lightPos.x;
endPoint.y = int(RAY_LENGTH * cos(float(renderNodeNum) * PI / 1024.0f)) + lightPos.y;

// vector approximation. Works, but has moire artifacts.
// I've also tried Bresenham's line algorithm, but it leaves a cross shape as the light fades which looks ugly.
vec2 dt = normalize(vec2(endPoint - lightPos));
vec2 t = vec2(lightPos);
for (int k = 0; k < RAY_LENGTH; k++) {
    coords.x = int(t.x);
    coords.y = int(t.y);

    // calculate transparency
    transpPixel = imageLoad(transpTex, coords);
    currentAlpha = (transpPixel.b + transpPixel.g * 10.0f + transpPixel.r * 100.0f) / 111.0f;
    // calculate color
    colorPixel = imageLoad(colorTex, coords);
    lightRay.rgb = min(colorPixel.rgb, lightRay.rgb) - (1.0f - currentAlpha) - transmit;
    currentOutPixel = imageLoad(img_output, coords);
    currentOutPixel.rgb = max(currentOutPixel.rgb, lightRay.rgb);
    currentOutPixel.a = lightRay.a;
    // write color
    imageStore(img_output, coords, currentOutPixel);

    t += dt;
}

Heres is how it looks:

Another example with coloured background that shows light propagation, with the same artifacts:

So I need a better algorithm to draw these light rays (or any other method that is highly parallellizable on a compute shader) that does not leave out some pixels (the black spots in the pic). I could oversample the lightray (for example shoot twice as many rays), but this would be very expensive, there must be a cheaper solution.

In principle you avoid using scatter (casting) behavior with GPU. They have offered random output coordinate write out since only shader model 5 as a need for extreme situations. But you should as general rule write your GPU code in a "gather" fashion.

The difference: the hardware threads are logically soft-locked to one output position in the render target. The scheduler decides to what rectangle (or cube) in the target buffers, the kicked thread group will output results.

So you should work around the designated destination, and figure out the start; instead of working from some start and computing a dynamic destination.

This way not only will you please the hardware by avoiding contention, and race conditions completely; but also you'll avoid holes.


Do we use 3x3 matrices in computer graphics? I've been doing some side-research on computer graphics as a hobby and came across this article on quaternions:
http://www.opengl-tutorial.org/assets/faq_quaternions/index.html#Q2

In the first section on matrices, it describes 2x2, 3x3, and 4x4 matrices and their functions.  Here is the entry on 3x3 matrices:

  3x3 matrices are used to perform low-budget 3D animation. Operations
  such as rotation and multiplication can be performed using matrix
  operations, but perspective depth projection is performed using
  standard optimised into pure divide operations.

I'm not sure if this blurb means that 3x3 matrices aren't used anymore or if we are still using them for simpler calculations on certain animations.  I understand what perspective depth projection is, but what does it mean it is performed using pure divide operations?

That quote is a very strange way to phrase things. We definitely use 3x3 matrices in computer graphics. They tend to be most useful for doing affine transformations of 2D objects. It allows you to have scale, rotation, shearing, and translation (in 2D), but not perspective transformations. I believe that's what the quote is trying to say.

To get a perspective projection of a 3D scene onto a 2D plane (such as your computer screen), at some point you will need to divide the x and y components of each point in your geometry by some factor that foreshortens lines in the z dimension. That's the divide that they are referring to. See this article for more details. In it, they describe:

  The simplest perspective projection uses the origin as the center of projection, and z = 1 as the image plane. The functional form of this transformation is then x' = x / z; y' = y / z.


Hooke's Law vector form I'm studying spring model. There is a suggested equation (Hooke's Law vector form) But, I couldn't understand how to derive that equation.

I'm reading 'Computer Animation Algorithms and Techniques Third Edition.
In the page 241, (7.88)
The equation is as follows:

($|v_{1}^* - v_{2}^*|$: the rest length)

($|v_{1} - v_{2}|$: the current length)

$F_s = (\frac{k_s|v_1 -v_2| - |v_{1}^* - v_{2}^*|}{{|v_{1}^* - v_{2}^*|}})\frac{(v_1 -v_2)}{|v_1 -v_2|}$ // I think the $k_s$ is typo.

(I referred to http://www2.cs.uregina.ca/~anima/408/Notes/Cloth/Cloth.htm)

So, maybe $F_s = k_s(\frac{|v_1 -v_2| - |v_{1}^* - v_{2}^*|}{\color{red}{|v_{1}^* - v_{2}^*|}})\frac{(v_1 -v_2)}{|v_1 -v_2|}\cdot\cdot$(1) is correct.

However, I wonder why the red part divides the equation?

If I know correctly about Hooke's Law, $F=-k_s(current_{lenght} - rest_{length})\cdot\cdot$(2).

It means that the red part should be removed like that $F_s = k_s(|v_1 -v_2| - |v_{1}^* - v_{2}^*|)\frac{(v_1 -v_2)}{|v_1 -v_2|}\cdot\cdot$(3).

For example, in 3D space, there is a spring that has the spring term as $k_s$.

The rest length is '3', so let's assume the left rest point, $v_2^*$, is fixed at (0,0,0) and the right rest point, $v_1^*$, is (3,0,0).

And then we moved $v_1^*$ to $v_1 $=(5, 0, 0) with some force.
We can calculate the force with (2), and the force is $-2k_s = -k_s(|(5,0,0) - (0,0,0)| - |(3,0,0) - (0,0,0)|)$

But if we use equation (1), the result is different.

$F_s = -k_s(\frac{|v_1 -v_2| - |v_{1}^* - v_{2}^*|}{{|v_{1}^* - v_{2}^*|}})\frac{(v_1 -v_2)}{|v_1 -v_2|} \\\quad= -k_s(\frac{|(5,0,0) -(0,0,0)| - |(3,0,0) - (0,0,0)|}{{|(3,0,0) - (0,0,0)|}})\frac{((5,0,0) -(0,0,0))}{|(5,0,0) -(0,0,0)|} \\\quad= -\frac{2}{3}k_s(1,0,0)$

The force amount of $-\frac{2}{3}k_s(1,0,0)$ is totally different from $-2k_s$

Could you explain what I'm wrong?

If we sidestep your typo (the last term has one absolute too much), both formulations are correct. They just express different things. The $k$ in Hooke's law is for a particular spring. $k_s$ is the siffness for a paricular material.

Now in the linear portion there is a direct relationship betwen these the material stffness is directly propotional to the spring factor and length of member. So for example when the rest length of a different spring with same material is 2 times as long its 1/2 as stiff.


Finding the maximum number of disconnected fragments In a plane, we have a convex polygon with n sides. What would be the maximum number of disconnected pieces / fragments if this polygon is clipped against a square?

A convex polygon has the property:

A line drawn through a convex polygon will intersect the polygon exactly twice.

From this follows that any line trough splits the convex polygon in 2 pieces.

Image 1: convex polygon split by a line.

Using this, and assuming the inner piece is discarded we can craw the conclusion that there are five cases:

When the lines of the square do not exit the polygon. You now have a piece with a hole or the entire polygon is contained and theres no hole and no piece.
When the lines exit 2 times you have one fragment*
when the lines exit 4 times you have two fragments*
when the exits are 6 times you have three fragments*
when the exits are 8 times you have four fragments *

Image 2: the five cases, clipping rectangle and remaining shards in black. Even a squares has same cases if the polygons vary, just easier to draw this way.

The last case is the maximum so answer is four or five if you count the clipped part as one piece.

* Since we haven't tackled the cases very rigorously we know that this is the highest possible answer, there could be less for each case. There shouldn't be but I so far haven't given evidence for that. But thats okay it wasn't asked.


Object rotating around origin instead of object center? I am implementing simple rotation but the object is not rotating around the local center instead it is rotating around world origin.

This is the code:

Matrix2D m = Matrix2D.Rotate((float) (Math.PI));

Point p =  new Point(new Vector2D(-0.1f,0.7f),this, new PixelData(255, 255, 0, 0));
Point p2 = new Point(new Vector2D(0,0.8f),this, new PixelData(255, 255, 0, 0));
Point p3 = new Point(new Vector2D(0.1f,0.7f),this, new PixelData(255, 255, 0, 0));

Vector2D v = Matrix2D.MatrixTimesVector2D(m, p.getPosition());
Vector2D v2 = Matrix2D.MatrixTimesVector2D(m, p2.getPosition());
Vector2D v3 = Matrix2D.MatrixTimesVector2D(m, p3.getPosition());

Point z = new Point(v, this, p.getColour());
Point z2 = new Point(v2, this, p.getColour());
Point z3 = new Point(v3, this, p.getColour());

new Triangle(z2.Transform(), z.Transform(), z3.Transform(),this).DrawTriangle();

I checked my matrices and matrix multiplication is correct.

Result without rotation:

And after rotation:

Why i am getting this weird rotation ?

Your triangle's coordinates:
(-0.1f,0.7f)
(0.0f,0.8f)
(0.1f,0.7f)
are defined with the origin at the center of the screen.

Multiplying a rotation matrix by a vertex position will rotate around the point (0,0,0). In your case, that is the center of the screen.

Possible solution:

Define the vertices as: (-0.1f,-0.5f) (0.0f,0.5f,) and (0.1f,-0.5f)
Rotate the vertices with your rotation matrix (Like you did):

Vector2D v = Matrix2D.MatrixTimesVector2D(m, p.getPosition());

Vector2D v2 = Matrix2D.MatrixTimesVector2D(m, p2.getPosition());

Vector2D v3 = Matrix2D.MatrixTimesVector2D(m, p3.getPosition());
If you want your triangle at the top of you screen, use a translation matrix:

I'm not sure what language/library you are using, but create a translation matrix by yourself or using a built in fuction. Above dx, dy and dz are the translations in x,y and z.

Multiply your positions v, v2 and v3 by the translation matrix.

Google tranlsation and rotation matrix and you will understand better! Good luck!


Why isn't multi-gpu handled by the driver? I hope I'm not off topic here, but this seemed like a good place to ask something like this.

I've been hearing about SLI and Crossfire for a long time and how support for it is lacking.

I never could understand why were the developers ones that needed to support it.

GPUs are immensely parallel things. They have thousands of cores that run same code at the same time (generally speaking). The game will run with same code on a GPU with 100 cores and with 1000 cores, it will just scale itself differently.

I want to know what stopped the GPU companies from dealing with this within the drivers.

Why not have the driver detect two physical GPUs and present them to the system as a one logical GPU? It would then make sure that the workload is evenly distributed, while using the SLI bridge to synchronise and exchange the data across the GPUs as needed.

If this was really that great as it looks to be, I'm sure they'd be doing it that way, but they aren't. My question is, why not, where does this idea fall apart?

Only reason I can think of is that the GPUs can't handle both the communication with the CPU and with each other fast enough to make the whole thing feasible, and in order to do so, architectural changes would be need which would not yield any improvements in single GPU systems, but only multi GPU system, which would be far, far rarer.

What gives?

Your assumption is wrong
It actually was this way, in fact, things have changed the other direction.
History
Currently (pre nvidia link consumer graphics) you've got to copy the memory over from separate GPUs to render in cross fire and SLI, as global memory transfer speeds will slow down scaling, and as far as I can tell it did not work as a general direct memory acessor, with only 2G/s bandwidth even with the HB version, it is only meant to be a framebuffer copier.   You otherwise had to send the same information across all GPUs via PCIE.

SLI allows two, three, or four graphics processing units (GPUs) to share the workload when rendering real-time 3D computer graphics. Ideally, identical GPUs are installed on the motherboard that contains enough PCI-Express slots, set up in a master-slave configuration. All graphics cards are given an equal workload to render, but the final output of each card is sent to the master card via a connector called the SLI Bridge. For example, in a two graphics card setup, the master works on the top half of the scene, the slave the bottom half. Once the slave is done, it sends its render to the master to combine into one image before sending it to the monitor.

(emphasis mine).
But you didn't typically actually treat this set up as one GPU in the way you want in old times because if you wanted to cooporate work loads instead of merely splitting workloads that required GPU->GPU communication, which necessarily had to be done VIA pcie, which would be very slow.
At least for opengl, SLI/Xfire rendering was handled via Driver treating the two as the same. https://stackoverflow.com/a/43812678/2036035 You didn't have the option of treating them as separate if they were linked.
Modern
Today however, vulkan and DX12 essentially give more control.

One feature in particular goes a long way toward filling a Vulkan gap relative to Microsoft's API: explicit multi-GPU support, which allows one program to spread its work across multiple GPUs. Unlike SLI and Crossfire of old, where the task of divvying up the rendering between GPUs was largely handled by the driver, this support gives control to the developer. With the addition, developers can create "device groups" that aggregate multiple physical GPUs into a single virtual device and choose how work is dispatched to the different physical GPUs. Resources from one physical GPU can be used by another GPU, different commands can be run on the different GPUs, and one GPU can show rendered images that were created by another GPU.

(emphasis mine)
Today you can look at these GPUs separately.


Debugging refraction in a sphere I have been working on recreating some of the scenes from Peter Shirley's raytracing books using compute shaders and the new Metal raytracing API.  It is going well but I am having some trouble with the glass material.
I've read Pete's blog post on this - http://psgraphics.blogspot.com/2020/04/deugging-refraction-in-sphre.html
But I am still unsure as to what is the cause of the issue that you can see in the pictures I have attached.
The glass appears grainy and you get an effect similar to z-fighting when you move the camera around.

This is the code for the glass material:
else if (instances.materials[instanceIndex] == glass)
{
    bool front_face = dot(normalize(ray.direction), worldSpaceSurfaceNormal) < 0.0 ? true : false;

    float kEtaRatio = front_face ? kEtaAir / kEtaGlass
                                 : kEtaGlass / kEtaAir;

    ray.origin = worldSpaceIntersectionPoint;
    ray.direction = refract(normalize(ray.direction), worldSpaceSurfaceNormal, kEtaRatio);
}

I have implemented all the code for the full material with Schlick approximation etc, however it appears that it is this first step shown here that is causing the issue.
I am only using one sample per pixel at the moment but I'm unsure why that would only cause problems with the glass and not the other materials?
Any advice would be greatly appreciated.

You need to make use of a discriminant, also when dealing with floating point comparisons it is wise to use an epsilon (small value for accommodating fp-error).
Also keep in mind that it matters whether the surface normal is pointing along the ray, or against it.
Below is a function which I use in my own raytracer, also based on the same book. It is written in C and refracts a given vector3. You would assign the return value to your ray.direction in your case.
// vector*vector means dot product here

#define EPSILON = 0.000001f

static inline vec3 refract(
    vec3 dir, // direction of incoming light ray
    vec3 n, // surface normal of refracting surface, pointing outwards (so roughly in the against the direction of light ray)
    float ni_over_nt // refractive indices fraction
){
    float dt = dir*n; // dot prod.
    float discriminant = 1.f - ni_over_nt*ni_over_nt*(1.f-dt*dt);
    if (discriminant > EPSILON){
        return (dir - n*dt)*ni_over_nt - n*sqrtf(discriminant);
    } else {
        // total internal reflection
        // reflect the direction against the surface normal
        return reflect(direction, normal);
    }
}

Usage of the function is as follows:
if (direction*normal) > EPSILON) // dot prod. of ray dir. and surf. normal
{
    // ray is entering the object
    nOut = -normal;
    ni_over_nt = mat->IOR; // surface material index of refraction.
} else {
    // ray is exiting the object
    nOut = normal;
    ni_over_nt = 1.f/mat->IOR;
}

vec3 refracted = refract(direction, nOut, ni_over_nt);
// this is your refracted ray direction

Added bonus: reflection function used in the refraction:
static inline vec3 reflect3(vec3 v, vec3 n){
    // reflect a vector v off of a plane with normal n
    return  v -n*2*(v*n); //v*n = dot prod.
}


What do the OpenGL version strings from glxinfo actually mean? Note: I wasn't 100% certain if this question belonged here or in Ask Ubuntu. I figured since it was about OpenGL programming I'd put it here, but if I should move it to Ask Ubuntu, please let me know.
Background
I'm new to OpenGL and needed to figure out which version I could program in on my PC. A few Google/Stack Exchange searches shows that the usual solution is to run glxinfo | grep version and read the results, which I did and got the following:
server glx version string: 1.4
client glx version string: 1.4
GLX version: 1.4
    Max core profile version: 4.5
    Max compat profile version: 3.0
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
OpenGL core profile version string: 4.5 (Core Profile) Mesa 18.0.5
OpenGL core profile shading language version string: 4.50
OpenGL version string: 3.0 Mesa 18.0.5
OpenGL shading language version string: 1.30
OpenGL ES profile version string: OpenGL ES 3.2 Mesa 18.0.5
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

The confusing part is the following three lines:
OpenGL core profile version string: 4.5 (Core Profile) Mesa 18.0.5
OpenGL version string: 3.0 Mesa 18.0.5
OpenGL shading language version string: 1.30

Everything I've read says to focus only on the Core Profile line, but I'm unable to draw anything unless I load functions and create a context under 3.0 or lower. Plus, when I call glGetString(GL_VERSION) I get 3.0 Mesa 18.0.5 printed to the terminal, but I'm not sure if that's because my hardware only supports 3.0 or if that's because I loaded functions for that level. Also, when I start writing shaders (I'm currently using Qt and an OpenGL Widget so haven't made a shader of my own yet) will I only be able to use GLSL 1.3 instead of 3.0+?
My Question
My big question is, what do the output strings actually mean? I can't find that answer. Which one tells me what my hardware can actually do? And, what do the other strings mean? Why am I getting a Core Profile of 4.5 when (as far as I can tell) I can't actually use 4.5? If my hardware can perform 4.5 operations, how do I make it do so?

When creating a OpenGL context you have to specify a profile mask, which is a way of telling the driver whether you want to use compatibility ("legacy") features or not.

Drivers are required to implement all core profile features, but compatibility profile is optional - creating a compatibility context can fail even if the driver supports the version you've requested.

The output from glxinfo you've pasted shows that you can create GL context with version up to 4.5 only when using core profile. If you want to use compatibility profile, the highest version you can request is 3.0.

One reason of the problem you are having is that you:

requested context version greater than 3.0, and
requested a compatibility profile (by setting GLX_CONTEXT_COMPATIBILITY_PROFILE_BIT_ARB in GLX_CONTEXT_PROFILE_MASK_ARB).

If removing compatibility bit doesn't help, you'll need to double check if other context attributes are correct.

What I described above is documented in a more detailed way here.


What is wrong with my glDrawElement code I am reading a model from an obj file and draw it using glDrawElements. For some reasons, only a small part of it is drawn, even though I dont even have back-face culling turned on.

// vertices
glEnableVertexAttribArray(0);
float * objectGeometry = scene.objects[k].verticesAsFloatArray();
glVertexAttribPointer(0, 4, GL_FLOAT, 0, 0, objectGeometry);

// normals
glEnableVertexAttribArray(1);
float * objectNormals = scene.objects[k].normalsAsFloatArray();
glVertexAttribPointer(1, 3, GL_FLOAT, 0, 0, objectNormals);

// indices
int * indices = scene.objects[k].facesAsIndiceArray();
glDrawElements(GL_TRIANGLES, scene.objects[k].vertices.size(), GL_UNSIGNED_INT, indices);

glDrawElements specifies the second argument to be the number of elements to be taken from the buffer of indices, not the number of vertices. The correct number would be the number of triangles times three.


Is anti-aliasing color processing correct in this image? I have the impression that there is a problem with anti-aliasing in a 2D GUI framework using opengl. To demonstrate the problem I made a program that draws gray ramps. The result is shown below.

The top and bottom gray ramps are identical and considered the reference ramps. It is a copy from "What every coder should know about gamma". This is what I currently assume that we should see when gray colors are correctly encoded. There seam to be a rounding error in this ramp, so an exact match is not required. The right most rectangle is not perfectly white.
The second gray ramp counting from the top is produced by drawing each rectangle as a stack of black lines not thicker than a pixel on a white background. The thickness of the lines reduces linearly as we go from black to white. The gray color result thus purely from anti-aliasing.
The third gray ramp is simply a sequence of rectangles filled with linearly increasing gray values.
I was expecting that all gray ramps would look identical, but clearly the second gray ramp is not. It seam that a gamma correction has been applied to it while I think it should not.
It generates artifacts which are strongly visible in font rendering as shown below.

The text a has been produced with the library in question, and the text b has been produced with another font rendering library.
Another test I did to put in evidence the artifacts resulting from the anti-aliasing color processing is to draw slightly slanted lines.

The lines above are produced by the library in question and the lines below by another library. The thickness of the lines below look more uniform. The lines above look wrong and distorted.
The author of the 2D graphic library claims that his color processing with anti-aliasing and filled rectangle is correct.
Who is right ? What argument could I use to resolve this disagreement ?
The more I work on this subject, the more I get confused. For instance, by using a color picker I see that the color in the center of the ramp of the reference image is close to 0x7F. Is the color picker giving the color in linear space or the sRGB color value ? The screen color picker is Gpick (Gnome Linux).
I know that we should work in linear color space when performing operations on images. What is unclear is what am I looking at ? The color values I gave to rectangles where linear, not sRGB, or not gamma corrected. The color ramp I see seam linear. The values given by the color picker seam to match the linear value I gave.

The anti-aliasing rendering is correct.
The reference gray ramp (top and bottom) is linear in sRGB color, not in brightness.
The second gray ramp made with anti-aliasing is linear in brightness. The rectangle color (black in color sRGB space) is mapped to linear space (remains black), then anti-aliasing is computed, and the resulting image is converted back to sRGB. It is thus not linear in sRGB value anymore.
The RGB values given by a screen color picker are in  sRGB color space, and the color values of a screen capture are in the sRGB color space.
SVG and nearly all browser graphic library (skia and cairo) are doing it wrong. They compute anti-aliasing over sRGB instead of linear color. They thus don't convert back from linear to sRGB. The same mistreatment  is also visible on Windows, MacOs and iOS.
The filled rectangle gray ramp doesn't have this problem because the conversion from sRGB to linear and its inverse cancel out.


WebGL Can't render anything, wrong projection? I'm trying to understand and learn WebGL and computer graphics from the bottom  up, that is why I started to work on my own little library for that. I've spent a couple of days looking for a right answer but I can't make it work.

I have column major matrices and I'm just trying to render a basic rectangle but for some reason no matter what I do after multiplication by a perspective matrix the vertex's Z is always out of bounds.

My perspective matrix calculation :

static perspective(fov, aspect, near, far) {
    const r = fov * aspect;
    const l = -4;
    const t = r;
    const b = l;
    const matrix = new Matrix4();
    matrix.n11 = (2 * near) / (r - l);
    matrix.n12 = 0;
    matrix.n13 = (r+l)/(r-l);
    matrix.n14 = 0;
    matrix.n21 = 0;
    matrix.n22 = (2 * near) / (t - b);
    matrix.n23 = (t+b)/(t-b);
    matrix.n24 = 0;
    matrix.n31 = 0;
    matrix.n32 = 0;
    matrix.n33 = (near + far) / (near - far);
    matrix.n34 = (2 * near * far) / (near - far);
    matrix.n41 = 0;
    matrix.n42 = 0;
    matrix.n43 = -1;
    matrix.n44 = 0;
    return matrix;
}

My vertex shader:

 this.vertexShaderScript =
        '\r\n' +
        'precision highp float;\r\n' +
        'uniform mat4 u_model;\r\n' +
        'uniform mat4 u_view;\r\n' +
        'uniform mat4 u_projection;\r\n' +
        'attribute vec3 a_position;\r\n' +
        'attribute vec4 a_color;\r\n' +
        'varying vec4 v_color;\r\n' +
        'void main(void) {\r\n' +
        '    v_color = a_color;\r\n' +
        '    gl_Position = u_projection * u_view *  u_model * vec4(a_position, 1.0);\r\n' +
        '}\r\n';

Try This... I guess you got the matrix wrong.

matrix.n11 = -(2 * near) / (r - l);
matrix.n12 = 0;
matrix.n13 = 0;
matrix.n14 = -near*(r+l)/(r-l);
matrix.n21 = 0;
matrix.n22 = (2 * near) / (t - b);
matrix.n23 = 0;
matrix.n24 = -near*(t+b)/(t-b);
matrix.n31 = 0;
matrix.n32 = 0;
matrix.n33 = (near + far) / (near - far);
matrix.n34 = (2 * near * far) / (near - far);
matrix.n41 = 0;
matrix.n42 = 0;
matrix.n43 = -1;
matrix.n44 = 0;

I referred the following site after I googled =>Link


Using imageLoad/imageStore to perform atomic add I have a rgba16f texture that I fill with values using nvidia GL_NV_shader_atomic_fp16_vector extension, which allows to perform imageAtomicAdd on half float RGBA images, with multiple shader threads using same index. However, it doesn't support rgba32f.
Can I use imageLoad/imageStore to perform same thing with such image buffer as written below?
vec4 prevVal = imageLoad(tex, index);
prevVal += currVal;
imageStore(tex, index, prevVal);

I'm not sure how correctly set the coherency of such operations with correct barriers usage because I have not found any example. Or if it is even possible.

If you could use imageLoad/Store to do atomic operations, there would be no need for atomic operations as a distinct class of operations. So no, you can't. There is no correct set of barriers or other properties that will allow this to work generally the way atomic operations do.


What's the difference between two code in shader? I'm doing a project with assimp.

I got confused with the weird situation for me.
I think the both code are exactly same, but the result is different.

Why the codes act differently?

(vertex shader)

#version 330 core
layout (location = 0) in vec3 aPos;
layout (location = 1) in vec3 aNormal;
layout (location = 2) in vec2 aTexCoords;

// declare an interface block; see 'Advanced GLSL' for what these are.
out VS_OUT {
    vec2 TexCoords;
} vs_out;

out vec2 TexCoords;

uniform mat4 model;
uniform mat4 projection;
uniform mat4 view;

void main()
{
    TexCoords = aTexCoords; ----(1)
    //vs_out.TexCoords = aTexCoords; ----(2)
    gl_Position = projection * view * model * vec4(aPos, 1.0);
}

(fragment shader)

#version 330 core
out vec4 FragColor;

in VS_OUT {
    vec2 TexCoords;
} fs_in;

in vec2 TexCoords;

uniform sampler2D texture_diffuse1;
uniform vec3 lightPos;
uniform vec3 viewPos;

void main()
{
    FragColor = vec4(texture(texture_diffuse1, TexCoords).rgb, 1.0); ----(1)
    //FragColor = vec4(texture(texture_diffuse1, fs_in.TexCoords).rgb, 1.0); ----(2)
}

If I use the code above, the result is as follows

but, instead of (1) pair, if I use the pair (2), then the result is strange.

Why it happened?

I don't have enough rep to comment :(

Hi,

It looks like with (2), fs_in.TexCoords is == vec2(0.0,0.0). Do you keep both in VS_OUT {}/ out VS_OUT {} and out vec2/in vec2 ? If so, it's possible that as both have the same name (TexCoords) the compiler is confused. Try using only one of the two options.


How is Anti Aliasing Implemented in Ray Tracing? After reading a few articles online I can confidently say I am clueless on how Anti-Aliasing works when using Ray Tracing.

All I understand is that A Single Pixel/Ray is split into 4 sub-pixels and 4 rays rather than 1.

Could somebody please explain how this is done (preferably with code)?

Raxvan is completely right that "traditional" anti aliasing techniques will work in raytracing, including those that use information such as depth to do antialiasing.  You could even do temporal anti aliasing in ray tracing for instance.

Julien expanded on Raxvan's 2nd item which was an explanation of super sampling, and showed how you'd actually do that, also mentioning that you can randomize the location of the samples within the pixel but then you are entering signal processing country which is a lot deeper, and it definitely is!

As Julien said, if you want to do $N$ samples per pixel, you can break the pixel up into $N$ evenly distributed sample points (on a grid basically) and average those samples.

If you do that, you can still get aliasing though.  It is better than NOT doing it, because you are increasing your sampling rate, so will be able to handle higher frequency data (aka smaller details), but it can still cause aliasing.

If you instead take $N$ random samples within a pixel, you are effectively trading aliasing for noise.  Noise is easier on the eyes and looks more natural than aliasing, so is the preferred result usually.  I believe it's even provably the ideal situation with higher sample counts but don't have more info on that ):

When you use just "regular" random numbers like you'd get from rand() or std::uniform_int_distribution, that is called "white noise" because it contains all frequencies, like how white light is made up of all other colors (frequencies) of light.

Using white noise to randomize the samples within a pixel has the problem that sometimes your samples will clump together.  For instance, if you average 100 samples in a pixel, but they ALL end up being in the upper left corner of the pixel, you aren't going to get ANY information about the other parts of the pixel, so your final resulting pixel color will be missing information about what color it should be.

A better approach is to use something called blue noise which only contains high frequency components (like how blue light is high frequency light).

The benefit of blue noise is that you get even coverage over the pixel, like you get with a uniform sampling grid, but, you still get some randomness, which turns aliasing into noise and gives you a better looking image.

Unfortunately, blue noise can be very costly to compute, and the best methods all seem to be patented (what the heck?!), but one way to do this, invented by pixar (and patented too i think but not 100% sure) is to make an even grid of sample points, then randomly offset each sample point a small amount - like a random amount between plus or minus half the width and height of the sampling grid.  This way you get a sort of blue noise sampling for pretty cheap.

Note that this is a form of stratified sampling, and poisson disk sampling is a form of that too, which is also a way of generating blue noise:
https://www.jasondavies.com/poisson-disc/

If you are interested in going deeper you probably will also want to check out this question and answer!

What is the fundamental reasoning for anti aliasing using multiple random samples within a pixel?

Lastly, this stuff is starting to stray into the realm of monte carlo path tracing which is the common method for doing photorealistic raytracing.  if you are interested in learning more about that, give this a read!

http://blog.demofox.org/2016/09/21/path-tracing-getting-started-with-diffuse-and-emissive/


3D scene construction and change of coordinates My book says:

  After a scene has been constructed we transfer object descriptions to the viewing-coordinate reference frame. This conversion of object descriptions is equivalent to a sequence of transformations that superimposes the viewing reference frame onto the world frame, so we translate  the  view reference frame onto the origin and  rotate by an angle to align the view to the world.

So, my confusion is: if we're transferring object descriptions from the world to view, shouldn't we superimpose the world onto the view,   and hence take the inverse  of  the above transformation? Why is my book saying things the other way around?

I assume the term object description refers to the set of 3D vertices a mesh is made of. And I also assume reference frame actually means coordinate space. For a book on computer graphics, it sure uses some very uncommon words.

In computer graphics, we often have to transform a set of vertices from one coordinate space to another. For example, an artist may have modelled a rubber ducky, aligned in such a way that it looks down the positive x-axis and it uses centimeters for its units.

When we place the ducky in our world (let's say it uses meters as the primary unit), we scale, rotate and translate it so that it neatly floats on the water in a bath tub.

Then we have our camera, which can move freely around in our scene with its own translation and rotation parameters. And finally there is the notion of projection, which takes into account the field of view and the aspect ratio of our frame to project the 3D point onto a 2D image.

Our job is to calculate exactly where in our 2D image the 3D vertex ends up, if at all. And for this, we will have to go through the sequence of all our coordinate spaces: from object to world to camera (a.k.a. view) to projection.

So that's how most graphics programmers visualise this process and hence why the book also follows this sequence: it takes the object and follows it through its transformations until it ends up in our 2D image.

It seems you prefer it the other way around: you take the camera as a fixed object and rotate the world around it. But you have to admit this isn't very logical, as the camera sure isn't fixed and the world does not revolve around it.


Rotation of meshes in PyTorch I am doing some project on object reconstruction and I need to rotate an object in PyTorch during the training to make sure I am augmenting the data properly. My data consists of points and it's normals. My question is, how should I (or should I train them at all) rotate the normals of my mesh. Note that this rotation also needs to keep the object inside the unit cube, where it was before the transformation.

My question is, how should I (...) rotate normals of my mesh.

Normals are rotated the same way as the rest of your mesh/point cloud. But make sure that you use a "pure rotation matrix" for this (no scaling, no translation). Since rotations preserve lengths, you do not need to renormalize your normals (usually - floating-point errors might accumulate).

This sentence of you

  Note that this rotation also needs to keep object in unit cube, where it was before running this code.

lets me guess, that you are actually asking how to rotate an object and its normals and not just the normals. In python, you can use the packages SciPy and NumPy for that.

The Rotation package (documentation) of SciPy helps you to generate an arbitrary rotation matrix. Then you can use NumPys matmul function to rotate your points and normals.

Here is a short example script that rotates a single point 20 degrees around the z axis:

from scipy.spatial.transform import Rotation as Rot
import numpy as np

p = np.array([1, 1, 0]).transpose()
rot_mat = Rot.from_euler(seq="z", angles=20, degrees=True).as_matrix()

p_r = np.matmul(rot_mat, p)

print(p)
print(p_r)
print(np.linalg.norm(p))
print(np.linalg.norm(p_r))

output:

[1 1 0]
[0.59767248 1.28171276 0.        ]
1.4142135623730951
1.414213562373095

  Info: Depending on your SciPy version you might need to replace as_matrix with as_dcm

As you can see, apart from some minor floating-point error, the length (last 2 outputs) did not change. Furthermore, I chose the example point, so that it gets outside the unit cube. To address this problem, you need to find the maximum absolute value inside the transformed data. You can use amax and amin to get the maximum and minimum value (see also here) if you put all your coordinates into one large 2d array (a matrix). Then compare the absolute values of the amax and amin results and pick the larger one. Divide all your coordinates (not the normals!) by this value, to scale your model back into the unit cube. Alternatively, you can create a uniform scaling matrix

$$
\begin{bmatrix}
s&0&0\\
0&s&0\\
0&0&s
\end{bmatrix}
$$

with $s$ being the reciprocal of the maximum absolute value. Multiply this matrix with all points.

To sum it up:

calculate the rotation matrix with SciPy
Multiply the rotation matrix with all points and normals
find the maximum absolute value in the rotated data
scale your coordinates with the found maximum value

Remark: If you put your points and normals into 2 matrices, you can apply the rotations and scaling matrices without a loop, which makes it much faster.


How can I perform a triangle inside test in polygon meshes? I have 3 vertices (V1, V2, V3) randomly selected on a regular triangle mesh. For these 3 vertices, I have computed the geodesic distance and the path (by using Dijkstra) among them and formed a triangle-like surface as in the above figure.

Now, I have the vertices that lie in each path and can compute geodesic distances from a given vertex.

What I want to do is to get the vertices or triangles that lies in triangle-like area. How can I do this?

There is an alternative method that relies on flood filling. First arrange your edge data into a loop where the edges are forming a counterclockwise loop. Then start at an arbitrary point on the loop and pick edges joining that point. Use the outbound boundary edge and cross it with the other outbound edge, if it points in the direction of the face normal then it's an edge to be included, if not discard it. From this edge continue until you hit a boundary edge, at which point you terminate the fill. Continue at a yet to be visited boundary edge vertex.


Automatic generation of UV maps Given an arbitrary triangle mesh, is it possible to automatically generate a UV mapping for it, and if so, how is it done?

Short answer: Yes, but it won't be pretty.

Explaination: While there are algorithms to automagically UV map polygon soup, the mapping probably wont be ideal. UV mapping is an art really. Choosing where to hide the seams and where to put them to limit stretching and optimize texture space.

That said, there are lots of tools that can make manual UV mapping a heck of a lot easier. For example: http://knowledge.autodesk.com/support/maya-lt/learn-explore/caas/CloudHelp/cloudhelp/2015/ENU/MayaLT/files/Create-UVs--Automatic-Mapping-htm.html


Parametrize set of unordered points in 3d space Let's assume that we have some set of points in 3d space sampled from some arbitrary surface, without self intersections and mostly flat. Is it possible to get any parametrization over this set of points?  And if it is - how?

My first idea was to find a plane (Maybe through PCA), or more sophisticated smooth surface which approximate this set of points, project each of these points on this fitted plane/surface and get corresponding parameters. But maybe there is better solutions for this problem.

Thanks for any help.

You could try natural neighbour interpolation or Sibson coordinates for this. These coordinates are based on a Voronoi tessellation of the points.
The coordinates have been extended to an approach where the points lie on some surface in this publication.


Texture glitch when using simple pixel shader I have an issue with rendering my textures in Silverlight. When I look at it from above everything looks fine:

But if I only change angle of watching it it looks terrible:

I am using very simple pixel shader (hlsl):
float4 main(VsOutput vertex) : COLOR
{
        float4 texColor = tex2D(textureSampler, texCoord);

        return texColor;
}

If I don' t use that pixel shader and load texture via BasicEffect like:
BasicEffect.texture = myTexture

Everything looks fine. Why my pixel shader affects texture so bad?

I expect that it might be somehow connected with mipmapping, but I am not sure - my texture is pretty small so I guess it shouldn' t have such a big effect. Anyway I need some advices :).

So I followed Alan Wolfe suggestion (in comment to my question) And turned out he was right. I was using SamplerState.LinearWrap and that was the issue. When I changed this to AnisotropicWrap it looked much better. Below are some examples of different sampling types and how they affect texture:

graphicsDevice.SamplerStates[0] = SamplerState.PointWrap;

graphicsDevice.SamplerStates[0] = SamplerState.LinearWrap;

graphicsDevice.SamplerStates[0] = SamplerState.AnisotropicWrap;


Separating photos from other images algorithmically? Can an algorithm be derived to classify digital images according to how they were generated, i.e. the classes would be:

Real-world photograph captured with an actual camera.
A Computer-Generated realistic scene.
A hand-drawn image or a painting.

Could perhaps a Machine Learning approach, geared towards analyzing statistics of the signal provide sufficient insight on the problem ?
In that case, is there any known implementation of such algorithm publicly available ?

Difficulty depends on what you really want to classify, i.e. what exactly your data base of images is made ok.
For simple hand-drawn or computer art, I would think of just looking the color histogram (but for complex art with gradients; but still these gradients are likely to induce a very smooth colormap).

At the other extreme, distinguishing realistic computer graphics image from reality is probably impossible.


Clarifying vulkan glsl std140 So I have a uniform block that looks something lile this

{
mat4;
mat4[128];
uint[16]
}

The block should total up to 8320 bytes. After aligning by 256 bytes for nvidia, that brings the total allocation size to 8448 bytes.

However, according to renderdoc and spirv-cross the total block size should be 8512 bytea. Can someone explain to me why there is a 64 byte padding at the end?

The block should total up to 8320 bytes.

No, it shouldn't. You use a uint[16] array. Well, std140 layout makes it clear that the array stride for any array is always rounded up to the stride of a vec4. So that array takes up 256 bytes.

You should instead use a uvec4[4] instead, which you would index from like a multidimensional array.


Difference between isnan and comparison to self In GLSL, what is the difference between isnan(x) and !(x == x)?

IEEE-754 defines the isNaN property, which determines if a value is NaN. However, it is also defined to be a non-signaling operation; calling isNaN on an sNaN will never result in a signaled failure. By contrast, equality testing may signal when given sNaN values.

Now, GLSL explicitly states that signaling doesn't matter:

  Support for signaling NaNs is not required and exceptions are never raised.

But of course, there's also the fact that isnan is a vector function, while !(x == x) only works on scalars. So if you want to test if any of the elements of a vec3 are NaN, you can do any(isnan(some_vec)), while the equality-testing equivalent would be even more obtuse and verbose: any(not(equal(some_vec, some_vec))).

And all of that ignores the simple usability reasons for having isnan. !(x == x) is an idiom which must be learned, while isnan is an easily looked up function whose name tells you want it does. After all, most users of floats tend to forget about NaN, so such an expression seems like a tautology, while "isnan" explicitly reminds such people that NaNs are a real thing and they are reading code where that fact is important.

Also, as previously stated, isNaN is a property defined by IEEE-754. It is also a function provided by the standard libraries of C99, C++11, Java, C#, Swift, Go, JavaScript, and I can keep going, but I think my point is clear.


Value range of normal distribution function Is the normal distribution term D in the Cook-Torrance BRDF supposed to be between 0 and 1? What does this number actually represent?

The term D is a probability density function (PDF) of microfacets' directions. Vaguely speaking, it says what "portion" of all microfacets points in the given direction.

Since it is a PDF, it doesn't have to be less or equal to 1. It can, and often is, higher than 1 for some directions.


How to set the GLFW OpenGL target? I'm programming on a Mac and I'm learning OpenGL in the library GLFW.

My problem is that my Mac supports OpenGL up to version 4.1, but when I try to compile the shaders with version 410 it says ERROR: 0:1: '' :  version '410' is not supported.

How do I set the version that GLFW should use?

I changed the core profile to 4.1 by setting the window hints.

glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3);
glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 2);
glfwWindowHint(GLFW_OPENGL_FORWARD_COMPAT, GL_TRUE);
glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE);


OpenGL Instanced Rendering with Level of Detail Problem:
I'm trying to draw a scene with a lot (about 10000) of similar objects (in my case bushes, but could be anything). For each bush I have three meshes with a different number of triangles and I want to use one of these meshes depending on the distance to the camera.

First approach:
I can select the correct mesh on the CPU, update the model matrix after every bush and then draw the VBO with the correct mesh. The problem is that I need to update the model matrix 10000 times per frame ... you can guess what this means performance wise.

Second Approach:
Again select the correct mesh on the CPU, but this time use instanced rendering to draw all bushes with lowest lod in one draw call, then the bushes with middle lod and then the bushes with high lod. I have stored the bush positions already on the GPU. But my problem here is that I still need to send an array as uniform variable which contains the information which mesh has to be drawn at which position. With this approach i run into the limit of uniform variables.

Question:
What I would like to do is to calculate the distance from the bush to the camera on the GPU (no problem for example in the vertex shader). And then draw only the VBO with the correct level of detail depending on the outcome. Therefore my question is if there is any possibility to select which VBO to draw on the GPU (I don't see any possibility to do this, since in the Vertex and Fragment shader I am a already to far down the pipeline)? Or do you have another idea that I could try?

Best, Albert

IMO, a variation of your second approach is going to be the simplest, and could be quite fast.

Have you profiled anything yet? A CPU should chew through 10000 matrix multiplications extremely quickly. If you wanted to get even more performance, you could SIMD / thread the update. That said, the slowest part is probably going to be the bandwidth of sending 10000 matrices over the PCIE bus.

But, again, this is all supposition. Try it out, and profile.

I was thinking something like this:

(I apologize for the DX functions. I'm not super familiar with the corresponding OpenGL functions. That said, OpenGL should be quite similar)

// Variable definitions

vector<matrix> lod0;
vector<matrix> lod1;
vector<matrix> lod2;

ID3D11Buffer lod0VertexBuffer;
uint lod0VertexCount;

ID3D11Buffer lod1VertexBuffer;
uint lod1VertexCount;

ID3D11Buffer lod2VertexBuffer;
uint lod2VertexCount;

ID3D11Buffer instanceBuffer;

/// ..................

/// Per frame...
lod0.clear();
lod1.clear();
lod2.clear();

for (uint i = 0; i < bushes.size(); ++i) {
    float distance = distance(bushes[i].pos, camera.origin);
    if (distance < 50.0f) {
        lod0.push_back(bushes[i].matrix);
    } else if (distance < 100.0f) {
        lod1.push_back(bushes[i].matrix);
    } else {
        lod2.push_back(bushes[i].matrix);
    }
}

D3D11_MAPPED_SUBRESOURCE mappedResource;
ID3D11Buffer vertexBuffers[] = {
    lod0VertexBuffer,
    instanceBuffer
};

// Update lod0 matrices
context->Map(instanceBuffer, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
memcpy(mappedResource.pData, &lod0[0], sizeof(matrix) * lod0.size());
context->Unmap(instanceBuffer, 0);

// Draw lod0
context->IASetVertexBuffers(0, 2, vertexBuffers, nullptr, nullptr);
context->DrawInstanced(lod0VertexCount, lod0.size(), 0, 0);

// Update lod1 matrices
context->Map(instanceBuffer, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
memcpy(mappedResource.pData, &lod1[0], sizeof(matrix) * lod1.size());
context->Unmap(instanceBuffer, 0);

// Draw lod1
vertexBuffers[0] = lod1VertexBuffer;
context->IASetVertexBuffers(0, 2, vertexBuffers, nullptr, nullptr);
context->DrawInstanced(lod1VertexCount, lod1.size(), 0, 0);

// Update lod2 matrices
context->Map(instanceBuffer, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
memcpy(mappedResource.pData, &lod2[0], sizeof(matrix) * lod1.size());
context->Unmap(instanceBuffer, 0);

// Draw lod2
vertexBuffers[0] = lod2VertexBuffer;
context->IASetVertexBuffers(0, 2, vertexBuffers, nullptr, nullptr);
context->DrawInstanced(lod1VertexCount, lod2.size(), 0, 0);

If this ends up being too slow, you could try to do this calculation in a compute shader, then draw the result using an indirect draw call. See AZDO.