What is the definition of "motion to photon" in VR? The rough definition of "MTP" is clear. But the exact one is not. You can refer to these links:

What is Motion-To-Photon Latency? | chioka.in
Motion-to-photon latency | xinreality.com/wiki.

But from different online articles, we get different definitions regarding how the value is measured:

From Atman Binstock (Oculus) blog, we can see:
MTP = T2(time to flip) - T1(Time of PO).
But, T2 is just beginning to display the image, not finish displaying.
From John Carmack (Oculus) blog:
MTP = T2(finish displaying) - T1(Time of PO).
From Alex Vlachos (Valve) slides: Advanced VR Rendering, GDC 2015:
MTP = T2(finish displaying) - T1(Time of PO)
From Qualcomm blog (sorry it is in Chinese):
MTP = T2(half of finish displaying) - T1(Time of PO)

I think Qualcomm's explanation is the best. If it is correct, we can relax the rendering time budget. What is your opinion?

Ideally, motion-to-photon latency is exactly what it says. It measures time from when the user physically makes a motion (say, begins to turn their head) to when the photons physically coming out of the display change to reflect that motion.

With a bit of apparatus, this can be measured directly—for instance by using a high-speed camera aimed at the headset. You give the headset a light bump with your finger (or use some automated device to bump it), and you can see in the high-speed video when the motion occurred and when the image visible on the screen updated to reflect the motion.

The complication leading to different definitions occurs because many VR headsets have rolling displays, where the physical pixels on the display light up progressively, one row at a time, as the image is scanned out from the host PC. So, when you define motion-to-photon latency, you have to say which photons you're talking about. The very first ones to scan out? The very last ones? Somewhere in the middle?

You can pretty much make whichever definition you want here about what point in the scanout process to use as your reference. There may be an argument for one definition or another based on psychovisual effects (i.e. how much of your visual field must be updated in order for your brain to perceive motion?) but this is beyond my knowledge.

On the other hand, if a headset has a global display instead of a rolling one, then the pixels are kept dark while being updated and then the whole display is flashed on and off at the end of scanout. In this case it's clear that the appropriate reference point to measure is the beginning of the flash, since all the photons come out at the same time.

Finally, it's worth noting that latency measurements incorporated into an app or API are approximations. The software can't observe the motion nor the photons directly, so instead we must use software-visible proxy events.

The app querying for the latest head pose is a proxy for user motion, but note that this discards any latency in the sensors, USB driver, sensor fusion processing, and time the data spent sitting around in a buffer waiting for the app to query it.
The GPU flipping its display buffers is a proxy for photons beginning to come out, but this discards any latency added in the scanout and display process.

So, ideally an app-measured latency figure should include some corrections to at least estimate the above factors. Or accept that you're not really measuring the full motion-to-photon latency, but just the app-query-to-flip latency or some such.


Ladder to DNA using Transformation Matrix In my last Graphic mid-term, the exam contained a Transformation Matrices problem, its statement was:

Question

  Rotate the following ladder to form a DNA-shaped ladder using a transformation matrix.

Notes

The ladder base must be kept as it is without any rotating.
The ladder lies in the  x-y plane.
The total rotating angle should equal 540 degrees.
The ladder step width = 2 * ladder step height (This image isn't drawn to scale).
Assume that the curves are straight lines.

First Idea

When I first read the problem statement, I thought of this basic trivial solution:

Translate the ladder such that the ladder's base is on the origin and the y-axis halves the ladder.
Rotate all points by the required angle which depends on the current point y value.
Translate the ladder back to it rotation.

Assume that $T(x,y)$ is the transformation matrix that will take care of translation to the origin and its inverse will return the ladder back to its original location, and $R_y(\theta)$ is the rotation matrix around y-axis to perform the rotation to achieve the DNA-shaped ladder.

If we calculate $\theta = 540/12 = 45$ degrees, then we need to rotate as following:

1st step by $0$ degrees (to maintain the fixed-base condition).
2nd step by $45$ degrees.
3rd step by $90 = (2*45)$ degrees.

The problem

I couldn't apply this solution and put it into valid transformation matrix because referring to my previous knowledge (and the logic behind transformation matrices too), I recalled that we cannot put variables (e.g. y) inside the cos, sin functions inside the rotation matrix. Then I thought of another solution if I used power to the rotation matrix by the value of y this will give me the same effect but I also couldn't put this idea into a transformation matrix.

So, to conclude the problem we have 2 ways for achieving this (which I think aren't valid):

$\begin{bmatrix}\cos(y\theta)&0&\sin(y\theta)&0\\0&1&0&0\\-\sin(y\theta)&0&\cos(y\theta)&0\\0&0&0&1\end{bmatrix}$

And the final answer would be: $P' =  (T(x,y))^{-1} R(45) T(x,y) P$

$\begin{bmatrix}\cos(\theta)&0&\sin(\theta)&0\\0&1&0&0\\-\sin(\theta)&0&\cos(\theta)&0\\0&0&0&1\end{bmatrix}$

And the final answer would be: $P' =  (T(x,y))^{-1} (R(45))^{y} T(x,y) P$

Doctor's solution

My doctor was a big supporter of the first solution and he said this is the correct transformation matrix to solve such problems.

The real question

Do any of these solutions achieve the required result while maintaining the transformation matrix concepts (i.e. avoid equations and direct substitutions and instead use matrices operations)?
If both solutions are wrong, can you kindly guide me to the valid solution?
If one of the solutions above is correct, can you please explain why?

You can't perform this transformation by applying constant transformation matrix to the ladder model since it's not linear transformation like joojaa said in the comments. What you would have to do instead is to define the transformation as a function of height as you have done in both of your proposals.

So both of your proposals are kind of correct, except for the typo in second one where you should use $R$ instead of $T$ that you raise to power of y. However, you might have difficulties raising a matrix to non-integer power (requires expansion of Taylor series, AFAIK), so your first proposal is much more practical.


Which 3D applications currently uses Vulkan API? I'm a newbie in terms of CG world. I'm curious to know about the applications that use Vulkan API and their performances compared to other APIs.

As I read, Vulkan has more access to the graphic cards which helps in performance. If so, do the current rendering engines use it?

Solidworks, Revit, Sketchup, Unity3D, Unreal Engine, After Effects, etc. ?

I'm a newbie in terms of CG world. I'm curious to know about the applications that use Vulkan API and their performances compared to other APIs

There are some popular magazine websites, such as PCGamer and AnandTech that has some comparison charts between other rendering APIs.

PCGamer compares OpenGL vs Vulkan for Doom (2016)

AnandTech compares Vulkan vs DirectX12 API Overhead

I would argue that comparisons make sense on the application-level - one API wouldn't necessarily be better on all occasions than the others. It totally depends on how the graphics developers of the software have architected their system to unlock the APIs full potential.

  As I read, Vulkan has more access to the graphic cards which helps in performance. If so, do the current rendering engines use it?
  <
  Solidworks, Revit, Sketchup, Unity3D, Unreal Engine, After Effects, etc. ?

Wikipedia: Software that supports Vulkan page has a list of software that supports Vulkan API. As for your question, quoting from Wiki page:

  Unreal Engine 4 – In February 2016, Epic Games announced Unreal Engine 4 support for Vulkan at Samsung's Galaxy S7 Unpacked
  event.[64][65]
  Unity – The engine has support for Vulkan since version 5.6.

There's plenty of others which are also popular, video games and game engines listed in the Wikipedia page, such as:

Dota 2
Doom
vkQuake
Star Citizen
Wolfenstein II
CryEngine
Source 2


benefit of perlin noise over value noise While investigating the inner workings of perlin noise, I wondered why one would use perlin noise instead of simple value noise. As far as I understand it right, the following applies:

Perlin noise is a lattice based noise function, which assigns a n-dimensional gradient (random for the original implementation, fixed for the improved one) for every point in the underlying noise space. Now you can query a value for every point in space by calculating the dot product between the distance vector and the gradient vector. After that you average all values calculated and get the queried value.

But isn't value noise kind of the same without using gradient vectors but random values? Since I also interpolate between values in value noise I cannot see any benefits by using an additional calculation step (the dot product) in perlin noise.

So why would I use perlin noise instead of value noise? Why is perlin noise so popular?

The benefit of perlin noise is the overall distribution of frequencies. Since value noise uses simple values that are interpolated, there is a higher chance, that a row of several values only differs a little. The consequence is, that some regions of your picture may contain little changes and some regions a lot of changes.

By using gradients you are reducing this effect because the interpolation is not done by value but instead calculated between tangents. Now it's more difficult to have a flat curve (both tangents must be collinear).

Source: As noted by Martin Ender the question was already posted on a different StackExchange community: see this Math.SE post.


How to decide which way to triangulate a quad? Apologies if this is a dumb question... but given four arbitrary (non-coplanar) points in 3D space, there are obviously two different ways to triangulate the quad that they form. One triangulation may produce much more natural results than the other- but how can I determine which one to pick?
Context: I’m doing Dual Contouring, and the quad forms part of an isosurface. I have access to the hermite/gradient data at that point.

A simple heuristic that many 3D content creation apps use is to split along the shorter of the two diagonals of the quad. This generally seems to work pretty well. It minimizes the appearance of long, skinny triangles, which are generally undesirable both for visual and performance reasons.
See this article: Deformation and Triangulation in Maya for some further discussion. In particular if you have a dynamically deforming mesh, you ideally want to retriangulate the quads based on the deformed geometry each frame, or you can get some pretty wonky artifacts if you stick with a fixed triangulation:

(image from the above linked blog post)


Frustum - problem with z dimension I'm beginner in OpenGL and I stumbled into a problem. I'm in OpenGL 2.1 for the time being, if that's of any help.

I set my Frustum as such:

glFrustum(-1366, 1366, -768, 768, 1, 20000);

Now, when I create a cube from quads like this:

0.0, 500.0, 0.0
500.0, 500.0, 0.0
500.0, 0.0, 0.0
0.0, 0.0, 0.0
0.0, 500.0, 500.0
500.0, 500.0, 500.0
500.0, 0.0, 500.0
etc.

It is not a cube, as I expected. It's extremely stretched along the z axis.

Here's a screenshot of how it looks now: https://www.dropbox.com/s/diznfumtev7g259/Screenshot%20from%202016-02-26%2017%3A01%3A58.png?dl=0 (I removed one of the faces temporarily)

The shape starts to resemble a cube a bit when I change the vertices to:

0.0, 500.0, 0.0
500.0, 500.0, 0.0
500.0, 0.0, 0.0
0.0, 0.0, 0.0
0.0, 500.0, 0.3
500.0, 500.0, 0.3
500.0, 0.0, 0.3
etc.

Code related to this (I am translating this from assembly which I am actually using).

Matrices setup on initialization:

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glFrustum(-1366.0, 1366.0, -768.0, 768.0, 1.0, 20000.0);

glMatrixMode(GL_MODELVIEW);
glLoadIdentity();

Transformations of the cube inside the "render()" function:

glLoadIdentity();
glTranslated(1000.0, 500.0, 0.0);
glRotated(degrees, 0.0, 0.0, 1.0); // degrees is +=2 every frame
glTranslated(-250.0, -250.0, distance); // distance is -=0.001 every frame

There are no other matrix operations in the code.

The code uses GLUT to initialize a window with double buffer and depth buffer. Depth testing is enabled.

It all seems to work as expected, the rotation is ok, movement also. I am only troubled by the fact that when I define a 500.0x500.0x500.0 cube, the actual output is not a cube.

What am I doing wrong? How to fix this?

Okay. So after 6 hours of tinkering with my code, trying different approaches and googling for solutions, I found this page:

http://www.glprogramming.com/red/chapter03.html

There is a short C listing which put me on the right track to solve this.

What I did wrong was that I should have set the frustum to something like this:

GLdouble x0 = -(1366/768);
GLdouble x1 = 1366/768;
glFrustum(x0, x1, -1, 1, 1, 100);
// the last two values aren't critical, as long as they are positive

But before that, I should have set the glViewport properly, before setting up the frustum:

glViewport(0, 0, 1366, 768);

I didn't do it and that's why my code produced such weird results.

Now I can create a unit cube with following vertices:

0.0, 1.0, 0.0
1.0, 1.0, 0.0
1.0, 0.0, 0.0
0.0, 0.0, 0.0
0.0, 1.0, 1.0
1.0, 1.0, 1.0
1.0, 0.0, 1.0
0.0, 0.0, 1.0

And it will display properly as a cube of size 1 in every direction.


Tesselation sine distortion? I am experimenting on a simple wave tessellation shader. I ported it from a working vertex shader. When my frequency is higher than 1, I get strange "saw-tooth" artifacting at peaks of the wave. Here are some images with a frequency of 3.

If I keep the frequency lower, it is smooth as butter. Any ideas what the problem could be?

#version 410 core
layout(quads, equal_spacing, ccw) in;
in vec2 teUv[];
out vec2 fUv;
out vec4 fWorld_pos;
uniform mat4 VP;
uniform mat4 M;
uniform sampler2D water_disp;
uniform float time;
const float M_PI = 3.1415926535;

float water_displacement(vec2 pos) {
    const float freq = 3.0; // <-- This is the problematic value.
    float d = length(pos);
    return (0.5 + 0.5 * sin(-M_PI * d * freq + time));
}

void main() {
    vec4 p1 = mix(gl_in[0].gl_Position,
            gl_in[1].gl_Position,
            gl_TessCoord.x);
    vec4 p2 = mix(gl_in[2].gl_Position,
            gl_in[3].gl_Position,
            gl_TessCoord.x);
    vec4 local_pos = mix(p1, p2, gl_TessCoord.y);
    fWorld_pos = M * local_pos;
    fWorld_pos.y += water_displacement(fWorld_pos.xz);

    gl_Position = VP * fWorld_pos;

    vec2 uv1 = mix(teUv[0], teUv[1], gl_TessCoord.x);
    vec2 uv2 = mix(teUv[2], teUv[3], gl_TessCoord.x);
    fUv = mix(uv2, uv1, gl_TessCoord.y);
}

And more examples:

Not enough triangles. The situation is analogous to the sampling theorem that states that you can not reconstruct a signal if your sample frequency is below a certain threshold. Although in this, case you're not attempting to rebuild the signal with a higher order filter so the result is even more dramatic. And it results in interpolation errors.

You can alleviate the problem with less points if your point wavefront follows your waves. So you may want to shift points towards the gradient maximum in other directions than UP.


What's the concept of "Ray" really? I'm learning Ray Tracing using this MIT tutorial.

I've heard Ray Tracing only in the context of lighting. I.e. I understand that ray casting is used to light an object that has been already drawn. However, I've also seen ray casting in the context of just drawing the picture (not lighting). So which one are rays about?

What's the concept of "ray" really?

E.g. in that tutorial one draws the picture like:

image.SetPixel( x, y, pixelColor );

where pixelColor has been acquired from the Ray Caster. But I read this to mean that pixelColor already has lighting information.

A ray is just a semi-infinite line, and casting the ray finds intersections between the ray and an object in the scene. It doesn't just tell you the position in space that the ray intersected the object: it also tells you the texture co-ordinates of that point on the object, the surface normal and tangent, and (depending on the algorithm you use) filter widths.

So in a ray-tracer, there are different kinds of rays. Primary rays and reflection and refraction rays use their intersection to run a shader and compute a fragment colour. They also fire shadow rays at lights, which don't compute a colour but just tell you whether any object is shadowing the light.

So rays are about finding the objects which are in the scene, and shading those objects to make an image. They're also about finding the reflections and refractions on shiny objects. They are used to draw shadows. They're also used for more advanced effects like ambient occlusion and photon tracing.

This is one of the main conceptual advantages of ray-tracing over scanline rasterisation: instead of having to use different techniques for computing reflections, shadows, and all those other effects, the same core ray-cast algorithm can be used recursively to compute the whole image.


Why negate z when constructing projection matrix OpenGL I constructed an orthographic projection matrix from  this article on scratchpixel.com:
Matrix4 ortho(float l, float r, float b, float t, float n, float f) {
   return Matrix4(
       2/(r-l), 0, 0, -(r+l)/(r-l), //row 0
       0, 2/(t-b), 0, -(t+b)/(t-b), //row 1
       0, 0, -2/(f-n), -(f+n)/(f-n), //row 2
       0, 0, 0, 1 //row 3
   );
}

, where r, l, t, b, n, f are right, left, top, bottom, near and far corners of the view box.
Now I tested out this matrix using the default parameters of the view box ortho(-1, 1, -1, 1, -1, 1) to see if I get an identity matrix. However I'm getting:
 1, 0, 0, 0,
 0, 1, 0, 0,
 0, 0, -1, 0,
 0, 0, 0, 1

Basically this will result in z coordinate of every transformed point to be negated.
Is it standard to negate the z coordinate when constructing projection matrix? What could be the purpose behind that?

I'm assuming your scene is constructed based on right-handed coordinates.
If you are using OpenGL, yes. If you are using Direct3D, no.
The projection matrix maps [-n, -f] into [0, 1]. This weird property comes from the fact that the eye coordinates are right-handed, but the clip space (NDC) uses left-handed coordinates. Hence the implementation of ortho().
Direct3D uses a z-axis that goes outwards the screen. Implementations for Direct3D projection matrices will not produce negated z when coupled with your application.


Why don't Direct3D/Metal support texture swizzles? To be clear, I'm talking about behavior like https://www.khronos.org/opengl/wiki/Texture#Swizzle_mask, not "normal" myColorVariable.rbgr swizzling in a shader.

I really did start missing this feature after switching from OpenGL to Direct3D11, and ended up implementing a poor man's version in HLSL. Initially, I just assumed this was another "feature" of OpenGL that didn't actually exist on any hardware and was just being emulated by every driver, but then I noticed that Vulkan supports it as well.

My understanding is that the Vulkan specification is not supposed to include "extras" that aren't supported by hardware, so what gives? Why is this feature present in OpenGL/Vulkan and not Direct3D/Metal?

Why is this feature present in OpenGL/Vulkan and not Direct3D/Metal?

It's present in OpenGL because someone thought it would be a good idea, wrote an OpenGL extension specification for it, released and supported the extension in their drivers, and the extension was eventually adopted into core profile. While it's usually pretty obvious why feature X is present in some API (because it's a thing that makes sense to have), it's generally hard to say why feature X is not present in some other API. There's a wide range of possible reasons. You would have to talk to whoever were the people responsible for the design of the API in question at the time to find out…

Concerning hardware support: In case of doubt, you can simply convince yourself by comparing rendering times with and without using a texture swizzle. You would notice if you were hitting a software path there. The fact that the extension was introduced by NVIDIA would lead me to the conclusion that there's at least one hardware vendor that is able to support this on their hardware; otherwise, why would they introduce it!? If you think about it: Texture sampling hardware already has to support lots of different formats (floating point, signed integer, unsigned integer, normalized signed integer, normalized unsigned integer, all for different numbers of components, component sizes, sRGB versions for some, compressed formats, …) including some where the order of components is different on the output side from what it is on the input side (e.g., BGRA). Adding a few bits to the texture descriptor to control which components go where in the output is probably not a big deal. I'm not a hardware engineer, but it would seem to me that it probably even makes the hardware simpler because it reduces the number of special cases. So I'd say chances are that hardware was already using a swizzle mask internally long before it was exposed on the API side…

Apart from that, while I don't know about Metal, Direct3D 12 does allow you to specify a shader component mapping when creating a resource view, which is effectively exactly the same thing as the OpenGL texture swizzle mask…


How did the hard, accurate shadows, work in early 3d (2000 - 2004) games? I've been hunting around the internet for a while now, and I can't find anything about the specific shadow technique that I'm looking for.

I can only recall one game in particular that used it, and I know that it happens on the PC port of the game too. The game in question is "Sonic Adventure 2: Battle" for GameCube. I remember it happening on "Super Mario Sunshine", but I could be mistaken.

The shadows are only cast from the main player character.
It is 100% sharp, no matter the resolution the game is rendered at.
They are cast perfectly parallel down onto the surface the player stands on.
Interestingly, if the shadow intersects with a vertical wall, the shadow is drawn all the way up the wall, even higher than the original intersection.
You can also see the shadow on ceilings directly above the player character, and even on floors above ceilings. It's as if there's a big character-silhouette shaped tube being cast onto surfaces.

How does this technique work, and how can I learn more about it?

I'm going to hunt for a picture of this particular technique, but I don't have one right now.

Having not worked on these games, I can only speculate... but from your description, it sounds like a crude shadow map with orthogonal projection and no depth information. It probably has (or had) a specific name but the way I suspect they did it is:

Render the character from above to a binary texture (eg. white there there's nothing, black where the character is), using an orthographic camera.
When rendering the scene, perform a lookup in that texture and multiply by that color. You would of course exclude your characters from that.

The calculations involved for the texture lookup would be trivial as everything is 2D, in a plane of the world coordinates. It also does not need programmable shaders which IIRC were not around yet at that time. What you would need as texture coordinates is simply the (x,z) location of the vertex (assuming y is toward the sky here), along with an offset equal to the (x,z) location of your character. In OpenGL, I believe that could be achieved by glTexGen along with a texture transformation matrix.

The reason for the artifacts is that this really does make a "character-silhouette shaped tube", as you put it. Looking from above, the shadow is painted on every surface regardless of it being in front of or behind the character (above or below, from the camera's view). It's like you're dropping black ink from the sky inside the outline of the character and it goes through everything.


How does Texture Cache work considering multiple shader units Modern GPUs have many parallel shading units. I'd like to know how texture cache is managed in that scenario.

At the top level, a GPU is subdivided into a number of shader cores.  A small GPU in a notebook or tablet may have only a few cores while a high-end desktop GPU may have dozens.

In addition to the shader cores there are also texture units. They may be grouped together with one texture unit per shader core, or one texture unit shared among two or three shader cores, depending on the GPU.

The whole chip shares a single L2 cache, but the different units will have individual L1 caches.  Texture units have texture caches, and shader units have caches for instructions and constants/uniforms, and maybe a separate cache for buffer data depending on whether buffer loads are a separate path from texture loads or not (varies by GPU architecture).

Texture units operate independently and asynchronously from shader cores. When a shader performs a texture read, it sends a request to the texture unit across a little bus between them; the shader can then continue executing if possible, or it may get suspended and allow other shader threads to run while it waits for the texture read to finish.

The texture unit batches up a bunch of requests and performs the addressing math on them—selecting mip levels and anisotropy, converting UVs to texel coordinates, applying clamp/wrap modes, etc. Once it knows which texels it needs, it reads them through the cache hierarchy, the same way that memory reads work on a CPU (look in L1 first, if not there then L2, then DRAM). If many pending texture requests all want the same or nearby texels (as they often do), then you get a lot of efficiency here, as you can satisfy many pending requests with only a few memory transactions.  All these operations are pipelined, so while the texture unit is waiting for memory on one batch it can be doing the addressing math for another batch of requests, and so on.

Once the data comes back, the texture unit will decode compressed formats, do sRGB conversion and filtering as necessary, then return the results back to the shader core.


Solving a problem from *Foundations of Computer Graphics*: From Gortler's Foundations of Computer Graphics (i.e., self-study):

  Let $\mathbf{\vec{e}}^t = \mathbf{\vec{w}}^t E$ and let $P$ be a camera matrix that is just some arbitrary $4$ by $4$ matrix with "dashes" in its 3rd row. Given the world coordinates of 6 points in space as well as their normalized device coordinates, how would one compute the 12 unknown entries in the matrix $PE^{-1}$? (Note: this can only be solved up to an unknown scale factor) (Hint: set up an appropriate homogeneous linear system with a right hand side of zero).

Here the "dashes" matrix the author refers to takes form

$$
\begin{bmatrix}
n_1 & 0 & a & 0 \\
0 & n_2 & b & 0 \\
- & - & - & - \\
0 & 0 & -1 & 0 \\
\end{bmatrix}
$$

where we do not care what the values with dashes are. How does one use the hint to solve this problem? I think we can infer that $a$ and $b$ (the scale factors) are undetermined, but that we can infer what the $n_1$ and $n_2$ are.

Let me try to make my comment into a complete answer.

The general idea is to build a linear system using those 6 point pairs and solve for the desired 12 unknowns. You may find this paper[1] useful if you want to learn the theoretical background, and you can find some more details in my blog post[2].

Here is my solution using Mathematica.

In the first step I will build a symbolic array which contains all variables.

v = Table[Symbol["m" <> ToString[i]], {i, 0, 15}];
M = ArrayReshape[v, {4, 4}]

Next we write down the coordinates for all vertices.

k = f/ n;

X = {
    {l, t, -n, 1},
    {l, b, -n, 1},
    {r, b, -n, 1},
    {r, t, -n, 1},
    {k l, k t, -f, 1},
    {k l, k b, -f, 1},
    {k r, k b, -f, 1},
    {k r, k t, -f, 1}};

U = {
    {-1, 1, -1},
    {-1, -1, -1},
    {1, -1, -1},
    {1, 1, -1},
    {-1, 1, 1},
    {-1, -1, 1},
    {1, -1, 1},
    {1, 1, 1}};

where l,r,b,t,n,and f denote left, right, bottom, top, near, and far planes respectively,
and 'X' is the coordinates in world space, U is the coordinates in NDC.

Then next we implement a function Generate to generate an equation for each pair of coordinates:

Generate[{i_, j_}] := M[[j]].X[[i]] - U[[i, j]] M[[4]].X[[i]] == 0;

Because we do not care about the third row,
we only solve for the unknowns in partialv:

partialv = {m0, m1, m2, m3, m4, m5, m6, m7, m12, m13, m14, m15};

Now we can build our linear system and solve it:

solve[subset_] := Solve[
    Map[Generate, Table[{i, j}, {i, subset}, {j, 1, 2}], {2}] // Flatten,
    partialv] // Simplify

Let me explain it a little bit.

Because only 6 pairs of points are given, we select a subset of 6 elements from the 8 pairs of points.
Then we call Generate function on this subset
and the first two rows of matrix(we do not care about the third row. And the fourth row is the "homogeneous" part, which is also ignored).

Finally we enumerate all subsets from these 8 pairs of points, and try to find a solution:

solutions = Map[solve, Subsets[{1, 2, 3, 4, 5, 6, 7, 8}, {6}]];

We can view one solution by:

MatrixForm[ArrayReshape[v /. solutions[[1]], {4, 4}]]

$$
\left(
\begin{array}{cccc}
 \text{m0} & 0 & \frac{\text{m0} (l+r)}{2 n} & 0 \\
 0 & \frac{\text{m0} (l-r)}{b-t} & \frac{\text{m0} (l-r) (b+t)}{2 n (b-t)} & 0 \\
 \text{m8} & \text{m9} & \text{m10} & \text{m11} \\
 0 & 0 & \frac{\text{m0} (l-r)}{2 n} & 0 \\
\end{array}
\right)
$$

The m0 is the "unknown scalar factor" which is mentioned in problem statement.

In fact if we let m0 equals 2 n/(r - l),
we would get the conventional projection matrix used in OpenGL:

% /. m0 -> 2 n/(r - l)
% // Simplify // MatrixForm

$$
\left(
\begin{array}{cccc}
 \frac{2 n}{r-l} & 0 & \frac{l+r}{r-l} & 0 \\
 0 & -\frac{2 n}{b-t} & -\frac{b+t}{b-t} & 0 \\
 \text{m8} & \text{m9} & \text{m10} & \text{m11} \\
 0 & 0 & -1 & 0 \\
\end{array}
\right)
$$

[1] Heckbert, Paul S. Fundamentals of texture mapping and image warping. MS thesis. University of California, Berkeley, 1989.

[2] Inferring projective mappings Inferring projective mappings | linearconstraints.net


What is name for far lowcontrast blueish landspace? What is the name of effect, that we see on far objects in outdoors ? These objects are low-contrast, foggy ? Is it only fog or some scattering ?

In computer graphics the effect is called "atmospheric scattering", that's sun light scattering from atoms and other particles in the atmosphere. In modern computer games atmospheric scattering is commonly modelled with Bruneton sky model by Eric Bruneton and is most prevalent in the clear sky due to long view distances, but also visible for distant objects on ground as you have observed.

Atmospheric scattering in Bruneton model is composed of Mie scattering and Rayleigh scattering. Mie scattering is mostly wavelength independent strongly anisotropic scattering from particles that are quite a bit larger than the wavelength of light (water droplets & pollution in the atmosphere). Mie scattering results in effects like halo around the sun.

Rayleigh scattering on the other hand is wavelength dependent scattering from atoms in the atmosphere whose size is close to the visible light wavelength range. Rayleigh scattering is responsible of effects such as the blueish sky at noon and blueish tint of objects in the distance.


Is there a way to script image creation? I am facing a though challange - I need to perform repetitive Image formating. Firstly, I need resize picture A so that it has some fixed width in pixels. Then, I need to place picture A in the middle of picture B. I need to perform this task repetitively for hundreds of images. Is there any way to script this, so that I can just type in for example:

perform pictureA.png pictureB.png output.png

Any help is more than welcome!

ImageMagick is a set of command-line tools that can do the sort of things you describe. For example, this command line will overlay picture B with a centered copy of picture A, resized to 100 pixels wide (keeping aspect ratio):

convert pictureB.png ( pictureA.png -resize 100 ) -gravity center -composite output.png

See the ImageMagick docs for more info about its command lines are structured...it's a powerful piece of software, but it takes a bit of work to get the hang of how to use it.


perspective projection transformation matrix In one text,  the  derivation for  perspective  projection  goes like  this:

If $x',y',z'$ represent any point along  the  projection line,and $x_{prp},y_{prp},z_{prp}$ are the projection reference point, then:

$$x'=x-(x-x_{prp})u$$
 $$y'=y-(y-y_{prp})u$$
$$z'=z-(z-z_{prp})u$$
where $0\le u \le 1$

Now,   finding the value of u from the third equation above and  replacing in the first and  second  equations,   we get at  $z_vp$:

$$ x_p=x'= x.\frac{z_{vp}-z_{prp}}{z-z_prp}+x_{prp}.\frac{z-z_{vp}}{z-z_{prp}}$$
$$ y_p=y'= y.\frac{z_{vp}-z_{prp}}{z-z_prp}+y_{prp}.\frac{z-z_{vp}}{z-z_{prp}}$$

and there is no projection transformation matrix  given.

In the second text,  the  derivation is using  similar triangles:

comparing the  similar triangles  we  get:
$$x_p=\frac{-x_c}{\frac{z_c}{d}}$$
$$y_p=\frac{y_c}{\frac{z_c}{d}}$$

and  the projection transformation for  homogeneous coordinates x,y,z is  given by:
$$
\begin{bmatrix}
-1&&0&&0&&0\\
0&&1&&0&&0\\
0&&0&&1&&0\\
0&&0&&\frac{1}{d}&&0
\end{bmatrix}
$$

so,  my questions are:

How do I find the projection matrix of coordinates as found from
derivation 1?
Both are given in the perspective projection area of
these different texts,  are both transformations the  same? If not,
how  are  they different?

Given your equations (which i just assume are correct):
$$
\begin{align}
x_p &= x \frac{z_{vp}-z_{prp}}{z-z_{prp}}+x_{prp} \frac{z-z_{vp}}{z-z_{prp}}\\
y_p &= y \frac{z_{vp}-z_{prp}}{z-z_{prp}}+y_{prp} \frac{z-z_{vp}}{z-z_{prp}}
\end{align},
$$

if we take $(x_{prp}, y_{prp}, z_{prp}) = (0, 0, 0)$, then we have

$$
\begin{align}
x_p &= x\frac{z_{vp}}{z}\\
y_p &= y\frac{z_{vp}}{z}
\end{align}
$$

Assuming the projection plane is at distance $d$ from the origin, and, by your drawing, that the projection plane is on the negative side of the Z axis, we have $z_{vp} = -d$, which leads to

$$
\begin{align}
x_p &= - x\frac{d}{z}\\
y_p &= - y\frac{d}{z}
\end{align}
$$

Now we can follow the same reasoning (homogeneous coordinates) as in my answer to your other question (using column notation) to write

$$
\begin{align}
\begin{bmatrix}
x\\y\\z\\ \frac{-z}{d}
\end{bmatrix}
=
\begin{bmatrix}
1&0&0&0\\
0&1&0&0\\
0&0&1&0\\
0&0&\frac{-1}{d}&0
\end{bmatrix}
\begin{bmatrix}
x\\y\\z\\1
\end{bmatrix}
\end{align}
$$

which after the division by the last coordinate will give the projected point. Notice that because we are writing it in column notation, the perspective matrix is the transpose of the matrix in your other question. Also, the negative sign is because the camera reference frame is a right-handed one.

As for the second textbook that you are reading, I cannot understand why you would have a negative sign only in the x axis without knowing more about the camera reference frame. By the diagram you posted, there should not be a negative sign.


Data structure of IFC-files I know OBJ-files essentially are one or more lists of n-tuples containing information about the location of vertices, texture and normals. However, I am not sure how this is structured in an IFC-file.

However, whenever I look for the data structure of IFC-files, I find information regarding the hierarchy of elements in IFC, but I cannot find information regarding how the geometry is saved. Also, how is this related to the hierarchy of entities in the model?

What should I learn to understand this?

You need to learn and understand STEP data exchange format (ISO 10303-21) which is the fundamental exchange format which the IFC schema is built on top of. This is no easy feat though, especially when comparing to a file format such as OBJ. OBJ files are designed to define geometry (and other information via associated MTL) and so are optimised for this purpose. IFC on the other hand, is designed to exchange many aspects of AEC including building, services, structural, furnishings, energy model etc and so is far more complex with the geometry definitions being only a subset of the information supplied in an IFC. It would be easier to use an existing IFC library to do this, and even then, an understanding of the structure of the IFC entities and their hierarchy is still required to traverse the relationships (inverse attributes) between the object definitions in an IFC file in order to reconstruct the desired geometry.

OBJ files explicitly define the vertices, normals and faces that represent a mesh, however while there are IFC entities which can represent meshes in a similar fashion (namely IfcConnectedFaceSet), most geometry defined in an IFC file should be considered more like a series of instructions/commands to recreate the geometry. For this reason you would also require a geometry library (one which can do things like extrusions, Boolean/CSG operations, curves, NURBS etc). For example, one of the main things that is represented in many IFC files would be the walls, doors, windows etc of a building, and while these could be represented as explicit tessellated data or breps, in my experience these tend to be represented as an IfcExtrudedAreaSolid (https://standards.buildingsmart.org/IFC/RELEASE/IFC2x3/TC1/HTML/ifcgeometricmodelresource/lexical/ifcextrudedareasolid.htm) which supplies the direction and length to extrude a path/outline curve (the curve being defined by an IfcProfileDef which is an attribute of IfcExtrudedAreaSolid, https://standards.buildingsmart.org/IFC/RELEASE/IFC2x3/TC1/HTML/ifcprofileresource/lexical/ifcprofiledef.htm). This gets even more complicated when there is an opening (i.e window recess or door opening in the wall) which again is related (through an inverse attribute) to the geometry representing the opening to be removed through a geometric Boolean operation (CSG). As you can see it starts to get complicated fairly quickly and isn't just a case of reading in raw vertices, normals and faces etc.

All geometry in an IFC file is defined by objects which are sub types of IfcRepresentationItem (http://dcom.arch.gatech.edu/aci/cic/schema/ifcgeometryresource/lexical/ifcrepresentationitem.htm) and so that would be the place to start to understand what types of geometry definitions are used by IFC. Other items within the model (namely furnishings) are far more likely to be represented by raw tessellated data (IfcFacetedBrep, https://standards.buildingsmart.org/IFC/RELEASE/IFC4_1/FINAL/HTML/schema/ifcgeometricmodelresource/lexical/ifcfacetedbrep.htm) which is geometry that would be more familiar to you, and there is a chance you can isolate these definitions within in an IFC file and concentrate on processing that (you would still need to understand STEP format, namely the inverse attributes involved, in order to faithfully generate the mesh geometry). Also this geometry could be either explicitly located (in world space), but in many cases is also positioned through IfcPlacements (https://standards.buildingsmart.org/IFC/RELEASE/IFC2x3/TC1/HTML/ifcgeometryresource/lexical/ifcaxis2placement3d.htm) locally, so you need to build up the hierarchy of transformations based on these definitions.

In short, if you wanted to RYO IFC parser to grab geometry, it would be a steep learning curve and require you to understand many other concepts of STEP/IFC outside just knowing the geometry types and transforms. Easier to use an IFC library coupled with a geometry library (although many IFC libs do come with geometry libs to do this for you), but even then, depending on the lib used, you may need to have a basic understanding of the building hierarchy involved in order to correctly place and reconstruct much of the geometry.

Also note, while many of the geometry concepts remain the same through the various versions of IFC (IFc2x3, IFC4, IFC4 ADD1 etc), in some cases there are subtle differences in the schema in relation to geometry object definitions. However, much of the version difference in IFC is do to with the building hierarchy, and associated information for the various disciplines in AEC.


How Image Based Lighting is applied to indoor scene? Now the mainstream engine's approach is to use image-based lighting to achieve approximate global illumination, but such a technique seems to be only applicable to outdoor scenes.What kind of technology is used in the indoor scene to achieve global illumination?For example, in a room with natural light, or in a completely enclosed indoor environment

There's two main things it seems like you've not understood correctly.

IBL is for indoor scenes and outdoor scenes

In the early days when IBL was hot, the Uffizi light-probe was probably the most popular for researchers. IBL works really well for indoor scenes with strong light coming from multiple directions, such as churches with stained glass. That kind of scene was the "killer app", because for most outdoor scenes, skylight simulation was already a popular, fast, and useful method. Environment maps were originally popular for outdoor scenes because simulated clouds didn't look right, even though the lighting was good.

The big limitation for indoor scenes is if the room is small. A light probe taken in the middle of a room won't be accurate for the corners of that room, because the light is coming from different directions. Just like with a reflection map, if your objects are moving far enough for that to be visible, you need to take multiple light probes and blend between them. This can also be a limit for outdoor scenes if the lighting is dominated by (say) the shadow of a building or tree, or a nearby artificial light.

IBL is not a global illumination technique

It's more helpful to think of it as pre-baked lighting. Just as you might use a light map to bake GI computed from your static lights (with some offline technique) into the textures for your static objects, you can also have that pre-computed GI affect your dynamic objects. But it's not an approximation to full GI, because dynamic lights and objects don't contribute at all.

So how do I use it indoors?

There's two ways. If you're trying to place CG objects into a real-world indoor scene, just capture a light probe of that scene exactly the same way as a real-world outdoor scene. If you're trying to light a CG scene (i.e. a room that only exists in your game), then put your scene into an offline renderer (such as a path-tracer with full GI) to create the light probe, then apply that light probe in your real-time engine.


Why the ray tracing result looks weird? I followed the book Ray Tracing in One Weekend and implemented a simple ray tracing procedure. There are 100 random samples every pixel for antialiasing, but the render result looks weird: There is a unnatural circle at the center of the screen, and a strange line lies on the center of the ground, Why?

Those are moire patterns. They are  an aliasing artifact that usually occurs when sampling on a regular grid. Did you jitter the positions of your samples? If you just sampled an evenly spaced 10x10 grid within each pixel, that could explain it. Also, numerical errors or inaccuracy could it.


What is the PDF for path tracing in the paper "Learning the light transport the reinforced way It's the algorithm which combines path tracing and reinforcement learning. I can't understand what $p_\omega$ is.

The algorithm is clear. The actions are the directions and the states are the hit points. For updating the Q-values, the hit points are considered as parts of a discretized diagram like Voronoi diagram and the directions of rays are generated by a discretization of a hemisphere surface which divides it into equal areas. The Q-values are initialized as equal for each state.

First, we generate a camera ray, and when it hits a diffuse surface, it scatters according to a discrete PDF which is calculated by normalizing the Q-values of the state of hit point.

Second, The Q-values are updated along the path, we have the state $s$ and the next state $s\prime$ is generated from the direction generated by PDF of Q-values in $s$.

The area of each path on the hemispheres is $2\pi/n$, and integral in the update equation of Q-values is calculated like this:

$\dfrac{2\pi}{n}\sum_{k=0}^{n-1}Q_k(y)f_s(\omega_k, y, -\omega)cos\theta_k$

I understand all of the algorithm, but I don't know what $p_\omega$ is in the second part. Is it $1/2\pi$ or $n/2\pi$ ? Could someone prove what it really is? I need the proof.

I guess it is $1/2\pi$ because the path tracer here only use the PDF generated by Q-values. It doesn't use the Q-values at all. Am I correct? If I am correct, how can I prove it?

EDIT: For clarification, a ray generated through a patch uniformly and the Q-values are generated as I said. Because we only change the directions so more samples are generated inside the patches which receive more light.

I used $n/2\pi$ , but scene was so dark, so the PDF must be either $1/2\pi$ or a normalized combination of PDF's of patches. If the second is true, what it really is?

EDIT 2: Should I ask this question in the statistics community of stack exchange?

I have to answer my question.
Consider that all patches have the same value. Then choosing a patch uniformly and selecting a random point uniformly on that patch is as the same as sampling the hemisphere uniformly. If $p_s=Pr(Q_k \in Q)=1/n$, which n is the number of patches, I can conclude that $p_\omega=p_s*(n/2\pi)=1/2\pi$ .

And if patches don't have the same value, then $p_s=Pr(Q_k \in Q)=q_k/(q_1+...+q_n)$ and
$p_\omega=p_s*(n/2\pi)=(q_k/(q_1+...+q_n))*(n/2\pi)$


Difference between rendering in OpenGL and 3D animation software With OpenGL and such I can render some pretty amazing looking things in "real time" 60 FPS.  However, if I try to make a video of that same scene in let's say Maya, or 3ds Max it would take MUCH MUCH longer for it to render even though it is the same resolution and FPS.

Why do these two types of rendering take different periods of time for the same result?

Note: Yes I do realize that 3D animation software can produce highly superior images to what could be done real time. But for this question I am referring to a scene of equal complexity.

The main difference would be that with OpenGL in let's say a video game you will have a process called rasterization which basically takes care of determining what part of the scene you see.

It needs to be fast so we can experience it as realtime.

Therefore the algorithm does a few simple steps.

check if a certain part of the scene is in my view frustum

check if something is in front of it which may needs to be rendered later using a depth buffer

order the objects we found to draw
draw them by projecting them on the screen
shade them based on textures / shaders / lights / ...

On the other hand a rendering software (Blender/Max/Maya/...) most likely uses some kind of raytracing

This involves alot more math to achieve a higher degree of realism.
It basically works in the same way:

create a camera and an image plane infront of it
shoot one ray (or multiple sample rays) through each pixel
check if the ray hits anything in the scene
closest hit is the one to be drawn in the pixel finally (like depth buffer)
calculate the light for the given point

....

I'll stopped listing here since this is the point where raytracing takes off.

Instead of only checking if a point is hit, most raytracer now begins to calculate:

the amount of light a surface penetrates
how much light gets reflected
cast new rays from the hitpoint into the scene until it may hit a light source

There are a ton of techniques with different degrees of realism which can be used to calculate the light of a certain point in the scene.

TL;DR
The gist would be that a raytracer mostly tries to be physically accurate when it comes the lighting and therefore has a lot of more calculations to do per pixel (sometimes shoot thousands of rays) and on the other hand games get their speed by drawing bigger chunks of the screen with simpler light calculations and a lot of shader tricks which let it look realistic.


How do I derive this transformation If I had two right handed frames where one has vectors [i, j, k] and another with corresponding vectors [u v w], how would I derive the transform M[i j k] <-- [u v w] when applied to any point P in [u v w] we get its representation in [i j k] ?

They noted that for every unit we move in the [i j k] frame we move 2 units in the [u v w] frame and the origin of the [u v w] frame can be represented as [4i 6j 2k]

Can someone help me derive this? I'm fairly new to computer graphics and need some help in detail.

I'm not going to dive into much details about affine transformations and such, you are better off reading up on these concepts from good graphics books.

I think The matrix you need is this.

$\begin{bmatrix}
0.5 & 0 & 0 & 4\\
0 & 0.5 & 0 & 6\\
0 & 0 & 0.5 & 2\\
0 & 0 & 0 & 1\\
\end{bmatrix}$

The reason why we need 4x4 matrices is because of affine transformations. For more details on why we have 4x4 matrices and why are we using 4D representation you can check my answer here. Briefly speaking, Vectors representing direction have an extra 0 at the end since translations don't affect these sort of vectors and Vectors representing points have an extra 1 at the end. This makes our vectors 4D so we can apply 4x4 transformation matrices. Since here we are concerned with points we will append an extra 1 to each vectors.

Let's begin by how did I derive it. First of all we can see that they provided the origin of the $UVW$ frame in terms of $IJK$ frame. This is $[4i, 6j, 2k]$. In other words the vector $[0,0,0]$ in $UVW$ frame must correspond to $[4i, 6j, 2k]$ in $IJK$ frame. Let's forget the 0.5's in the diagonal for now. We have,

$\begin{bmatrix}
0 & 0 & 0 & 4\\
0 & 0 & 0 & 6\\
0 & 0 & 0 & 2\\
0 & 0 & 0 & 1\\
\end{bmatrix}
*
\begin{bmatrix}
0\\
0\\
0\\
1\\
\end{bmatrix}
=
\begin{bmatrix}
4\\
6\\
2\\
1\\
\end{bmatrix} $

Next the condition is if we move 1 unit in $IJK$ we move 2 units in $UVW$ or in other words if we move 1 unit in $UVW$ we move 0.5 units in $IJK$. This means if we move from origin to $[1,1,1]$ in $UVW$ we moved half a unit in the $IJK$ frame in each of the axis. Testing the original matrix now we have,

$\begin{bmatrix}
0.5 & 0 & 0 & 4\\
0 & 0.5 & 0 & 6\\
0 & 0 & 0.5 & 2\\
0 & 0 & 0 & 1\\
\end{bmatrix}
*
\begin{bmatrix}
1\\
1\\
1\\
1\\
\end{bmatrix}
=
\begin{bmatrix}
4.5\\
6.5\\
2.5\\
1\\
\end{bmatrix} $

When constructing matrices this way, you want the translation factor in the last column. This factor is given to you by telling you the origin of the $UVW$ frame in terms of $IJK$ frame. This means the $UVW$ frame is that much displaced/translated when viewed from $IJK$ frame.

Secondly any scaling factor goes along the diagonal like above. This factor was given to you by telling you "If you move $X$ units in 1 frame, you move $Y$ units in the other".
Which is what we want I guess. Is that what you need?


How does the view-independence of radiosity increase the calculations involved? In the style of trichoplax' question, I want to talk about yet another Wikipedia article: Radiosity (computer graphics). The article states:

  Radiosity is viewpoint independent, which increases the calculations involved, but makes them useful for all viewpoints.

The technique works with diffuse surfaces only. This way, form factors can be precalculated and are independent of the viewer. The lighting needs to be updated only if some light source changed. If, on the other hand, the technique would support specular reflection, the form factors would depend on the viewer. Form factors and lighting would need to be updated constantly when the camera moves.

How does the limitation to diffuse surfaces increase calculations? Diffuse surfaces need to take light from all directions into account, which is more complex than taking light only from a smaller specular lobe. Is this what this sentence means? Is it just me or should this be rephrased?

While it may not entirely clear from the formulation of the wikipedia article, the author raises an important issue: In contrast to many other approaches, Radiosity needs to perform its calculations for all existing patches, not only the visible ones.
It is not the limitation to diffuse surfaces that increases the involved computations, but the fact that radiance is computed for all surfaces in the entire scene, not only the visible ones.

This is in strong contrast to other global illumination techniques like path-tracing, where radiance is computed only for visible samples. While the view-paths may still reach every point of the scene, there might be parts of a scene that are never reached by any view-rays/paths. Therefore, there are no computations at all. Comparing with local GI the "problem of viewpoint independence" its even more apparent.

On the other hand, as the Wikipedia article suggests, this may also be seen as a very useful property, since the computations do not need to be performed for each different viewpoint. This is not the case for most other techniques.


Convert RGB, Hex or any other color format to "Standard Color" Programmatically Forgive me in advance for my ignorance.

Using the PHP function imagecolorat, I have a solution that presently delivers a set of the most dominant hex color codes available inside of an image.

Is there a known library that will allow me to then take the hex color code, RGB code (or any other standard color code) and simplify them down, or rather "match" them to a limited set of "common colors", such as red, brown, green, yellow, grey, blue, etc?

For example, #944D70, which is a shade of purple, would be matched to the color 'purple'.

Alternatively:

Is there a particular pattern or range to RGB colors that would allow me to write such a library myself?

For example, 0,0,0 thru 20,20,20 might all be considered a "black" color. However, it seems as if organizing such a library would be very exhausting if done manually.

Problem is that people do not really agree on a 1:1 mapping of color names. But yes such tools exists, this is a simple nearest neighborhood test. You could use any spatial query module to do this. Here is a simple (naive) python example:

from __future__ import print_function
import webcolors
from scipy.spatial import KDTree

# lets populate some names into spatial name database
hexnames = webcolors.css3_hex_to_names
names = []
positions = []

for hex, name in hexnames.iteritems():
    names.append(name)
    positions.append(webcolors.hex_to_rgb(hex))

spacedb = KDTree(positions)

# query nearest point
querycolor = (10,88, 200)
dist, index = spacedb.query(querycolor)

print('The color %r is closest to %s.'%(querycolor, names[index]))

Which results in: The color (10, 88, 200) is closest to royalblue.

I'm using here the 140 css3 defined colors. Bigger and more comprehensive lists can be found online, if you want a smallerlist that would also work. Theres all kinds of possible things you could do, you could for example linearize the sRGB space before calculating distances to name. You could ask for 3 closest names and give double names if the names are halfway.


Standard ways to create xml scenes for research purposes I am a newbie in computer graphics and just started working on physically based rendering engines such as Mitsuba.

I found it difficult to create my own scenes, that are xml files. To create one, there are three ways as I know so far:

Build a scene in Blender and export it to a xml file using prasers, e.g. https://www.mitsuba-renderer.org/plugins.html.
Download free xml scenes, e.g. from https://benedikt-bitterli.me/resources/.
Hand-write an xml scene with Python from scratch, e.g. this example http://www.djmannion.net/psych_programming/vision/mitsuba/mitsuba.html.

However, I am suffering from following issues:

It takes long time for me to create a scene via 1 and 3, yet not customizable via 2.
I do not know where to get free meshes (and license issues?)

Questions

Are there any better ways to create an xml scene?
What is the canonical way to create delicate scenes, e.g. the ones in SIGGRAPH papers?

(In case someone encounters similar issue)

I end-up in Way 1, and it seems the standard way in academia.

Once I am familar with Blender, the scene creation becomes less challenging; still, it takes some time to create a simple scene. Fortunately for academic papers the number of scenes are a few (< 10).

Unfortunately the old Mitsuba parser does not work for the latest Blender version. So I wrote a custom 2.8x Blender add-on for Mitsuba, as well as creating new, custom material nodes for my own renderer.


Programmatically generating bin files for GLTF models I have a need to generate a series of GLB files. I have a base template for a GLTF file which I have to derive off of. The models thus generated will only have varying textures on them. The base mesh doesn't change at all.

What Im stuck with

Im able to generate the GLB, GLTF files from blender when exporting 3D models. After feeding these two files to: https://sbtron.github.io/makeglb/ Im able to create a GLB file which loads and works fine, with all the textures included.

However I am unable to generate the bin file programmatically. Ive tried searching google for generate bin file from images, but the first few pages dont seem to have anything relating to this.

What Ive tried

Went through this question, but the selected answer there isnt dealing with images:

How to generate binary gltf array buffers?

How do I go about generating a BIN file from a series of images?

The .bin file that goes with a .gltf only includes mesh data (position, normal, uv, other vertex attributes as needed).  It does not normally include any images (unless you're using the base64 export "glTF Embedded", but don't use that for this, use "glTF Separate" instead).

If you look in the JSON text of the .gltf file, you'll see an images array, where image files are called out by uri (filename, if local, or relative URLs for the web).

Let's look at the Damaged Helmet sample model for an example.  The JPG images are called out in the .gltf file, separately from the .bin file.  The data from the .bin file is used by buffers, which in turn are used by bufferViews, accessors, and then mesh primitives.  It is exclusively vertex mesh data, no images.  This is a separate chain from how the JPGs are being used.  The JPGs are called out by images, which are referenced by textures which in turn are used by materials, which are referenced by mesh primitives.  For a more graphical version of this explanation, take a look at the Overview Chart for glTF.

So when exporting from Blender, choose glTF Separate from the combo box in the export options panel.  This will produce a .gltf file, a .bin file, and may copy several images (be sure to export to a different folder from your source files, so nothing gets overwritten).  To change the images, just update the uri fields in the images array inside of the .gltf file, and keep the same .bin file around (the one that came out of Blender).

That way, you can swap out images with other images, but keep the same mesh data.


What is an index buffer and how is it related to vertex buffers? I have a vertex buffer like this:

0.0, 0.0,
1.0, 0.0,
0.0, 0.6,
1.0, 0.6,
0.5, 1.0

I have the following index buffer:

0, 2,
2, 4,
4, 3,
3, 2,
2, 1,
1, 0,
0, 3,
3, 1

I know I want to draw gl.LINES using WebGL, means multiple seperated line segments.

gl.drawElements(gl.LINES, 16, gl.UNSIGNED_SHORT, indexBuffer);

It seems to enable drawing of multiple line segments in a single draw call in WebGL.

Could someone ELI5 to me, what is an index buffer and how is it related to vertex buffers? How to generate index buffers from my primitives?

Could someone ELI5 to me, what is an index buffer and how is it
  related to vertex buffers

Your vertex buffer contains the X and Y coordinates of 5 vertices. They are:

index |  X  |  Y
  0   | 0.0 | 0.0
  1   | 1.0 | 0.0
  2   | 0.0 | 0.6
  3   | 1.0 | 0.6
  4   | 0.5 | 1.0

Your index buffer contains information about which lines to draw between these vertices. It uses the index of each vertex in the vertex buffer as a value.

Since you are drawing lines, each pair of consecutive values in your index buffer indicate a line segment. For example, if the index buffer starts with 0, 2, it means draw a line between the 0th and 2nd vertices in the vertex array, which in this case would draw a line going from [0.0, 0.0] to [0.0, 0.6].

In the following graphic each pair of indices is color coordinated with the line it indicates:

Similarly, if you were drawing triangles instead of lines, you would need to supply an index buffer where each 3 consecutive values indicate the indices of three vertices in the vertex buffer, e.g.

0, 1, 2,
2, 1, 3,
2, 3, 4,


Looking for a cylinder / axis-aligned box volume intersection test Given:

An arbitrary circular cylinder (defined by startpoint, endpoint, and radius). An Infinite cylinder is acceptable as well, as long as it passes through those points and has the same radius.
An axis-aligned box (defined by its minimum and maximum point)
The box being completely inside the volume of the cylinder should also count as an intersection

To test if the cylinder and the box intersect.

Does such a test exist?

EDIT: I found this on the mathmatics stack exchange, though at the moment I do not follow it. If there is an example of it in code form that would be fantastic.

As mentioned the problem of cylinder box intersection reduces to the problem of line-parallelogram intersection, and line segment-cylinder intersection.

I will assume that you know how to intersect a line and a parallelogram. Here is a sketch for solving the line segment-cylinder intersection:

The canonical equation of an elliptic cylinder aligned with the $Z$ axis and having center $(c_x, c_y, c_z)$ is given as:
$$\frac{(x-c_x)^2}{a^2} + \frac{(y-c_y)^2}{b^2} = 1$$
The parametric form of a ray is given as:
$$\vec{r}(t) = \vec{o} + t\vec{d}$$
Then by plugging r into the canonical equation one gets:
$$\frac{(o_x + td_x -c_x)^2}{a^2} + \frac{(o_y + td_y -c_y)^2}{b^2} = 1$$
Solve the quadratic equation for $t$ and you have your intersection. If the discriminant is negative then there is no intersection. To get a rotated cylinder, the easiest way is to apply the inverse rotation to the ray (the equation may be modified however too). Note that for a line segment, you simply need to check whether $t \in [t_0, t_1]$, where $\vec{a} = \vec{o} + t_0\vec{d}$ is the first point of the line segment, and $\vec{b} = \vec{o} + t_1\vec{d}$ is the second one.

EDIT: To clarify you further questions in the comments:

1) a,b in the denominator are the radii along the $X$ and $Y$ axes respectively. In the trivial case where $a=b=r$ we have the canonical equation for a circular cylinder:
$$(x-c_x)^2 + (y-c_y)^2 = r^2$$

2) For the intersection between the cylinder axis and the box a rotation is only required if your box is OOBB and you want to use your AABB intersection code.

3) The solution of the quadratic equation is trivial (note, I renamed $a,b$ to $r_1,r_2$ to further emphasize that they are the radii):

$$\frac{(o_x + td_x -c_x)^2}{r_1^2} + \frac{(o_y + td_y -c_y)^2}{r_2^2} = 1$$
$$r_2^2(o_x + td_x - c_x)^2 + r_1^2(o_y + td_y -c_y)^2 = r_1^2r_2^2$$
$$(r_2^2d_x^2+r_1^2d_y^2)t^2 - 2(r_2^2d_x(c_x-o_x) + r_1^2d_y(c_y-o_y))t + r_2^2(c_x-o_x)^2 + r_1^2(c_y-o_y)^2 - r_1^2r_2^2 = 0$$

Let us substitute:
$$A = (r_2^2d_x^2+r_1^2d_y^2)$$
$$B = (r_2^2d_x(c_x-o_x) + r_1^2d_y(c_y-o_y))$$
$$C = r_2^2(c_x-o_x)^2 + r_1^2(c_y-o_y)^2 - r_1^2r_2^2$$

Then you have the quadratic equation:

$$At^2 -2Bt + C = 0$$
$$D = B^2-AC$$

If $D<0$ there is no intersection between the ray $\vec{o}+t\vec{d}$ and the cylinder, otherwise:

$$t_1 = \frac{B-\sqrt{D}}{A}, t_2 = \frac{B+\sqrt{D}}{A},$$

give you the two intersections (it is one intersection if $D=0$).

Now let your segment vertices be $\vec{v}_a = \vec{o}+t_a\vec{d}$ and $\vec{v}_b = \vec{o} + t_b\vec{d}$. If $t_a \leq t_1 \leq t_b$ then you have an intersection at $t_1$, if $t_a \leq t_2 \leq t_b$ then you have an intersection at $t_2$, otherwise there is not intersection.


Help understanding Perlin Noise I am following this tutorial: Understanding Perlin Noise | Blog

To understand Perlin Noise. In particular, right now I am focusing on the following code section:

int xi = (int)x & 255; // Calculate the "unit cube" that the point asked will be located in
int yi = (int)y & 255; // The left bound is ( |_x_|,|_y_|,|_z_| ) and the right bound is that
int zi = (int)z & 255; // plus 1.  Next we calculate the location (from 0.0 to 1.0) in that cube.
double xf = x-(int)x;
double yf = y-(int)y;
double zf = z-(int)z;

If I am correct, the first 3 lines of code are calculating a set of 3 coordinates (xi,yi,zi) that range from 0 to 255 inclusive, as integers. This is, in other words, a 255x255x255 3D grid of unit cubes.

The next 3 lines are calculating the relative 0-1 coordinates inside of the cube in the grid the point (x,y,z) is mapped to.

My question is:

Is there any reason for this number to be 255? Or is this simply a number that was chosen because it made the algorithm faster (smaller grid, less numbers, faster hash lookup, caching...) and could easily be extended to any other integer (theoretically).

In either case, won't this generate a repetitive pattern?

What I mean by the second question is: If I have a set of (x,y,z) values that extend to "infinity" at regular intervals (let's say they all change at intervals of 1), won't I start seeing a clearly predictable/repetitive pattern emerge after around 255 steps in a given direction?

Finally, if a repetitive pattern emerges, how can you create more variation while keeping pseudo randomness?

Yep, you've got that right. In Perlin's reference implementation of "improved noise", the noise will be periodic, repeating after 256 units along each axis. It's usually not very noticeable even if you have a large extent of noise visible, since there's no large-scale features for the eye to track.

But there's no particular reason it needs to tile after 256 units; that's pretty much just an arbitrary choice made by Perlin. He uses a permutation table as a hash function to map grid coordinates to gradient vectors, and the hard-coded table is of size 256, so has to be tiled after that.

However, any other hash function you like could be used as well. It just has to pseudorandomly map the integer parts of the x, y, z coordinates to a gradient vector.


Shadow rays in Raytracing After spending a few days making very little headway with a simple Raytracing program that implements Phong illumination (with shadows and no attenuation), I'm convinced I've made a logic error that I'm overlooking.

As I understand it, the pseudo-code should look like this:

ambient light
for each light source
    direction of light = light point - point on the object
    test for an intersection starting at the point on the object, until you hit something
    if (the intersection test returned a distance > distance from point on object to light point)
    (do  diffuse and specular shading)

A one quick thing to mention about my implementation: the intersection test will return 9999999 if there is no intersection.

Some of the things I've tried are: inverting the direction of the shadow ray - no effect. Making the comparison intersection distance - light distance > (some small epsilon value) - no change.  Manually moving the light source to the opposite side - the specular reflection moved to a new spot, but otherwise, no change.

Here's a picture of my most recent output, I'm trying to get a shadow in the bottom left corner of the light gray plane.

Any help would be greatly appreciated.

Here's the relevant part of my code:

    //iterate through all lights
    if (myLights.size() > 0)
    {
        for (int i = 0; i < myLights.size(); i++)
        {
            //calculate the direction of the light
            glm::vec3 ldir = (myLights[i].position - surfacePoint);
            //this the length of the distance between the light and the surface
            float ldist= sqrt(pow(ldir.x, 2.0f) + pow(ldir.y, 2.0f) + pow(ldir.z, 2.0f));
//check if the light can see the surface point
            float intersections = myObjGroup->testIntersections(surfacePoint, normalize(ldir));
            if (abs(intersections-ldist)  > 0.0000001f)
            {
                //diffuse and specular shading have been cut for brevity
            }
        }
    }

edit: New image:

Your if condition makes me suspicious. You should include the diffuse and specular shading if the intersection test didn't hit an object; that is, if intersections > ldist. So, your code should look as follows:

if (intersections > ldist)
{
    colour += diffuse(...);
    colour += specular(...);
}

Your comparison with 0.0000001f suggests that you've also been having trouble with self-intersections (that is, with the ray intersecting the object being shaded). In a one-hit tracer, you can't fix that problem by simply discarding the intersection, because you still don't know whether the shadow ray would have intersected another object further along. To avoid self-intersections, you need to push the start point of the ray along its direction by epsilon:

glm::vec3 direction = normalize(ldir);
glm::vec3 start = surfacePoint + epsilon * direction;
float intersections = myObjGroup->testIntersections(start, direction);

One more thing: don't forget to test that the light is in front of the surface that you're illuminating. You can do this with the dot product of ldir and the surface normal: if the dot product is positive, the light is on the correct side; if it's negative, the light is behind the surface and thus shadowed before you even cast a ray.


Correct way to set normal strength I am working on a normal mapping. I need to have the ability to scale normal vectors to adjust the normal strength of the surface. Currently the way I am doing it basically shows plausible results. But I am still not sure that it's 100% correct and to my surprise I found no info on the web regarding this subject.
So here is how I do that (my lightning is in camera space):

vertex shader:

vec3 N = normalize(normalMatrix * a_normal);
vec3 T = normalize(normalMatrix * a_tangent.xyz) * nStrength;
vec3 B = normalize(normalMatrix * (cross(a_normal, a_tangent.xyz) * a_tangent.w) ) * nStrength;

v_TBN =  mat3(T,B,N);

normalMatrix - inverse-transpose of model-view matrix.

The shading itself happens in the fragment shader.
The only relevant part which is happening there is transforming normal from
the texture with v_TBN varying:

Fragment shader:

 vec3 GetNormal(){
    vec3 n = texture(normalMap,uv).xyz;
    n = normalize(n * 2.0 - 1.0);
    return  normalize (v_TBN * n);
 }

So, as you can see, I scale tangent and bi-tangent vectors with nStrength scalar.That's the most optimal setup I figured out that gives most correct looking effect.Is this the right way to strengthen the normals?

Assuming that the normal was created by differentiating some height field f(u,v):

N = normalize(df_du, df_dv, 1)

What you probably want to achieve is to "strengthen" your normals as-if by scaling that height field by a factor C:

g(u,v) = C*f(u,v)

Then your new normal would be:

M = normalize(dg_du, dg_dv, 1) = normalize(C*N0, C*N1, N2)

Translating that to GLSL in your shader:

vec3 GetNormal(){
    vec3 n = texture(normalMap,uv).xyz;
    n = n * 2.0 - 1.0;
    n.xy *= C;
    n = normalize(n);
    return  normalize (v_TBN * n);
}

By constructing a normal map texture that stores only the height-field gradient (df_du, df_dv) without normalization, using the internal format GL_RG8_SNORM for the texture, and making sure that v_TBN is orthonormal, you can simplify it further to:

vec3 GetNormal() {
    vec3 n = normalize(vec3(texture(normalMap,uv).xy, C));
    return v_TBN*n;
}


The Process of Spherical Harmonics I have read about spherical harmonics from Here about the usage of SH for generating Irradiance Environment Maps and how it can make the process of generating EM faster than the regular operations. what they say is :

"if we want to generate a diffuse cube map with an edge length of just 32 texels, from an input cube map with an edge length of 64 texels, the cost is 32x32x6x64x64x6 around 151 million operations. real-time generation of irradiance environment maps cannot be achieved using such a brute-force algorithm.
the runtime complexity for Spherical harmonics method in the example is just 9 x 64 x 64 x 6 around thousand operations per function"

my questions are :

how the cost of generating cube map with the edge length of 32 texels from a 64 one has been calculated as 32x32x6x64x64x6 and why not 32x32x6+64*64*6? and why for SH, the cost of generating is 9x64x64x6 ?
brute-force algorithm finds the shortest path from one node to another. how this algorithm used for that cubemap generation ?

1
For each texel of the generated diffuse cube map (which has $6$ faces times $32 \times 32$ texels), the diffuse equation combines all texels of the environment map (which has $6$ faces times $64 \times 64$ texels). Thus $32 \times 32 \times 6 \times 64 \times 64 \times 6$ as stated.
Since the article considers SH with $9$ coefficients, and each coefficient needs to combine all texels of the environment map, that's $9 \times 64 \times 64 \times 6$.
2
"Bruteforce" doesn't necessarily refer to graph theory. In this article it refers to the naive approach of going through every single texel of the diffuse map and each time combine every single texel of the environment map.

Appendix
Let's describe the problem at hand.
As an input, we have an environment map: it's a set of six textures forming a cube, that represents the light coming from all directions. The light is assumed to be coming from sources far enough that, at our scale, it's as if they were infinitely far. An environment map typically looks like the left figure of this illustration from the GPU Gems article:

As an output, for a any point of a given object that we're rendering, we want to know the diffuse irradiance affecting it. What does that mean?
A parenthesis
Models describing how the perceived color of a surface varies depending the light reaching it, typically consist in integrating all incoming light over an hemisphere, with a weight that depends both on the direction of the light and of the observer.
For opaque materials, it is common to consider two components contributing to that color: the diffuse and the specular. The diffuse is due to light slightly penetrating the surface, getting partly absorbed, then getting out in random directions. The specular is due to light bouncing off the surface without entering it.

Back to the problem
So we want to know the color of the points of an object, we know that to compute it we need to integrate light in all directions, and we have a cube map that represents light coming from all directions. So far so good.
A problem though is integrating basically means we need to go over every single texel of the hemisphere (so half the environment map: $(64 \times 64 \times 6) / 2$ texels), do some math, and add that to the result. That's a lot of computation for each point, which we'd like to avoid if possible.
We know that the contribution of one light depends on the light direction. If we consider the environment to be static (the lighting doesn't change), then we can isolate the part of the computation that only depends on the light and surface normal (and not on the material or the observer), pre-compute it and store it to use later. For the diffuse term, that's the diffuse irradiance mentioned earlier, and it typically looks like the center figure of the illustration. Each pixel represents the irradiance term for a given surface normal. The right figure is the specular environment map, computed in a similar way but with a different integral.

With that texture, instead of painfully integrating, we just fetch the precomputed result, combine it with the rest of the equation (the material diffuse color, etc.) and voilà.
Finding the 32x32x6x64x64x6
Since the result looks very blurry, it probably doesn't need to be $64 \times 64$ for storing, and the article assumes $32 \times 32$. But still, each of those $32 \times 32 \times 6$ texels' color is computed separately, by integrating over the entire corresponding hemisphere. That part never disappeared: we just moved it from the rendering stage to a pre-computation stage.
So that's still $(32 \times 32 \times 6) \times (64 \times 64 \times 6) / 2$. That's half the figure stated in the article. I suppose the author was either assuming a model that integrates of the entire sphere, or just dropped the $0.5$ factor for the sake of clarity.
Finding the 9x64x64x6
We can't help but notice this irradiance diffuse environment map is really blurry: surely there must be a way to store its information using a lot less than $32 \times 32 \times 6$ terms, and hopefully reduce the amount of work as well. That's what spherical harmonics do, by compressing the irradiance in just a few coefficients.
I am not familiar enough with the math to engage in an explanation, but suffice to say it is generally considered that for diffuse, order three SH are sufficient. Order three SH involve nine coefficient, and again, for each of these we need to integrate over the entire hemisphere.
So that's $(9 \times 64 \times 64 \times 6) / 2$ texels to fetch and process. Here too a $2$ factor is missing from the article.


How to use the data manipulated in OpenGL Compute Shader I'm working with OpenGL and facing some difficulties because I'm not familiar with OpenGL. I tried to search related example in Google, but I could not find some useful code.

There are five arrays.
I bound and changed the data in compute shahder as follows:

computeShader.use();
int bufferSize = sizeof(vertices) * vertices.size();
glGenBuffers(1, &vbo_Previous);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Previous);
glBufferData(GL_SHADER_STORAGE_BUFFER, bufferSize, &vertices[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, vbo_Previous);

glGenBuffers(1, &vbo_Current);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Current);
glBufferData(GL_SHADER_STORAGE_BUFFER, bufferSize, &vertices[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 1, vbo_Current);

glGenBuffers(1, &vbo_Next);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Next);
glBufferData(GL_SHADER_STORAGE_BUFFER, bufferSize, &vertices[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 2, vbo_Next);

glGenBuffers(1, &vbo_Normal);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Normal);
glBufferData(GL_SHADER_STORAGE_BUFFER, bufferSize, &normals[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 3, vbo_Normal);

glGenBuffers(1, &vbo_Tangent);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Tangent);
glBufferData(GL_SHADER_STORAGE_BUFFER, bufferSize, &tangents[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 4, vbo_Tangent);

glGenBuffers(1, &vbo_Binormal);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Binormal);
glBufferData(GL_SHADER_STORAGE_BUFFER, bufferSize, &binormals[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 5, vbo_Binormal);
computeShader.unUse();

And I want to use the data in vertex shader.
I tried like that, but I doesn't work.

glGenVertexArrays(1, &vao);
glGenBuffers(1, &ebo);
glBindVertexArray(vao);

int bufferOffset = 0;
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Next);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 4, GL_FLOAT, GL_FALSE, 0, (void*)(bufferOffset));

bufferOffset += sizeof(glm::vec4) * vertices.size();
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Normal);
glEnableVertexAttribArray(1);
glVertexAttribPointer(1, 4, GL_FLOAT, GL_FALSE, 0, (void*)(bufferOffset));

bufferOffset += sizeof(glm::vec4) * normals.size();
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Tangent);
glEnableVertexAttribArray(2);
glVertexAttribPointer(2, 4, GL_FLOAT, GL_FALSE, 0, (void*)(bufferOffset));

bufferOffset += sizeof(glm::vec4) * tangents.size();
glBindBuffer(GL_SHADER_STORAGE_BUFFER, vbo_Binormal);
glEnableVertexAttribArray(3);
glVertexAttribPointer(3, 4, GL_FLOAT, GL_FALSE, 0, (void*)(bufferOffset));

//Because the texture coordinate are not changed in compute shader I generated the vbo for texture coordinate data
bufferOffset += sizeof(glm::vec4) * binormals.size();
glGenBuffers(1, &vbo);
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glBufferData(GL_ARRAY_BUFFER, sizeof(glm::vec2)*texCoords.size(), &texCoords[0], GL_STATIC_DRAW);
glEnableVertexAttribArray(4);
glVertexAttribPointer(4, 2, GL_FLOAT, GL_FALSE, 0, (void*)(bufferOffset));

I hope to use the data in vertex shader as follows:

#version 430 core
layout(location = 0) in vec4 aPos;
layout(location = 1) in vec4 aNormal;
layout(location = 2) in vec4 aTangent;
layout(location = 3) in vec4 aBinormal;
layout(location = 4) in vec2 aTexCoord;

How to set vertex object array to use the data?

I can see a few things wrong here.

First, you're setting your GL buffer size to sizeof(vertices) * vertices.size()
This looks wrong. What is vertices? A std::array of vec4? Then you want vertices.size() * sizeof(glm::vec4), or just sizeof(vertices)

Second, as wandering-warrior said, you want to be binding to GL_ARRAY_BUFFER before calling glVertexAttribPointer

Third, I don't know what you're doing with bufferOffset in the second part of the code. This is a valid approach if the data is laid out sequentially in one large buffer (vertices then normals then tangents etc. etc.). But in your case you're using a separate buffer for each data set - so each attribute should start reading from the beginning of its own buffer, i.e. bufferOffset should always be 0.

Fourth, you don't seem to be actually using your compute shader anywhere. It's not enough to just call glUseProgram, you need to call glDispatchCompute after you bind your buffers to actually run it.

Also, it's not common practice to store binormals in an array, since they can be reconstructed in the fragment shader from the tangent and normal using a cross product.

I would do it something like this (using OpenGL 4.5 with the new Direct State Access functions to save all those buffer bindings).

int bufferSize = sizeof(vertices);

glCreateBuffers(1, &vbo_Previous);
glNamedBufferData(vbo_Previous, bufferSize, &vertices[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 0, vbo_Previous);

glCreateBuffers(1, &vbo_Current);
glNamedBufferData(vbo_Current, bufferSize, &vertices[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 1, vbo_Current);

glCreateBuffers(1, &vbo_Next);
glNamedBufferData(vbo_Next, bufferSize, &vertices[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 2, vbo_Next);

glCreateBuffers(1, &vbo_Normal);
glNamedBufferData(vbo_Normal, bufferSize, &normals[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 3, vbo_Normal);

glCreateBuffers(1, &vbo_Tangent);
glNamedBufferData(vbo_Tangent, bufferSize, &tangents[0], GL_STATIC_DRAW);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, 4, vbo_Tangent);

glCreateBuffers(1, &vbo_Texcoords);
glNamedBufferData(vbo_Texcoords, sizeof(texCoords), &texCoords[0],
GL_STATIC_DRAW);

glCreateBuffers(1, &ebo);
glNamedBufferData(ebo, sizeof(elements), &elements[0], GL_STATIC_DRAW);
//you have element data in an array, right?

glCreateVertexArrays(1, &vao);
glBindVertexArray(vao);

glBindVertexBuffer(0, vbo_Next, 0, 0);
glVertexAttribBinding(0,0);
glVertexAttribFormat(0, 4, GL_FLOAT, GL_FALSE, 0);
glBindVertexBuffer(1, vbo_Normal, 0, 0);
glVertexAttribBinding(1,1);
glVertexAttribFormat(1, 4, GL_FLOAT, GL_FALSE, 0);
glBindVertexBuffer(2, vbo_Tangent, 0, 0);
glVertexAttribBinding(2,2);
glVertexAttribFormat(2, 4, GL_FLOAT, GL_FALSE, 0);
glBindVertexBuffer(3, vbo_Texcoords, 0, 0);
glVertexAttribBinding(3,3);
glVertexAttribFormat(3, 2, GL_FLOAT, GL_FALSE, 0);
glVertexArrayElementBuffer(vao, ebo);

glEnableVertexAttribArray(0);
glEnableVertexAttribArray(1);
glEnableVertexAttribArray(2);
glEnableVertexAttribArray(3);

computeShader.use();
glDispatchCompute (/*number of groups*/)
computeShader.unUse();

glMemoryBarrier(GL_VERTEX_ATTRIB_ARRAY_BARRIER_BIT);

renderShader.use();
glDrawElements(GL_TRIANGLES, elements.size(), GL_UNSIGNED_INT, 0);
renderShader.unUse();


Why is pixel 0,0 at the top left of some formats and bottom left of others? It seems that every image format (jpeg, png, tiff, bmp, jp2000...) doesn't place their origin at the same corner.

Do you know where I could find some info about this differences?

I can only say that PNG and JPEG are different.

It comes down to history. The original interfaces for drawing stuff on the screen were for text, not images. Since the people who designed those early computers spoke languages that are read left-to-right, top-to-bottom, it made sense for the first glyph to be the top-left one. This also works conveniently with how the image was transmitted to the display. TV signals also start in the top-left, so those CRT displays also started displaying a frame with the top-left pixel. When computers added "graphics modes" to their "text modes", it made sense for the origin to stay in the same place.

This all changed with packages specifically designed for mathematical and scientific computing. Charts are usually drawn with (0,0) at the bottom-left, so scientists wanted their co-ordinate system to work that way.

This is why we have two common options. Generally for an image format, it depends on the use-case the format was originally designed to satisfy, and what kind of people worked on it. Image formats are usually designed by engineers with experience of low-level graphics implementation, so top-down is more popular, but some formats support both for flexibility.


How does a rigged model work? What black magic is behind this?

Starting from a pile of polygons, a skeleton, and an initial conformation, how is a new conformation turned into a new pile of polygons?

The magic is that the mesh is attached to the skeleton.  In it's simplest form, this is done by assigning each vertex to a bone.

When a vertex is assigned to a bone, that means that it will always keep the same position relative to that bone's position, and orientation (normal, tangent, bitangent aka the bone's local X,Y,Z axis) as the bone moves as directed by the animation data, physics impulses, or whatever else driving it.

As you might expect, some triangles will have vertices that belong to different bones.  When this happens, since the triangle is made up of those vertices, the triangle will stretch or squish as the bones move.

The link you mention explains a way to set up the vertices for this specific type of movement such that the vertices stretch and squish in realistic and well behaved ways.

In modern games, vertices are usually attached to multiple bones with a weight per bone so that each vertex is actually affected by the movement of multiple bones.  This gives a more natural and organic look to the mesh.

This answer only scratches the surface though, and this is a very deep topic which has many sub topics that are active areas of research!

If you want to read more about skeletal animation, this link is a good starting point:
http://blog.demofox.org/2012/09/21/anatomy-of-a-skeletal-animation-system-part-1/


Paint pixels on the image around the cursor for painting application I'm trying to create a simple painting application. And I was wondering about an efficient way to draw pixels on the screen.
I have a pixel struct:
struct Pixel {
 unsigned char r, g, b, a;
};

And I have two methods for converting between pixel coordinates and normalized pixel coordinates. w and h are image width and height respectively.
int getPixelIndex(float2 norm, int w, int h) {
   return (int)(norm.x*h)*w + norm.y*h;
}

float2 getPixelNormCoords(int index, int w, int h) {
   float2 norm;
   norm.x = (index % w) / (float)(w-1);
   norm.y = (index / h) / (float)(h-1);
   return norm;
}

Now if I want to draw a red pixel under the cursor I could do it like this:
int pixelIndex = getPixelIndex(cursorNorm.x, cursorNorm.y, w, h);

pixels[pixelIndex] = {255, 0, 0, 255};

However what if I want to fill more pixels around the cursor, to create different brushes. Say I want to create a square brush with a given offset d. One hacky way to do this would be:
float delta = 10e-4;
for (float i = -d; i < d; i+=delta) {
  for (float j = -d; j < d; j+=delta) {
    int pixelIndex = getPixelIndex(cursorNorm.x+i, cursorNorm.y+j, w, h);
    pixels[pixelIndex] = {255, 0, 0, 255};
  }
}

Another way would be to iterate over each pixel:
for (int i = 0; i < size(pixels); i++) {
   float2 norm = getPixelNormCoords(i, w, h);

   if (fabs(norm.x-cursorNorm.x) < d && fabs(norm.y-cursorNorm.y) < d)
      pixels[i] = {255, 0, 0, 255};
}

However this way seems to be extremely costly.
Are there more efficient and less hacky methods to accomplish what I'm trying to do?

I don't know, why you consider your first approach hacky, but an alternative would be to analyze the shape of your brush and how it interacts with your underlying data structure. For example for your rectangle:
Let's assume the brush's sides are parallel to the sides of the image. Then you can calculate how man pixels are covered by your offset d. Let's call be the number of pixels between two corners n. Then you can determine the index of the upper left pixel of your rectangle and fill the next n indices since they represent the left border of your rectangle. Then add h to the index of the upper left point to get to the pixel right of it. Fill the next n pixels. Repeat this n times and you have finished filling your rectangle. Here is some untested code that should demonstrate the approach:
float left_x = cursorNorm.x - d;
float upper_y = cursorNorm.y - d;
float lower_y = cursorNorm.y + d;

int index_upper_left = getPixelIndex(left_x, upper_y, w, h);
int index_lower_left = getPixelIndex(left_x, lower_y, w, h);

int n = index_lower_left - index_upper_left;

for (int i=0; i < n; ++i)
    for (int j=0; j < n; ++j)
        int index = index_upper_left + j*h + i
        pixels[index] = {255, 0, 0, 255};

Since this is all symmetric, you can vary this to start at the center point, run i and j just up to n/2 and do something like that:
for (int i=0; i < n/2; ++i)
    for (int j=0; j < n/2; ++j)
        int index_left = index_center - j*h
        int index_right = index_center + j*h

        pixels[index_left + i] = {255, 0, 0, 255};
        pixels[index_left - i] = {255, 0, 0, 255};
        pixels[index_right + i] = {255, 0, 0, 255};
        pixels[index_right - i] = {255, 0, 0, 255};

More complex shapes like circles get a little bit more complicated but you can calculate for each affected column of pixels the index of the upper and lower affected pixel and fill between them. Remember that you can abuse symmetry here. Meaning if you found the upper pixel of a column, the lower pixel has the same y distance to the y-value of the center point. Additionally, the bounds to the left and to the right of the center are also identical. A helpful algorithm to find the bounds might be Bresenham’s circle drawing algorithm.


Dealing with OpenGL versions across multiple computers I'm new to OpenGL, and I'm wondering what I need to pay attention to regarding versioning. The things I've been reading have been clear about there being version differences, and needing the appropriate OpenGL version for the functionality you're using. But they've mainly been talking about compiling and running programs on a single machine. For the most part they haven't explicitly dealt with complexities regarding compiling on one machine and running on another. (Specifically, what happens if the computer you're running on has an older version of OpenGL than the one you're compiling on.)

For example, take glDrawElementsInstanced. The Khronos page indicates this is available with OpenGL 3.1 and later. Obviously, if the headers for OpenGL on the machine I'm compiling for are version 3.0 or earlier, I'll get a compiler error. So on the machine I'm compiling on, I'll need to have OpenGL 3.1+ available.

However, I'm wondering what might happen if the machine I compile for has OpenGL 3.1+ available, but the machine I'm running on only has support for 3.0 or earlier? Will the program that uses glDrawElementsInstanced refuse to load? Will it run but crash once I call the glDrawElementsInstanced() function? Will it work fine because the actual implementation is compiled into my program, and it only uses common back-end functionality? Is the answer different for static versus dynamic linkage? -- And perhaps most importantly, if it wasn't glDrawElementsInstanced but some other function, is there a way I can determine this on my own?

I'm interested in distributing on Windows, Mac and Linux, so if there's any differences in behavior between the platforms in this regards, I'd be interested in knowing.

After running across an aside in a tutorial and doing some further digging, I think I understand this now.

According to the OpenGL ABI specification, only the functions defined in OpenGL 1.2 need to be present in the GL/gl.h header file. (At least for Linux - other platforms may be different. This answer indicates that Windows may expose only OpenGL 1.1 functions by default.) Functionality from more recent OpenGL versions (even commonly used functions) may not even be defined in the header files. -- This seems to be the case for glDrawElementsInstanced. In the headers of one Linux version I looked at, by default glDrawElementsInstanced isn't exposed by the the headers, even though OpenGL 4.5 is supported.

Instead, the functionality from newer OpenGL versions are apparently handled through the OpenGL extension system. (Even if it's not strictly an "extension", but part of standard functionality.) That is, each individual user is nominally responsible for loading the function address into the appropriate function variable using the appropriate (platform-specific) *GetProcAddress() function from the GL/glext.h header. The usage is to pass the appropriate string to the function, which will return a function pointer to the appropriate function (if present), or a null pointer if the functionality can't be found. (See the tutorial section for more detail on usage.)

This is, obviously, somewhat less than satisfactory if you're using modern OpenGL features. There is a preprocessor define GL_GLEXT_PROTOTYPES which, when defined prior to including GL/glext.h, will enable the definitions of a large number of standard functions. However, this may not work robustly if you're running on a computer which doesn't implement all of those functions which you're using. (You're likely to get missing symbol error during shared library loading.)

The better way to handle things is to use the OpenGL Extension Wrangler (GLEW). This hides the heavy lifting of the whole *GetProcAddress() process, defining symbols like glDrawElementsInstanced directly in the glew.h header. So long as you call glewInit() after initializing OpenGL but before using any version dependent function, these functions should be initialized appropriately if available, or to a null pointer if not.

You can either test for the function pointer being non-null, or use the GLEW-provided version test variables like GLEW_VERSION_3_1, which are booleans (not compile-time defines) indicating if that OpenGL version is supported on the computer the program is running on. Alternatively, you can test on a feature-by-feature level by using glewIsSupported()/glewGetExtension(). You can look up the respective strings for individual extensions at the OpenGL Registry. It doesn't lend itself to forward searching, but may permit use of features prior to the nominal version of OpenGL in use.

So why wasn't this clearer to me before? I think it's because most of the tutorials and documentation about OpenGL simply assume that you're using GLEW. Despite its name, GLEW isn't just about extensions, it also abstracts much of the version-to-version compatibility issues. As long as you put use behind version checks and don't actually attempt to call a non-supported function, GLEW hides all the pesky details about functions which aren't actually being implemented.