understanding SDF[Signed Distance Functions] for a torus This topic would contain a lot of sub questions because I have one for each shape but for today I just want to understand the function for a torus and a capped torus
This is the article for all shapes and I am trying to understand the code for torus
float sdTorus( vec3 p, vec2 t )
{
  vec2 q = vec2(length(p.xz)-
  t.x,p.y);
  return length(q)-t.y;
 }

and capped torus
 float sdCappedTorus(in vec3 p,
  in vec2 sc, in float ra, in
  float rb)
{
  p.x = abs(p.x);

  float k = (sc.y*p.x>sc.x*p.y) ?
  dot(p.xy,sc) : length(p.xy);

   return sqrt( dot(p,p) + ra*ra
 - 2.0*ra*k ) - rb;
}

I have so many questions. I know parameter P is the point for testing but what is t? what is ra and rb? what calculations are they doing what are the mathematics?
Just a theoretical explanation or some images explaining the general formula would be greatly appreciated.

This turned into a rabbit hole for me hopefully it helps answer your question, I recommend trying out the various functions on the ShaderToy website. Just drop them in and change the call to "sdCircle". I used the values R=0.75 and r=0.35 to test them out.
Part 1: Defining what an SDF means
Signed Distance Field or SDF, in this context, is a function that returns the signed distance from a point to a primitive. The returned value is always a float that can take on 3 context specific meanings.

zero: This is on the surface, while it is rare to hit exactly on the surface of the primitive we can approximate a hit with a simple inequality like abs(d)<sigma where d is the signed distance and sigma is some small value like 0.001.
negative: In this context negative values represent hits inside the primitive but choosing the meaning of a negative return value is context specific. Smaller (more negative) values represent values that are deeper in the interior of the primitive.
positive: Again this is context specific, but in this case this is a hit outside the primitive and larger values mean it is farther away from the primitive.

Stop for a second and take another look at the first case, "zero". This may seem unimportant at first glance but it turns out that by defining a generic base primitive and then changing the value of sigma we can easily compute a primitive of any size. In the case of a torus we could have a very thin torus or one that is so big it envelopes itself. Then by adding one more variable representing the large Radius of the torus, the torus can now be changed to fit our needs as desired just by controlling two variables.
Part 2: Defining the primitive
Take a look at IQ's 2D function for a circle.
float sdCircle( vec2 p, float r ){ return length(p) - r; }

All this function is doing is computing the "distance" using the length function of our point "p" from our circle (which is sitting at the origin) then subtracting our radius "r". And easy as that any point greater then radius is positive, any point less then radius is negative.
This is almost a torus already, we are just missing the signed distance for the inside diameter of the 2D torus. But we already know that is really just some sigma value from our original circle, so all we have to do is add a second radius and compute the sigma.
So:
float sdTorus2D( vec2 p, float R, float r )
{
  float q = length(p)-R;
  return abs(q)-r;
}

No need to recalculate the distance, so just figure out sigma to our second circle with abs(q)-r.
Part 3: Going 3D
If you stop and think about it for a second (ok...maybe more then a second) then it is easy to realize that our 2D torus is just a slice of a 3D torus. When you think about it like that then we can take another slice of our torus in 2D on its edge just by rotating our torus 90 degrees. This should result in two SDF circles side by side.
But turning the torus on its side is really just a matter of computing our signed distances in the opposite order...lets try:
float sdTorus2DSide( vec2 p, float R, float r )
{
  vec2 q = vec2(abs(p.x)-R,p.y);
  return length(q)-r;
}

The two different views essentially compute the distance in two different ways, so by combining them we should be able to view our torus in 3D from any angle just by adding an extra component "z" so that both computations can be independent of each other. Then combine our two radii into a vec2 and we end up with a function that looks just like IQ's.
float sdTorus( vec3 p, vec2 t )
{
  vec2 q = vec2(length(p.xz)-t.x,p.y);
  return length(q)-t.y;
}

This would still look two dimensional in the "circle" shadertoy, but now we can vary z to look at the torus slices. Or rotate our view point and see it emerge from the 2D world.  Neat stuff!
BTW you can go through this same process with many primitives and get many different shapes as IQ has demonstrated on his website.


How do we approximate the integral of the measurement contribution function in Metropolis Light Transport? In Metropolis Light Transport, we need to approximate the integral of the measurement contribution function. I've read that this can be done using a traditional approach like path tracing. However, how exactly do we need to implement this approximation?

I've read the explanation in PBR3, but I still don't understand how this approximation is done exactly.

More precisely, let's make the following definitions (ignoring rigor): Let $M\subseteq\mathbb R^3$ denote the scene surfaces, $\sigma_M$ denote the surface measure on $M$, $$\Omega_n:=\left\{x\in M^{\mathbb N_0}:x_i\ne x_j\text{ for all }0\le i<j\le n\text{ and }x_N=x_n\text{ for all }N\ge n\right\}$$ $$\iota_n:\Omega_n\to M^{\left\{0,\:\ldots\:,\:n\right\}}\;,\;\;\;x\mapsto(x_0,\ldots,x_n)$$ $$\mu_n(A):=\sigma_M^{\otimes\left\{0,\:\ldots\:,\:n\right\}}(\iota_n(A))\;\;\;\text{for }A\subseteq\Omega_n$$ $$\Omega:=\biguplus_{n\in\mathbb N}\Omega_n$$ $$\mu(A):=\sum_{n=1}^\infty\mu_n(A\cap\Omega_n)\;\;\;\text{for }A\subseteq\Omega$$ Moreover, with the usual notation, let $$T(x_0,\ldots,x_n):=\prod_{i=1}^{n-1}f_r(x_{i+1}\to x_i\to x_{i-1})G(x_i\leftrightarrow x_{i+1})$$ $$f(x):=L_e(x_n\to x_{n-1})T(x_0,\ldots,x_n)\;,\;\;\;\text{if }x\in\Omega_n$$ $$h_j(x):=G(x_0\leftrightarrow x_1)W_e^{(j)}(x_0\to x_1)$$ Then, the measurement $I_j$ of the $j$th pixel is $$I_j=\int fh_j\:{\rm d}\mu.$$

Now, the question is: How exactly is $$b:=\int f\:{\rm d}\mu$$ approximated using (unidirectional) path tracing?

This approximation is typically done by running a bidirectional path tracer with a modest amount of samples per pixel. This means multiple paths per pixel to approximate the integral of $f_j$, the measurement contribution function. Veach's original MLT paper explains how it can be done in a way that eliminates start-up bias (see Section 5.1).


Gravitational calculation of a particle via GLSL / WebGL I have a running particle system which emits particles in a given radius. Now I want to add some gravity to it. The particles and velocities of each particle is saved in a 2D Texture, now I want another 2D Texture, where I can save the gravity, but i'm missing the knowledge to do so.
What's the actual formula on creating some gravitation or how can it be simulated?

For example to save particle's vel, I have
    uniform sampler2D pv;
which is just an image which outputs an image and takes the output as input into the shader. (Kind of transform feedback)

uniform float spread;
vec2 vel = ...;
gl_FragColor = vec4(spread*vel, 0.0, 0.0);

(Part of my code)

And now I simply want another uniform sampler2D pg; which holds gravity and right here, I'm missing the math...

Gravity is a form of acceleration. So what I've done in the past is continually add it into the velocity. On each frame, for each particle, I'd do something like this:

// Gravitational acceleration, adjust as necessary
const vec2 gravity = vec2(0.0, -9.8);

// Calculate the new velocity for the particle
velocity = (particleVelocity + gravity) * timeDelta;

// Calculate the new position given the new velocity
newParticlePosition = particlePosition + velocity;

// Update our parameters for the next time around
particleVelocity = velocity / timeDelta;
particlePosition = newParticlePosition;

In this case timeDelta is the fraction of a second between the previous frame and the current frame. So if you're running your animation at 60 fps, it would be 1.0 / 60.0 = 0.01666…

I'm not sure that using a texture for the gravity would help you here as gravity is usually a constant when you're close to the surface of the planet, as it sounds like your particles are from the description given.


What is the simplest way to create basic 3d/isometric style images programatically like autocad/blender I need the simplest route to programatically, via command line, through python or node, to create something like this using basic input of distance and 90 degree turn etc: https://i.imgur.com/SdY02gQ.png
I normally use blender for such graphics, but manually. I have bee told its over kill to use for my basic need via python, not sure where to start.
I have been told three.js is the wrong choice to try and do something like that as its for browser use mainly and I just want to create rendered images to disk, from input, on the command line.
any advice would be appreciated. I am building a fencing quote system and would like to have basic graphics accompany it. I feel like autocad/freecad would be perfect if it had such options.

Since your actuallyu doing a PDF file you should do this as a vector drawing not a image format. This makes for way more clearer end results. I will generate a image in EPS, EPS is easy to build into your PDF file with say distiller, ghostscript, sent to printer or injected into TeX (though then you may want to use Tix or or PStricks) .
Ok first you have to undertsand that EPS is a full programing language (Postscript), in addition to being a vector format (the Encapsulation part). I will be uisng the features to remove repetition. The file begins with a encapsulation marker so that you can more easily insert it programmatically.
So frist draw the thing form a overhead view, then we will transform that into isometric with a affine transform:

Image 1: Preview of listing 1
%!PS-Adobe-3.0 EPSF-3.0
%%BoundingBox: 0 0 538 300
%%Title: listing 1
%%Creator: joojaa
%%CreationDate:  2020-09-23
%%EndComments

% postscript style shorthand relative
/m {moveto} def
/rm {rmoveto} def
/rl {rlineto} def

10 10 translate % add some padding
72 25.4 div 100 div  dup scale  % draw in  1:100
70 setlinewidth %0.7 mm technical pen

newpath  % coordinates and ops follow
0 4100 m
0 5500 rl
9000  0 rl
0 -4000 rl
4000 0 rl
0 -1300 rl
0 -800 rm
0 -1000 rl
stroke

%%EOF

Ok then we transform the entire thing to isometric

Image 2: Image after Listing 2. Only stroke changed
%!PS-Adobe-3.0 EPSF-3.0
%%BoundingBox: 0 0 538 300
%%Title: listing
%%Creator: joojaa
%%CreationDate:  2020-09-23
%%EndComments

% postscript style shorthand relative
/m {moveto} def
/rm {rmoveto} def
/rl {rlineto} def

10 10 translate % add some padding
72 25.4 div 100 div  dup scale  % draw in  1:100
70 setlinewidth %0.7 mm technical pen

newpath  % coordinates and ops follow
0 4100 m

0 5500 rl
9000  0 rl
0 -4000 rl
4000 0 rl
0 -1300 rl
0 -800 rm
0 -1000 rl

%changed stroke to this
gsave
 false upath
  9600 60 sin mul 0 translate
  [60 sin 60 cos -60 sin -60 cos 0 0] concat
  ustroke
grestore
%eof changed

%%EOF

Then we duplicate this and offset

Image 3: After offsetting
%!PS-Adobe-3.0 EPSF-3.0
%%BoundingBox: 0 0 538 300
%%Title: listing 3
%%Creator: joojaa
%%CreationDate:  2020-09-23
%%EndComments

% postscript style shorthand relative
/m {moveto} def
/rm {rmoveto} def
/rl {rlineto} def

10 10 translate % add some padding
72 25.4 div 100 div  dup scale  % draw in  1:100
70 setlinewidth %0.7 mm technical pen

newpath  % coordinates and ops follow
0 4100 m

0 5500 rl
9000  0 rl
0 -4000 rl
4000 0 rl
0 -1300 rl
0 -800 rm
0 -1000 rl

gsave
 false upath
  9600 60 sin mul -1100 translate
  [60 sin 60 cos -60 sin -60 cos 0 0] concat
  ustroke
grestore

gsave
 false upath
  9600 60 sin mul 10 translate
  [60 sin 60 cos -60 sin -60 cos 0 0] concat
  ustroke
grestore

%%EOF

Next draw connecting lines

Image 4: Listing 4 result
%!PS-Adobe-3.0 EPSF-3.0
%%BoundingBox: 0 0 538 300
%%Title: listing 4
%%Creator: joojaa
%%CreationDate:  2020-09-23
%%EndComments

% postscript style shorthand relative
/m {moveto} def
/rm {rmoveto} def
/rl {rlineto} def
%draw connecting line

/tf { 9600 60 sin mul 10 translate
    [60 sin 60 cos -60 sin -60 cos 0 0] concat} def

/dc {gsave
    newpath
    9600 60 sin mul 10 translate
    [60 sin 60 cos -60 sin -60 cos 0 0]
    concat m -1100 -1100 rl
    stroke
    grestore
     } def

10 10 translate % add some padding
72 25.4 div 100 div  dup scale  % draw in  1:100
70 setlinewidth %0.7 mm technical pen

newpath  % coordinates and ops follow

0 4100 m
0 5500 rl
9000  0 rl
0 -4000 rl
4000 0 rl
0 -1300 rl
0 -800 rm
0 -1000 rl

%drawing no need to change
gsave
 false upath
  tf
ustroke
grestore

gsave
 false upath
  0 -1100 translate
  tf
  ustroke
grestore

{dc} {dc} {} {} pathforall

%%EOF

Todoo:

Change stroke to round
Draw in paperspace
Draw distances.
Compute bound and center
Etc.

I might return later


Using extensions in Android OpenGL ES 2.0 It is no secret that according to the official documentation extensions are not available under OpenGL ES 2.0. Nevertheless, the glext.h file present in the NDK platform-include directories makes me think that extensions are indeed available. I know that working with OpenGL under NDK doesn't differ from working with standalone OpenGL. So, if I make something like a JNI bridge between my java engine interface and these extensions, I could use them.

So the question is: what architectural solution should I use if I want to use available OpenGL ES extensions on ES2.0 devices?

I have encountered devices that advertise certain extensions via glGetString(GL_EXTENSIONS) and do not expose the entry points. I have also encountered the opposite, the extension was not in the extension strings but the entry points were there and it worked. It depends on whether the vendor has disabled any features in the driver from the GPU provider.

You will need to check both the extension strings and check if the entry point is not NULL via eglGetProcAddress.

Apart from that, each extension should conform to the Khronos registry of extensions so you can include gl2ext.h directly from there if you wanted.


Blurring image with RGB values without convolving it with a kernel I'm  using an app for face redaction that doesn't allow access to the source code but only allows me to pass pixel values for red, green and blue channel upon which it creates a matrix with the same average RGB values for every ROI pixel value. For eg. if I give Red=32,Blue=123 and Green=233 it will assign these RGB values for every pixel of the ROI and then draws a colored patch on the face.

So I was wondering is there a general combination of RGB values of a pixel to distort it and make it look like it's blurred. I can also set the opacity value in the app.

Thanks.

Blurring an image depends on being able to read from the image. If you don’t have access to the pixel data, or external information about the image contents (e.g. average hue / luminance), there’s no way to find an overlay color that will look like you’ve blurred what it’s on top of.


What are the advantages of having two chroma channels and a luma channel vs. three chroma channels? RGB has the advantage over schemes like YPBPR in that it's more intuitive to understand, so why use the latter?  The only advantage I can think of right now is that it allows you to encode greyscale images more easily.  Is this the reason, or are there other reasons?

From a historical context, the YUV color encoding was originally designed to support the existing black-and-white infrastructure. In addition, as jooja mentioned, humans are generally less perceptive to changes in chroma and more perceptive to changes in luma. Both NTSC and PAL use less bandwidth for the chroma channel than the luma channel. In a digital context, we can use chroma subsampling to achieve the same effect.

YUV/YIQ/YDbDr is used for color PAL/NTSC/SECAM television broadcasts.
YPbPr is used for component video cables.
YCbCr is used for everything else.
YCgCo is like YCbCr but is easier to compute.


Shader Prototyping Frameworks I wonder if there exists any shader prototyping framework where I can debug (on cpu) my shaders, experiment with them and instantly see results and even write unit tests. The framework does not have to emulate rendering pipeline, compute only will do.

If you really need this level of debugging (spoiler: you don't, more on that later), AFAIK only NVIDIA's NSight can provide it. However, in order to remain interactive, you probably need two machines - one to actually run your shader, and the other to debug it - because stopping a GPU for debugging will - quite understandably - freeze the display of the computer, making it unusable.

One step down from that is RenderDoc and its ability to swap shaders in a capture on the fly, as well as step through Direct3D bytecode and inspect its registers (i.e. a virtual machine for the DXBC, it's most often sufficiently detailed to actually find shader bugs).

And what most graphics programmers do most days is simply what I like to call "your mom's debugger", which on the CPU means sprinkling your code with print statements with the intention of later reviewing the log; but in shaders, you just stuff your shader outputs with debug values instead and inspect those values. There are also ways to "print text" from shaders (for instance, this ShaderToy).


Inverted yaw angle in the view matrix I'm trying to construct a FPS view matrix for my OpenGL renderer using yaw and pitch angles instead of the typical LookAt view matrix.

The view matrix is the inverse of the camera world transform $\mathbf{M}_{\textrm{view}} = (\mathbf{T}\mathbf{R}_p\mathbf{R}_y)^{-1}$, hence:

$$\small\begin{align}
(\mathbf{T}\mathbf{R}_p\mathbf{R}_y)^{-1} &= \mathbf{R}_y^{T}\mathbf{R}_p^{T}\mathbf{T}^{-1} \\
&= \begin{bmatrix}
\cos{y} & 0 & \sin{y} & 0 \\
0 & 1 & 0 & 0 \\
-\sin{y} & 0 & \cos{y} & 0 \\
0 & 0 & 0 & 1
\end{bmatrix}^T
\begin{bmatrix}
1 & 0 & 0 & 0 \\
0 & \cos{p} & -\sin{p} & 0 \\
0 & \sin{p} & \cos{p} & 0 \\
0 & 0 & 0 & 1
\end{bmatrix}^T
\begin{bmatrix}
1 & 0 & 0 & e_0 \\
0 & 1 & 0 & e_1 \\
0 & 0 & 1 & e_2 \\
0 & 0 & 0 & 1
\end{bmatrix}^{-1} \\
&=
\begin{bmatrix}
\cos{y} & 0 & -\sin{y} & 0\\
0 & 1 & 0 & 0 \\
\sin{y} & 0 & \cos{y} & 0\\
0 & 0 & 0 & 1
\end{bmatrix}
\begin{bmatrix}
1 & 0 & 0 & 0\\
0 & \cos{p} & \sin{p} & 0\\
0 & -\sin{p} & \cos{p} & 0\\
0 & 0 & 0 & 1
\end{bmatrix}
\begin{bmatrix}
1 & 0 & 0 & -e_0\\
0 & 1 & 0 & -e_1\\
0 & 0 & 1 & -e_2\\
0 & 0 & 0 & 1\\
\end{bmatrix} \\
&=
\begin{bmatrix}
\cos{y} & \sin{p}\sin{y} & -\cos{p}\sin{y} & -e_0\cos{y} - e_1\sin{p}\sin{y} + e_2\cos{p}\sin{y}\\
0 & \cos{p} & \sin{p} & -e_0 \cdot 0 - e_1 \cos{p} - e_2 \sin{p}\\
\sin{y} & -\cos{y}\sin{p} & \cos{p}\cos{y} & -e_0\sin{y} + e_1\cos{y}\sin{p} - e_2\cos{p}\cos{y}\\
0 & 0 & 0 & 1
\end{bmatrix}
\end{align}$$

Every step is double checked using WolframAlpha.

The implementation:

inline Matrix4
FPSViewRH(const Vector3& eyePosition, float yaw, float pitch) noexcept
{
  yaw = Utils::Radians(yaw);
  pitch = Utils::Radians(pitch);

  const auto sinYaw = std::sin(yaw);
  const auto cosYaw = std::cos(yaw);

  const auto sinPitch = std::sin(pitch);
  const auto cosPitch = std::cos(pitch);

  const Vector3 i{cosYaw, sinPitch * sinYaw, -cosPitch * sinYaw};
  const Vector3 j{0, cosPitch, sinPitch};
  const Vector3 k{sinYaw, -cosYaw * sinPitch, cosPitch * cosYaw};

  return {
    {i[0], i[1], i[2], -i.Dot(eyePosition)},
    {j[0], j[1], j[2], -j.Dot(eyePosition)},
    {k[0], k[1], k[2], -k.Dot(eyePosition)},
    {0,    0,    0,    1},
  };
}

It works fine except that the yaw is inverted, i.e. increasing the angle causes the object to go right while it should go left.

I can negate the yaw angle or change every $\sin{y}$ to $-\sin{y}$, but I don't really understand why this is happening?

It's happening because they just happened to define the rotation matrix with counterclockwise rotation direction, which is the common convention for polar$\rightarrow$cartesian coordinate system transformation. If you google for the transformation, you'll see that the angle is universally shown to rotate to the counterclockwise direction like below


What is the performance tradeoff between forward and deferred rendering? Forward rendering is the process of computing a radiance value for a surface fragment directly from input geometry and lighting information. Deferred rendering splits that process into two steps: first producing a screen-space buffer containing material properties (a geometry buffer, or G-buffer) built by rasterizing the input geometry, and second producing a radiance value for each pixel by combining the G-buffer with lighting information.

Deferred rendering is often presented as an optimization of forward rendering. One explanation is that lighting is fairly expensive and if you have any overdraw then you are lighting pixels that will never be seen on screen, whereas if you store material properties into a G-buffer and light afterwards, you are only lighting a pixel that will actually appear on-screen. Is this actually an advantage of deferred, given that you can also do a depth pre-pass and then do a forward rendering pass with depth test set to D3D11_COMPARISON_EQUAL or GL_EQUAL or the equivalent?

Deferred rendering also has the potential to schedule better on the GPU. Splitting one large warp/wavefront into a smaller geometry wavefront and then smaller lighting wavefronts later improves occupancy (more wavefronts in flight simultaneously). But you also end up with a lot more bandwidth use (writing out a large number of channels to the G-buffer, then reading them back in during lighting). Obviously the specifics here depend a lot on your GPU, but what are the general principles?

Are there other practical performance considerations when deciding between forward and deferred rendering? (Assume that we can use variations of each technique if necessary: i.e. we can compare tiled forward to tiled deferred as well.)

It is possible to avoid overdraw from opaque objects even with forward rendering by doing a depth pre-pass and using that information to reject any pixel that is not actually visible. However, depending on the vertex cost of your scene, a depth pre-pass may add an unacceptable amount of performance overhead. Additionally, rendering using the pixel shading pipeline of the GPU means that you don't pay a cost per pixel that is rendered, you pay a cost per 2x2 pixel quad that is rendered. So even doing a depth pre-pass still causes triangle edges to waste work shading pixels that will be discarded.

GPU scheduling is a complex topic, and the tradeoff between forward and deferred does not boil down simply to "runs faster but uses more bandwidth." If you have two equally cheap operations that run in sequence and each use the same number of resources, there's no reason to split them into separate shaders: two small wavefronts that each use X resources don't fundamentally work better than a single longer wavefront that also uses X resources. If you have a cheap operation and an expensive operation to run in sequence, though, it may benefit from splitting into separate shaders: the shader in general will reserve the maximum amount of resources that it might use at any point. It's conceivable that forward rendering may not be able to use all the bandwidth of your GPU because there are so few wavefronts in flight that it cannot issue enough operations to saturate the bandwidth. But if you are bandwidth limited, there may be no advantage to deferred rendering (since it will probably use more bandwidth).

An additional performance concern is that forward rendering supports different material types (different BRDFs, say) simply by using a different shader for that object. A straightforward deferred renderer needs to handle different material types in a different way (potentially a branch in the shader), since work is no longer grouped into warps/wavefronts coherently depending on the object being rendered. This can be mitigated with a tiled renderer—if only specific areas of the screen use an alternate material type (say, for hair), then you can use the shader variation with a material type branch only for tiles that contain any pixels with that material.


Calculate Transformation between two point sets - but with constrained Degrees of Freedom How can I calculate the rigid transformation [R|t] between two 3d triangles, but restricted to a given N degrees of freedom (for N = 1..6) ?

I know for N=6 I can get a least-squares solution via SVD of a certain matrix, but how can I integrate further constraints (fewer DOF) into the system?

Every 3D rotation has an associated space that can be generated by 3 basis vectors, which can be seen as the axis of the coordenate system of this space. These vectors are also the 3 columns of the transformation matrix. In order to decrease the dof of the rotation, you should provide one or more of these basis vectors and solve for the rest of the matrix. The same principle applies to the translation vector, you should provide constants for one or more distances of the translation. Of course, if you restrict the transformation too much you will end with an unsolvable exact system or a transformation that take the first triangle too far away from the second one, if you are using a non-linear solver.


"Light intensity" of an RGB value EDIT: I'm not fully clear on terminology here, what I previously understood to be called "brightness" seems poorly defined. I think I am asking about radiant intensity, that is, the measurable intensity of energy emitted, not weighted for subjective perception.

You can calculate subjective brightness of an RGB value by weighting the three channels according to their perceived brightness, e.g. with something like:

(0.21 × R) + (0.72 × G) + (0.07 × B)

But, ignoring human biology and perception, is there an accepted way to calculate the objective, theoretical "light intensity" of an RGB value? (Theoretical because the real value will vary between displays)

My first thought, given that the RGB values correspond to the intensity of the pixel's three lights, is to take the average:

(R + G + B) ÷ 3

So, for instance, that makes yellow (#ffff00) twice as intense as red (#ff0000).

That makes sense, thinking about two lights being on rather than one, but looking at the colours I would have guessed they were of somewhat equal intensity.

Unlike dark yellow (#808000) which actually is equal to the red in intensity!

I suspect there isn't a linear relationship which is confusing the issue.

Can objective light intensity be calculated from an RGB value?

The short answer: no, but if you are interested in details, please keep reading (:

About lighting units

Light “brightness” is indeed quite poor/ambiguous layman’s definition for brightness of a light source. Below is a list of different lighting properties/units commonly used in lighting calculations that define the light “brightness”. I listed both radiometric and photometric [in brackets] properties/units. Radiometric units are Human Visual System (HVS) neutral "physical" units defined based on Watts (W), while photometric units are HVS weighted units defined based on Lumens (lm).

Photometric units are commonly used in rendering because practically all consumer image input and output devices operate in these units as they are designed & optimized for image acquisition & consumption by normal tristimulus (LMS) HVS. Proper conversion between photometric & radiometric units isn’t simply a matter of multiplication by some factor (like between kilometers & miles or kilograms & pounds), but different projection from Spectral Power Distribution (SPD) of visible light spectrum (explained later). Some people use constant luminous efficacy of 683 lm/W to convert between the units, but this has very little touch with reality and results in unrealistic values.

Radiant Flux in Watts (W) [Luminous Flux in Lumen (lm)] - Total power emitted by a light (regardless
of shape or size)
Radiant Exitance in W/m^2 [Luminous Exitance in lm/m^2 or Lux] - Power emitted by a light per unit area to all directions (think different size light sources with equivalent Radiant Flux)
Radiant Intensity in W/sr (sr=steradian) [Luminous Intensity in lm/sr or Candela] - Power emitted by a light to certain direction (think of a spot vs omni light with equivalent Radiant Flux)
Radiance in W/sr/m^2 [Luminance in lm/sr/m^2 or Nits] - Power that falls on a patch of surface from some direction (e.g. pixel on screen or patch of surface from certain direction)
Irradiance in W/m^2 [Illuminance in lm/m^2 or Lux] - Power that falls on a patch of surface regardless of direction (e.g. hemispherical sky illuminating a patch of ground).

It’s important to understand the difference between these properties when performing lighting calculations, just like it is important not to confuse properties of length and speed when doing physics calculations. There are various other light properties but having good grasp of the above gets you pretty far already.

Visible light spectrum and RGB projection

Visible light spectrum contains “infinite” number of electromagnetic wavelengths from ~390nm to ~730nm.

The power spectrum emitted by light is defined with SPD (in W/m^2/nm) and you can calculate the “HVS neutral” physical light intensity, i.e. Radiant Exitance $M_e$ (in W/m^2) by integrating the spectrum over the visible light range:
$$M_e=\int_{390}^{730} S(\lambda)d\lambda$$
However, RGB color space is defined in photometric units by projecting SPD using HVS weighted color matching functions. This projection is done because storing SPD would consume immense amount of memory for little gain for image visualization purposes. The projection of SPD to RGB is defined with integrals over the visible light range as follows:
$$R=\int_{390}^{730} S(\lambda)\bar{r}(\lambda)d\lambda$$
$$G=\int_{390}^{730} S(\lambda)\bar{g}(\lambda)d\lambda$$
$$B=\int_{390}^{730} S(\lambda)\bar{b}(\lambda)d\lambda$$
CIE 1931 defines standard observer RGB color matching functions as shown below, based on measurement of 10 human observers

As you can see the functions are not discrete wavelengths for RGB, but rather weighted averages of (overlapping) wavelengths.

It’s not possible to unambiguously calculate Radiant Exitance from RGB values because many different SPD's may resulting different Radiant Exitance but project to the same RGB. This is called metamerism.


Mis understanding of the Heat Method I have been trying to understand a paper in CG for a while, called the Heat Method by Ken
Many things have clicked but I don't fully understand it yet. In particular.
In the following $u$ is a vector with dimensions equal to the number of vertices in a mesh.
The paper states that solving $(id - t\Delta)u_t = u_0$ is a time discretized approximation of the heat flow on a surface. Rewriting gives: $u_t = u_0 + t\Delta u_t $ or in English, the heat flow at time $t$ is equal to the heat flow at time 0 plus the laplacian of $u$ at time $t$. Which is just a backwards Euler method pretty straightforward.
Then we get that: $(M - tL_C)u = δγ,$ which, to summarize, is the same as above except we have discretized the medium as a triangular mesh.
$M$ is a square diagonal matrix where the entry $m_{i,i}$ is 2 times one third of the sum of the areas of the triangles that contain vertex $i$. In math $m_{i,i} = 2\sum_j A_j / 3$ where $A_j$ are the areas of the triangles containing vertex $i$. And $L_C$ is also a square diagonal matrix where $(L_C u)_i = \sum_j (\cot \alpha_{ij} + \cot \beta_{ij})(u_j-u_i)$ which in short, is an approximation of the laplace operator at vertex $i$.
Thus manipulating the formula: $(id - tM^{-1}L_C)u = M^{-1}δγ,$ Which is the spatially discretized form of: $(id - t\Delta)u_t = u_0$.
So, this makes sense algebraically, but now comes the part I don;t understand.
Why does this work? Why would solving that equation give the correct heat diffusion? Let me explain what I mean. In my head, a geodesic distance (or heat flow, which for the purposes of this method are the same thing) is highly dependent on the shape of a mesh. So in order to know the geodesic distance at vertex $i$ I must first know the geodesic distance at the vertices that come before it relative to the source.
This mehtod however seems to imply that, given an arbitrary mesh and an arbitrary point $p_0$ on that mesh. I can grab any arbitrary point $p_1$ on the mesh and tell you what the geodesic distance from $p_1$ to $p_2$ is without having to look at the full connectivity of the mesh to determine a connectivity graph.
I am not entirely sure if what i am asking is clear. I understand the algebra of the problem, but I cannot link how solving this equation gives you the correct heat flow everywhere on a mesh. Why can you do it in parallel without any regards for the specific connectivity information?

When solving the equation $(M - tL_C)u = \delta_\gamma$, you effectively have to invert the operator:
$$
u = (M - tL_C)^{-1} \delta_\gamma
$$
Note that while the individual operators $M$ and $L_C$ are only local, containing information about the individual vertices and edges of the mesh, the inverse operator is decidedly not local. Inversion is a global operation that incorporates information across the whole matrix being inverted, which means information across the whole mesh. (Note that $M$ has a trivial inverse since it's diagonal, but $L_C$ definitely doesn't.)
In practice, you might not literally compute the inverse matrix but rather use Gaussian elimination or some such to solve the $u$ for a given $\delta_\gamma$. In that case, it's the solving process that integrates information across the whole mesh. Effectively $L_C$ is encoding a bunch of constraints between neighboring vertices, and the solver has to satisfy all the constraints across the whole mesh at once.
(Note that it's not a trivially parallelizable job. There are methods for using parallelism to accelerate solving large linear systems, but they're going to be multi-pass methods that transfer information up and down between different "scales" of the problem. Think like parallel prefix scan, or FFT—that sort of thing.)
BTW, another perspective on this equation is that it is solving for the fixed point of $u$ in
$$
u = M^{-1}(\delta_\gamma + tL_Cu)
$$
This is yet another way of algebraically rewriting that same equation. In this form it looks a lot like the rendering equation $L = L_e + \int L \, f_{\text{brdf}}$! It has a similar structure, where $\delta_\gamma$ is like the emitted radiance and $L_C$ is like the scattering at surfaces. Just as in the rendering equation, you're looking for a global equilibrium solution. (And if rendering was discretized, we could similarly solve it by turning the scattering operator into a matrix and throwing the whole thing into a linear solver.)


Is there any open code to get information about COLLADA or OBJ files? I need a source code that takes as input a COLLADA file and/or an OBJ file, and gives as output basic information about the file, such as file size, model dimension, volume and if possible texture issues.

For OBJ Files

Free tools like MeshLab, Blender, and GLC_Player are able to load OBJ files and give you basic information about their content.

Apparently, GLC_Player can also load COLLADA.

In code you can use the following standard tools:

Assimp - Library for loading many common formats including OBJ
TinyObjLoader - .h/.cc only, no external dependencies


Do we ever consider the size of the pixel to calculate the color in Ray Tracing? From what I'm reading, it looks like a ray tracer views a pixel as one color and computes that color using the ray starting in the center of that pixel. Yet a pixel has a width and a height.

Do we ever use the actual square (now a day pixels are considered squares) that the pixel represents to calculate it true color from all possible source within that entire square?

I am thinking that this is how we come up with colors on edges of shapes in scanline rendering and was wondering whether the same could be applied to ray tracing.

There is a picture which I hope better represents what I am trying to ask here. Should we only consider the pixels to be single points? Or are we to consider pixels as large areas?

Yes, and there are several ways this can apply. First off though, we need to fix a misconception in your diagram. Even if pixels really were square, the volume contributing to the pixel gets wider as it gets further away from the image plane, because of perspective. For a square pixel, the volume would be a square pyramid, whose apex is at the camera. It's not quite a regular square pyramid, because the apex is not in the centre of the square: the outer pixels are very skewed. But pixels aren't square anyway, they're cone-shaped.

First off, some people have experimented with cone tracing, where instead of a single ray, you trace a cone with a half-angle determined from the area of the image covered. It's not very popular, because when the cone intersects the edge of a polygon, you need special handling for the remainder of the cone - the part that wasn't blocked by the polygon. For a square pyramid, it's a little easier, because you can change the shape of the base as parts of the pyramid are blocked, but it's still tricky and slow.

You also don't gain much from this technique compared to just supersampling. Even if you trace rays, you don't just trace a single ray through the centre of each pixel. You trace many rays across the area of the pixel, and use a reconstruction filter to take the shape of the pixel into account. This technique has many possibilities that cone tracing doesn't offer: it works better with volumes, you can jitter your samples to avoid structured artefacts, you can use full path-tracing techniques, etc.

Another area where the pixel size matters is in texture sampling. If you want to get nice texturing, you don't just take a point sample of the texture, you set some filter width according to the area of the texture the sample relates to. This doesn't just make your sampling more efficient by using the right mipmap level: it is required for correct sampling to avoid sampling artefacts. In a GPU fragment shader, you might use the dFdx and dFdy functions to do this. In a ray-tracer, you might have use a cone-tracing style of carrying the filter width with the ray, initially based on the level of supersampling at the camera, but updated whenever a child ray is spawned.


Lambertian shading illuminates all the front of a sphere I've sucessfully implemented a ray tracer from scratch in C and it's marvelous:

The surface normal on this sphere is taken by computing the intersection point, A, and treating this as a euclidean vector with origin, B, the center of the sphere. To normalize it, I first subtracted A with B, yielding C:
C = (Ax-Bx,Ay-By)
From this, it's divided by the magnitude:
N = C/|C|
The illumination is calculated by taking the dot product of N and LIGHT_SOURCE and multiplying it with 255:
if (dot_product<0.0) dot_product = 0.0;
illumination = (unsigned char)(dot_product*255.0);

When I tried setting the light source to (0.0, 0.0, -1.0), it produce something like this:

I expected the shade to be something like it's brighter on the middle and becoming more dimmer on the side.
Anyone know what's causing the problem?
Source code: https://gist.github.com/harieamjari/6297a35400460edbabbd8e39720ed0f5
This can be compiled if you already have libpng16.

I think there is a problem with your ray-sphere intersection.
It should go like this
sphere_normal = ray.origin - sphere_center;
b = dot(sphere_normal , ray.normalized_dir);
c = dot(sphere_normal , sphere_normal ) - sphere_radius*spheres_radius;
float disc = b * b - c;

float t = -1.0f;
if(disc >= 0.0f)
t = -b +- sqrt(disc);

if(t >= 0) // means ray intersects...

Check out scratchapixel to understand how I got this.
https://www.scratchapixel.com/lessons/3d-basic-rendering/minimal-ray-tracer-rendering-simple-shapes/ray-sphere-intersection


Which technique should be used for rendering under water images, Ray tracing or Radiosity? Rendering is the process of computing a three-dimensional geometric description of a scene into a two-dimensional image representation that looks real. Ray-tracing and Radiosity are two of the most famous rendering techniques for photo realistic images, right?

So which technique is suitable for rendering ten under-water images of marine creatures showing total internal reflection needed by ABC Marine Department. The department needs the images to be more natural and can wait for image generation for a month.

Radiosity is only efficient for diffuse to diffuse rendering. Path tracing is more versatile on the type of light transport that could be computed. The main issue with underwater rendering is the caustics generated from specular from the water-air interface. These caustics are very important to generate a realistic underwater image (except for deepwater rendering here the sunlight is completely attenuated).

The problem of rendering caustics from path tracing is the lack of non-efficient sampling strategies to generate a valid path. Density estimation [1] (a.k.a. photon mapping) could be a good candidate for rendering these difficult caustics, but a large scene will require light subpaths guiding to get a reasonable rendering time. Metropolis light transport (MLT) [2,3] provide this guiding automatically and could be used in a density estimation framework [4].

Recently, path guiding techniques [5,6] was proposed to render these difficult paths efficiently with unidirectional path tracing. The simplest approach to use/implement is the technique proposed by Muller et al. [5]. A Mitsuba code plugin is available on GitHub: https://github.com/Tom94/practical-path-guiding

Note that, inside water, there might are volume scattering events [7] to take into account, depending on the amount of particle floating.

[1] "Stochastic progressive photon mapping" Hachisuka et al., 2009

[2] "Metropolis light transport" Veach et al. 1997

[3] "Primary sample space MLT" Kelemen et al. 2002

[4] "Robust Adaptive Photon Tracing using Photon Path Visibility", Hachisuka et al., 2011

[5] "Practical path guiding for efficient light-transport simulation", Muller et al. 2017

[6] "On-line Learning of Parametric Mixture Models for Light Transport Simulation", Vorba et al. 2014

[7] "Monte Carlo Methods for Physically Based Volume Rendering", Novak et al., 2018


Map a texture onto a hyperbolic triangle I've been stuck on how to approach this for a while, so any suggestions would be gratefully appreciated!

I want to map a texture in the form of a lower right euclidean triangle to a hyperbolic triangle on the Poincare Disk.

Here's the texture (the top left triangle of the texture is transparent and unused). You might recognise this as part of Escher's Circle Limit I

Sorry, see the comment as I am not allowed to post more than two links it seems!

And this is what my polygon looks like (it's centred at the origin, which means that two edges are straight lines, however in general all three edges will be circular arcs):

The centre of the polygon is the incentre of the euclidean triangle formed by its vertices and I'm UV mapping the texture using it's incentre, dividing it into the same number of faces as the polygon has and mapping each face onto the corresponding polygon face. However the the result looks like this:

If anybody thinks this is solvable using UV mapping I'd be happy to provide some example code, however I'm beginning to think this might not be possible and I'll have to write my own mapping functions.

SOLVED with some refinement of @Nathan's answer below since the lines AB, AC, BC may actually be arcs not lines.

Method: pick the longest side, say BC, then subdivide this into an even number of parts. Subdivide the other two side into the same number of parts. Then the lines connecting these (DE in the answer below) must actually also be arcs, not straight lines. Subdivide these new arcs as required, add the new triangles as faces then UV map the lower right triangle of the texture to these new faces.

Simple? Not at all. But it worked.

My guess is that to get the texture to look right, you'll have to subdivide the interior of the triangle as well, and approximate the non-linear UV mapping within it.

Currently, it looks like you're subdividing around the edges of the triangle, and forming a fan of smaller triangles between the edge and the incenter. This would be fine if you were just rendering the triangle in a solid color. But when applying a texture, you're getting discontinuities at the boundaries of those subdivisions, because the texture is just stretched linearly across each subdivided triangle and not correctly approximating the hyperbolic mapping along the radial direction.

You'll need to subdivide along both axes, something like this:

and position all the vertices appropriately in screen space and UV space to approximate the hyperbolic coordinates in the interior of the triangle. If you subdivide finely enough, this should produce the illusion of a continuously curving texture mapping.


What is the equivalent of rotation matrix in Euler angle space for Quaternions? I have to load a mesh in Blender (which uses Z and Y as up and forward axes), modify it and export it (with setting up and forward axes to Y and -Z) and import it into another software which uses a coordinate system with Y up and -Z forward. Later on I export the mesh from this software and import it in Blender for rendering. Since I wanted to get the same rotation in the other software that I get in Blender, I changed the rotation matrix of the other software from Identity to the following:

[1.0, 0.0, 0.0;
0.0, 0.0, -1.0;
0.0, 1.0, 0.0]

Now I get the same exact rotation in that software that I get in Blender when using Euler angles. However, I have to actually use Quaternions and I realized that Quaternions do not have a rotation matrix. I wonder, what is the equivalent version of rotation matrix for Quaternions? Also, how can I change that equivalent-rotation-matrix of Quaternions so that I get the same rotation that I get in Blender?

It's not true to say quaternions don't have a rotation matrix. They are both ways of completely describing a rotation in 3D, so they are freely convertible. Each quaternion corresponds to exactly one rotation matrix, and each rotation matrix corresponds to one quaternion and its negative. Quaternions are just more compact and easier to interpolate.

To work out the rotation matrix for a quaternion, you need to take the effect of the quaternion, $q = (w, x, y, z) = (w, \textbf{v}) $ on some point $\textbf{p}$, that is, $(w, \textbf{v}) * (0, \textbf{p}) * (w, \textbf{-v}) $, and express this formula as a matrix operating on $\textbf{p}$.

Multiplying out using the rules of quaternions gives us $(w^2 - \textbf{v.v})\textbf{p} + 2( w\textbf{v}\times \textbf{p} + \textbf{(v.p)v})  $

This can be rearranged into 3 matrix terms - the first part is just a constant value down the diagonal, the second part with the cross product has values everywhere except the diagonal, and the third one is simply $\textbf{v}_m *  \textbf{v}_n $ for rows m and columns n.

Adding these 3 matrices together gives you

$\begin{bmatrix} 1-2(y^2+z^2) &2(xy-zw) & 2(xz+yw)\\2(xy + zw)&1-2(x^2+z^2) & 2(yz-xw)\\2(xz-yw) & 2(yz + xw) & 1 - 2(x^2+y^2)\end{bmatrix}$

which is the conversion you need.

In your particular use case, though, you are just doing a simple rotate-x through 90 degrees, which simplifies things a lot. Remember a quaternion is $ ( cos(\theta/ 2) , sin(\theta/ 2)\textbf{a})$   where $\textbf{a}$  is your normalized axis of rotation. So the quaternion representing ±90 degrees rotation around x will be $( \frac{1}{\sqrt{2}},±\frac{1}{\sqrt{2}},0,0)$


Projected grid water horizon detail I'm trying to implement an ocean scene with C++ and DirectX11. Currently I have a projected grid, Gerstner waves and a basic shading. My problem is that when I aim my camera horizontally, so I can see the water horizon, in the distance, the projected grid becomes insufficient, even at high vertex numbers. These screenshots illustrate the problem:

I know the cause of the problem is in the concept of the projected grid (the grid is detailed near the camera, rough far from it), but there has to be a best practice to solve this.

Any ideas?

I believe a common solution is to split the camera transform used to project the grid from the camera transform that is used to render the grid. At perspectives close to top-down, the two cameras coincide, but as the viewing camera gets close to a horizontal perspective, the projection camera deviates and tries to keep a minimum inclination, i.e. it hovers somewhere above the view camera and looks down slightly.

The tricky bit is making sure that the field of view of the projection camera always covers the region of the scene seen from the render camera. I don't have a resource at hand that details how to compute the appropriate transforms, and it might be tedious to derive by hand.

A different solution is to grab the signal processing toolbox: The artifacts seen in your image are essentially aliasing, caused by insufficient sampling of the wave heightfield by the projected grid. Therefore, one solution is to filter the heightfield appropriately, depending on the projected area of a grid cell. I believe this is used in offline rendering of oceans, and it essentially makes sure that waves at the horizon go flat. However, I'm not sure how feasible this is in real-time rendering, since you would need high-quality anisotropic filtering to make this approach look reasonable.


Artificially increase the region of interest of an image I am solving an image segmentation problem. To increase the accuracy of the model, I came across the following preprocessing step-

First, the set of pixels of the exterior border of the ROI is de-
termined, i.e., pixels that are outside the ROI and are neighbors
(using four-neighborhood) to pixels inside it. Then, each pixel value
of this set is replaced with the mean value of its neighbors (this
time using eight-neighborhood) inside the ROI. Finally, the ROI is
expanded by inclusion of this altered set of pixels. This process is
repeated and can be seen as artificially increasing the ROI.

The image after applying the following algorithm is below-

Can anyone please help me how to approach this problem, I am pretty much clue-less how to do it, how to detect the boundary of the image.My image is of the following type-

Any help in solving this kind of problem is highly appreciated.

It looks like your image will always have the same black area due to the mechanism used to take the picture. If that's the case then you don't need to detect the edges. Instead, you can create a static mask ahead of time. The mask can be a single channel image that's white where the eye image is and black elsewhere. You can use that 1 channel mask as the alpha mask for the RGB image you have.

If the image can move around, it looks to me that you could threshold the red channel and say any red value under 0.1, say, is outside of the eye. Another possibility is just let the user select a circular region for the big circle, and then let them select a smaller one for the bump.

To expand the ROI once you've got the mask, you can go pixel by pixel and if the mask is black, follow the vector from the current pixel towards the center of the image until you hit a non-black pixel. When you hit one, just copy that pixel to the one you started with. Do that with each pixel that's not in the eye portion of the image.

EDIT: Given that the mask is static there's an even easier way. In your other question I pointed out that the center of the larger circle was at approximately (279, 283.5) and its radius was approximately 265. So what you could do is simply iterate over all the pixels in the image and if they're farther than 265 pixels away from the center, replace them with the pixel that is 265 pixels away from the center in the same direction. Something like this untested pseudocode:

float centerX = 279;
float centerY = 283.5;
float radius = 265;
for (int row = 0; row < MAX_ROWS; row++)
{
    for (int col = 0; col < MAX_COLS; col++)
    {
        float deltaX = col - centerX;
        float deltaY = row - centerY;
        float dist = sqrt(deltaX * deltaX + deltaY * deltaY);
        if (dist > radius)
        {
            float dirX = deltaX / dist;
            float dirY = delatY / dist;
            float edgeX = centerX + radius * dirX;
            float edgeY = centerY + radius * dirY;
            SetPixel(col, row, GetPixel(edgeX, edgeY));
        }
    }
}

Note that the smaller circle presents another issue. You can test whether a given pixel is in the big circle as above, and if not, then see what the distance to the smaller circle is. If that distance is smaller than the distance to the larger circle, then get the pixel from the smaller circle rather than the larger one.


Why isn't a BRDF a ratio of radiances? I'm Learning about BRDFs and wondering why the BRDF is defined as the ratio of outgoing radiance to a given direction and incoming irradiance from another direction. Why isn't the BRDF defined as ratio of radiances?

There are  a couple of ways to answer this question: an algebraic way and a geometric way.

Algebraically, we can identify the units that the BRDF must have by looking at its place in the rendering equation. The classic rendering equation is:

$$L_\text{outgoing}(\omega) = L_\text{emitted}(\omega) + \int_\Omega L_\text{incoming}(\omega') \, f_\text{BRDF}(\omega, \omega') \, (n \cdot \omega') \, d\omega'$$

The output value on the left is a radiance, so the result of the integral must also be a radiance. The integrand contains a radiance multiplied by a solid angle $d\omega'$, so something else in the integrand has to cancel out that factor of solid angle. The $n \cdot \omega'$ factor is dimensionless, and the only other thing there is the BRDF—so to make the whole thing work out, the BRDF must have units of inverse solid angle. Equivalently, the BRDF can be seen as a ratio of radiance to irradiance, since they differ by a factor of solid angle in the denominator of radiance.

Another way to see it is that the BRDF plays a role similar to a probability density. If you look at how probability densities work, they have units inverse to the volume of their domain. For instance, a 1D probability density has units of inverse length (probability per unit length, but probability itself is dimensionless), a 2D one has units of inverse area, and so on. The BRDF acts much like a probability density defined on the hemisphere, giving a likelihood for a photon coming in from a given direction to be reflected into some other direction. So, like any other probability density on a spherical domain, it has units of inverse solid angle.

Geometrically, we can get right down to brass tacks and take apart what's going on in the integral in the rendering equation. Recall that an integral means to subdivide the domain into tiny pieces and sum the integrand over all the pieces (in the limit as the pieces get infinitesimally small). Let's look at one such piece. The integrand should result in an infinitesimal amount of radiance $dL$, since we're going to sum over many pieces to arrive at a finite outgoing radiance. So a single infinitesimal piece of the integral looks like:

$$dL = L_\text{incoming} \, f_\text{BRDF} \, (n \cdot \omega') \, d\omega'$$

If we regroup the factors a bit, the combination $L_\text{incoming} \, (n \cdot \omega') \, d\omega'$ calculates the irradiance on the surface due to the light coming from the infinitesimal solid angle $d\omega'$. Since it's arriving from an infinitesimal amount of solid angle, it produces an infinitesimal irradiance $dE$.

$$dL = f_\text{BRDF} \, dE$$

or

$$f_\text{BRDF} = \frac{dL}{dE}$$

So the BRDF acts as a proportionality constant between the infinitesimal irradiance arriving at the surface from an infinitesimal solid angle, and the infinitesimal outgoing radiance generated thereby. It couldn't be a ratio of radiances, because we have a finite incoming radiance, and we need an infinitesimal outgoing radiance if we want to sum up many pieces of the integral and get a finite result. To make that happen, the BRDF would have to be infinitesimal-valued, which...isn't a thing, in standard mathematics. :)

I hope some of this helps. There are a variety of equivalent ways to look at this problem, as with so many things in math and physics.


Convert shader from GLsl 1.50 to 1.30 I'm a newbie with OpenGl and I'm stuck in following the tutorial.
I found this tutorial with these shaders

// Shader sources
const GLchar* vertexSource = R"glsl(
    #version 150 core
    in vec2 position;
    in vec3 color;
    in vec2 texcoord;
    out vec3 Color;
    out vec2 Texcoord;
    void main()
    {
        Color = color;
        Texcoord = texcoord;
        gl_Position = vec4(position, 0.0, 1.0);
    }
)glsl";
const GLchar* fragmentSource = R"glsl(
    #version 150 core
    in vec3 Color;
    in vec2 Texcoord;
    out vec4 outColor;
    uniform sampler2D tex0;
    void main()
    {
        outColor = texture(tex0, Texcoord) * vec4(Color, 1.0);
    }
)glsl";

I noticed that the proposed shaders couldn't compile in my computer, since my GLsl version is 1.30, but changing the version declaration into

#version 130 core

the ouput of this instruction is glGetShaderInfoLog(vertexShader, 512, NULL, infoLog);

  0:2(10): error: illegal text following version number

I wuould like to understand what is wrong in this shader (in version 1.30) or at least where to find enough information to "repair" this situation.
Thanks

#UPDATE 1
I managed to have compiling shaders with no errors, but still the program does not work.

// Shader sources
//in --> attribute
//out --> varying
const GLchar* vertexSource = R"glsl(
    #version 130
    attribute vec2 position;
    attribute vec3 color;
    attribute vec2 texcoord;
    varying vec3 Color;
    varying vec2 Texcoord;
    void main()
    {
        Color = color;
        Texcoord = texcoord;
        gl_Position = vec4(position, 0.0, 1.0);
    }
)glsl";

//in --> varying
//out --> (delete)
const GLchar* fragmentSource = R"glsl(
    #version 130
    varying vec3 Color;
    varying vec2 Texcoord;
    uniform sampler2D tex0;
    void main()
    {
       gl_FragColor = texture2D(tex0, Texcoord) * vec4(Color, 1.0);
    }
)glsl";

I used some functions like this to confirm that shaders are successfully compiled and linked

int success; char infoLog[512];
glGetShaderiv(vertexShader, GL_COMPILE_STATUS, &success);
//can also use GL_LINK_STATUS flag
if(!success)
{
    glGetShaderInfoLog(vertexShader, 512, NULL, infoLog);
    std::cout << "ERROR::SHADER::VERTEX::COMPILATION_FAILED\n" << infoLog << std::endl;
    std::cout <<  glGetString(GL_SHADING_LANGUAGE_VERSION) << std::endl;
    std::cout <<  success << std::endl;
};

There is no such thing as a core profile in GLSL 1.30. The concept of API profiles was introduced with OpenGL 3.2 which happened after GLSL 1.30 was released. The syntax for the version specification in GLSL 1.30 is

#version <number>

Apart from that, I highly recommend that you check to confirm that GLSL 1.30 really is the maximum version you're supposed to be able to use on your system. It seems not unlikely that you just don't have a proper OpenGL driver installed. Even software implementations like MESA support more recent versions than that these days…


FLOP - exact definition I am interested in how much a GPU can actually calculate. When learning about rendering, it seems like a good idea as well.
My Question:
WHAT EXACTLY is a Floating Point Operation?
I know what a floating point number is.
Now, when reading other questions and articles about it, they always say, any simple operation, + - * / , would be a floating point operation, which makes sense in terms of the word itself
BUT:   A multiplication is (obviously) often "far" more (calculating) than an addition, isn't it?!
(Apart from the sheer size of the calculation)
Following, just a simple example for demonstration:
5+5=       5+5
5*5=       5+5+5+5+5
So does the term "flop" actually just describe additions and subtractions
or is a flop in the end just a very rough definition?
Or is the GPU somehow equally fast at all of those calculations?
Thanks in advance.

Just to add to whychmaster's reply, in my experience, on a GPU, a floating-point operation, when used in quoting FLOPS (floating-point operations per second) benchmark figures will refer to Addition (equiv subtraction) and Multiplication.
The other standard IEEE operators, division + sqrt, are more computationally expensive and so unlikely to be included in the figure. (Note that, there is often HW support in GPUs for accelerating operations like 1/x, 1/sqrt(x) etc, perhaps to make them as (or nearly as) fast as muls & adds)
Having said that, it is very likely that a GPU will actually implement a Fused Multipy-Add operation, FMAD, i.e. do  A * B + C in "one step" as this is

more accurate than doing separate multiply and add instructions
is faster than separate ops
is about the same silicon cost
is a very common building block of CG operations (e.g dot products, convolutions)

I suspect that in quoted figures, though, an FMAD would be considered 2 operations.


Weird glColor3f results in framebuffer objects The code is

glColor3f(1, 1, 1);
gluSphere(gluq, 0.2, 10, 10);

glColor3f(1, 0, 0);
glTranslatef(1, 0, 0);
gluSphere(gluq, 0.2, 10, 10);

In short, when I draw directly to screen, it's correct.

But when I create a framebuffer object, attach a texture as color component, draw on the fbo, then draw the texture as rectangle on to the screen, the colors went wrong

It seems like the right sphere at (1,0,0) changed the color of the sphere drawn before it.

And even more bizzare is that if i change the first sphere's color to blue (0,0,1), it becomes black after the second sphere is drawn.

The way I rendered it is like this :

glBindFramebuffer(GL_FRAMEBUFFER, fbo[0]);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
{
    glViewport(0, 0, SCRWIDTH, SCRHEIGHT);
    glMatrixMode(GL_PROJECTION); glLoadIdentity(); gluPerspective(45, (float)SCRWIDTH / (float)SCRHEIGHT, 0.0f, 100.0f);
    glMatrixMode(GL_MODELVIEW); glLoadIdentity(); gluLookAt(0, 0, 5, 0, 0, 0, 0, 1, 0);

    glColor3f(0, 0, 1);
    gluSphere(gluq, 0.2, 10, 10);

    glColor3f(1, 0, 0);
    glTranslatef(1, 0, 0);
    gluSphere(gluq, 0.2, 10, 10);

}
glBindFramebuffer(GL_FRAMEBUFFER, 0);

glBindTexture(GL_TEXTURE_2D, fbo_tex[0]);
glMatrixMode(GL_PROJECTION); glLoadIdentity(); glOrtho(0, SCRWIDTH, SCRHEIGHT, 0, -1, 1);
glMatrixMode(GL_MODELVIEW); glLoadIdentity();
{
    glBegin(GL_QUADS);
    glTexCoord2f(0, 0); glVertex3i(0, 0, 0);
    glTexCoord2f(0, 1); glVertex3i(0, SCRHEIGHT, 0);
    glTexCoord2f(1, 1); glVertex3i(SCRWIDTH, SCRHEIGHT, 0);
    glTexCoord2f(1, 0); glVertex3i(SCRWIDTH, 0, 0);
    glEnd();
}
glBindTexture(GL_TEXTURE_2D, 0);

And I created the fbo and texture like this :

glGenTextures(1, &fbo_tex[0]);

glBindTexture(GL_TEXTURE_2D, fbo_tex[0]);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F, SCRWIDTH, SCRHEIGHT, 0, GL_RGBA, GL_FLOAT, 0);
glBindTexture(GL_TEXTURE_2D, 0);

glGenFramebuffers(1, &fbo[0]);
glBindFramebuffer(GL_FRAMEBUFFER, fbo[0]);
glFramebufferTexture(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, fbo_tex[0], 0);
glBindFramebuffer(GL_FRAMEBUFFER, 0);

If feels like that consequtive glColor3f caused the colors to mix, taking the minimum RGB values of previous used colors. So white becomes red, but blue turns to black. And the first sphere has the color written to the texture, then the second sphere somehow picks the mixed color and overwrite the first one's pixels. How can I fix this?

You need to set the colour back to white before drawing the framebuffer to the screen. Otherwise, the whole framebuffer gets multiplied with red.

glBindFramebuffer(GL_FRAMEBUFFER, 0);

glColor3f(1, 1, 1); // <------- add this (doesn't have to be exactly here)

glBindTexture(GL_TEXTURE_2D, fbo_tex[0]);
glMatrixMode(GL_PROJECTION); glLoadIdentity(); glOrtho(0, SCRWIDTH, SCRHEIGHT, 0, -1, 1);
glMatrixMode(GL_MODELVIEW); glLoadIdentity();
{
    glBegin(GL_QUADS);
    glTexCoord2f(0, 0); glVertex3i(0, 0, 0);
    glTexCoord2f(0, 1); glVertex3i(0, SCRHEIGHT, 0);
    glTexCoord2f(1, 1); glVertex3i(SCRWIDTH, SCRHEIGHT, 0);
    glTexCoord2f(1, 0); glVertex3i(SCRWIDTH, 0, 0);
    glEnd();
}
glBindTexture(GL_TEXTURE_2D, 0);


How do I include all extensions my system supports using GLAD? When using the GLAD web service, I am allowed to pick any version of OpenGL and any extensions I want.

If I pick the latest version of OpenGL and "all extensions", will the header file returned by the Web Service #define extensions on my machine that I do not actually have or does it just define those that exist?

In other words, if I include the GL_ARB_SHADER_CLOCK extension, on GLAD, does it get defined even if not available on the machine I use?

I don't see how it could be selectively defining extensions because the script is generated on a server ahead of time but I want to make sure that I am not trying to use unsupported extensions.

In this github issue the GLAD author offers the following way to correctly check if there is an extension is available at runtime:

  The defines just allow you to check at compile time which extensions
  (and opengl versions) are potentially available. After
  loading/initilaizing glad you can check with GLAD_
  if the extension is available at runtime (= supported by the driver
  and loaded).

// [initialize glad] // ...
if (!GLAD_GL_EXT_memory_object) {
    die("GL_EXT_memory_object not supported!");
}


Is using many texture maps bad for caching? If I have many textures (say 5+ maps) bound to the same texture unit, does it work worse for cache than if I had only 2 or 3 textures?

Just adding to imallett's answer, it is true that increasing the number of accesses to different texture data in a shader will increase pressure on the GPU cache(s), but there are several other factors that can significantly influence the effect. It's also possibly complicated by the fact that, like CPU caches, there may be several layers of cache in a GPU, ie. Texture Unit <= L0 <= L1 <= ..Memory

Avoid aliasing -> Use MIP maps

If you have a scene that has minification of texture data, be it due to perspective or simple scaling and you aren't using MIP mapping, then you will get aliasing. This is not just going to produce visual artefacts; it is very likely to be a performance problem.

As soon as you get aliasing, the address accesses to the texture will become incoherent which will not only end up thrashing the caches but introduce a lot of DRAM "page breaks" (more correctly, row breaks) which can be costly. MIP mapping helps reduce the incoherency.

Texture compression

Perhaps a bit of an obvious option, but if you can use texture compression (e.g. DXTn|ETC*|PVRTC*|etc) targeting from 8bpp down to, say, 2bpp, you can greatly increase the effectiveness of the memory bandwidth/cache by factors of 4x through to 16x.
Now I can't speak for all GPUs, but some texture compression schemes (e.g. those listed above) are so simple to decode in hardware, that the data could stay compressed throughout the entire cache hierarchy and only be decompressed in the texture unit, thus effectively multiplying the size of those caches.

Data size

Obviously, some data, e.g. render targets used as texture data in subsequent renders, can't employ texture compression. Whenever you can, use the smallest pixel format that will do the job, i.e, if 32/16bpp (A)RGB will do, don't use 4x32 float formats!

Sparse access

This is somewhat related to the aliasing example above, but we've seen cases where large render targets are created, but then only very sparsely sampled. Cache lines, be it in CPUs or GPUs, are quite long so if you are only using one pixel in each cache line, you will be wasting transfers.

Also, WRT compressed textures, these achieve compression by effectively sharing data between a local region of texels. If you don't have coherent access then, apart from the memory footprint reduction, the compression probably won't help.

Dependent Texture Reads

Not so much of a cache issue (well, unless the computed accesses are quite incoherent), but texture accesses that aren't directly defined by the UV coordinates supplied with vertices might be slower than those that are directly defined.

Tiled/Morton VS Strided textures

Although I suspect most textures these days will be stored in either a tiled or Morton-like (aka Twiddled/Swizzled) order (or even a combination of both), some textures might still be in scan-line order, which means that rotation of the texture is likely to lead to a significant number of cache misses/page breaks. Unfortunately, I don't really know how to spot if a particular format is arranged in such a way.

(For background reading, try Blinn's The Truth About Texture Mapping. FWIW, taking that a few steps further led to the use of Twiddled-order (i.e. Morton order) textures in at least some early PC hardware).


Benchmark GLSL fragment shader How can I benchmark a fragment shader?
What my shader does is basically lots of floating point operations, since it's doing some procedural stuff.

A lot goes in to benching performance. I think most would agree that measuring the frame time of your application is the best metric you can gather about how well you're doing at run-time. An easy way to "bench" your shaders in your engine/environment is to simply record frame times for a fixed duration with a fixed view and fixed scene. Then save those frame times out to a CSV and plot them in something like Excel. This should give you a good idea of how much your frame time is fluctuating and if you're actually hitting your frame time targets most of the time.

However if you want to see if your shaders are properly utilizing the GPU you may want to dig into your application with something like AMD's GPU Perf Studio or Nvidia's NSight tool. I'm more familiar with Perf Studio but it can give you a static shader analysis to see if you're actually taking full advantage of the GPU or if you're overloading certain registers.


PBR : Understanding importance sampling I have read this article
On PBR  and in it we have NDF function defined as follows

Then I have read this article on random vector generation
Here are what I feel are the important bits.

Now jump to Importance sampling here is the function

Xi is the random vector returned from Hammersley function.
Notice how I have left the theory on top of the function. He clearly says "We combine the NDF with the Hammersley equation in spherical space"
This is my question. How did he do that? I have read the epic games real shading pdf and they don't provide any explanation whatsoever

All I need to understand is how did they derive the cosTheta and sinTheta terms.
Any explanation even a one liner as an comment would be immensely appreciated.
Thank you.

Leaving the actual derivation of mathematics aside, I'll try to give a general description of how can we combine the NDF with importance sampling in spherical coordinates.
What is NDF ?
The problem is the following. Intiutively, we need the NDF for "zooming in" from the realm of macrosurfaces to that of microfacets.
Mathematically it is needed for the following equivalence:
$$dA_h = NDF(h) dw_{h} A$$.

$A$: flat macrosurface patch

$dw_{h}$: is the area of a portion of a unit spherical cap defined by the orientation $h$

$dA_{h}$: is the total area that is being approximated by the orientation $h$

see here for more.
These terms are understood better when we considere the following equivalence:
$$d\Omega = 4\pi (\frac{dS}{A})(\hat{r} \cdot \hat{n})$$
You can read this equivalence as the following:
One can approximate a solid angle subtended by a flat surface whose area equals $dS$, orientation $\hat{n}$, and distance from the viewer $r$ where $\hat{r} = \frac{\vec{r}}{r}$ and $A = 4\pi r^2$. The $\vec{r}$ is a position vector of the surface area $dS$ with respect to the center of the sphere. You can read more about it in wikipedia
The point here is that whenever we consider approximating solid angles with projected surfaces, we need that direction and distance term.
Whether we consider solid angles over microfacets or macrosurfaces don't make a difference. From this perspective, you can think of $NDF$ as "regulation" term for two different granularities involved in approximating the same structure.
Since we are modeling the same structure from two different granularities, we can't "touch" properties that coincides with physical/geometric properties of the structure, but we can "touch" the regulation term, hence different ways of computing ndf are provided by different researchers.
Whatever it maybe though, it has to satisfy the following equivalence for unit sphere:
$$1=\frac{1}{A}\int_{S} (n \cdot h) dA_{h} = \int_{\Omega} NDF(h) (n \cdot h) dw_{h}$$
Where S is the domain of the area of the macrosurface patch. This reads simply as the following: The area we are calculating as a spherical cap using solid angle bits, and the area we are calculating as a macrosurface patch using macro surface patch bits has to be the same.

What is Importance Sampling ?
Here is the second bit of information necessary to understand the procedure.
Let X be a random variable, ie a function whose signature is $X: M \to R $, where $M$ is the sample space and $R$ is the set of real numbers. Expected value $E[X]$ is given under the probability distribution $P$ as the following $E[X;P] = \frac{1}{n} \sum_{i=1}^n x_i$ where n is the total number of samples drawn from the distribution and $x_i$ is sample generated from $X$ using the distribution P.
The precision of the estimate calculated this way depends on the variance of the random variable $X$ under the probability distribution $P$. If we have infinite number of samples, then its okay to use low precision estimates, but we never do. In fact, it would be great if we have a small number of samples, with a more precise way of computing the estimate.
Now think of the following, we have some random variable X, that outputs values according to some distribution, and we want to understand how X works under that distribution. Wouldn't it be better to select the input values for X so that the graph made by the output values look more like the actual graph of the X under distribution P. This is what we are trying to do with importance sampling. The key here is to understand that though we may not know how X maps values in its domain to its codomain, we may know how the probability distribution for choosing samples works. Hence if we choose a function that is similar to random variable's actual probability distribution, we can significantly reduce variance.

How they interact ?
Now coming to your question. We have seen that in order to use importance sampling we need to use function that resembles to the actual probability distribution used for sampling the spherical cap area. So if our function is $f=NDF(h) (n \cdot h)$, we need use a function that is similar to the actual probability distribution of this function to sample values from it in order to get an estimate with higher precision.
Since we are dealing with points on unit sphere, we are dealing with spherical coordinates. The vectors associated with these points are also in spherical coordinates. Now spherical coordinates are defined using two angles and a distance to the center of the sphere. In unit sphere the distance is 1. This leaves us with
the two angles when considering vectors.
The probability distribution associated with $f$ then has two parameters, $\theta$ and $\phi$ due to using spherical coordinates.
Now the probability density function (pdf) of $f$ is given in here. From that we calculate its cumulative distribution function (cdf) which is also given in here.
We need the cdf because of the following property: If $T$ is a continuous random variable with cumulative distribution function $cdf(T)$, then the random variable $Y = cdf(T)$ has a uniform distribution on [0, 1]. Meaning that the inverse of $cdf(T)$ whose input would come from a uniform distribution on [0, 1] would output a value according to the probability distribution of $T$. More on that on wikipedia as suggested by peter.
Coming back to our point. We are looking for function closer to the actual probability distribution of $f$. Why ? Because it is the actual probability distribution that "causes" $f$ to output $x_i$ (sample generated in accordance with the probability distribution of random variable as stated in importance sampling section), since the probability distribution of $f$ takes two parameters, the angles of spherical coordinate, we need to adapt pdf and cdf to deal with two parameters.
It turns out, we can sample one of the angles uniformly, hence
phi = X_i.x * 2 * PI given that X_i.x is a random number between [0,1].
Then we only need to deal with pdf and cdf of a single parameter.
If we can find out manually the inverse of cdf of that parameter, we can map a random value between [0, 1] to a value according to correct distribution of that parameter. The piece of code that deal with X_i.y represents that manually found inverse of cdf.
You can read about the involved mathematics from:

Here

Here. This gives the full derivation

and Here


Sub-pixel antialiasing rules I recently had an issue with sub-pixel anti-aliasing of text, which produces very harsh colours and made me wonder how it is meant to be done properly:

I've drawn some cases of a black tile covering thirds of pixels below.

The colours match the ones I'm seeing, however when I look at properly antialiased text, the result is not nearly as bright and distracting:

I assume there has to be a balance between a good light intensity and the right colour. What methods are used for sub-pixel anti-aliasing that give such good results?

Update:

I've rendered a white teapot to an image with 3x width and with multisampling. Below I compare averaging every 3 pixels with assigning each to RGB. The colours still seem overly bright in some cases (especially compared to the example above from here), not that my phone captures them well.

OK, so my monitor needs a little dusting

I'm not sure what happens there. Looking at the rendering of "considerations" in your example, the "cons" looks like what you get when you do crude sub pixel anti-aliasing, without applying the necessary "fix up" to make it look good. But then the "der" in the same word looks like there's no sub-pixel anti-aliasing.

You can find a good article about how good looking sub-pixel anti-aliasing works here https://www.grc.com/ct/ctwhat.htm and for the "fix up" part here https://www.grc.com/ct/freeandclear.htm and here https://www.grc.com/ct/cttech.htm

What it comes down to is: if you just compute the R/G/B intensities by calculating the covered area for each sub-pixel, the resulting color fringes are very ugly/irritating. To fix it you have to apply a horizontal blur/low-pass filter to the sub-pixel image. The result is an image that's less sharp but also has far less saturated colors. I.e. something like your "Check you" example.


DLT algorithm (4+ points) not working, what's missing? I'm doing a direct implementation of DLT algorithm but I can't figure out why, at the end the matrix H I get doesn't (even remotely) produce x2 = H*x1

The algorithm is not that complicated so I can't see what I'm missing at this stage.

%%point matching section (manual for now)

% point selection

[originalpoints1, normpoints1, translation1, scaling1 ]= pointprocessing(rootimage, false);
[originalpoints2, normpoints2, translation2, scaling2 ]= pointprocessing(sideimage, true);

if(length (originalpoints1) ~= length (originalpoints2))
    error('points must be referenced in both images');
end

%% DLT transform application

% getting A
i = 1;
A = zeros(2*length(originalpoints1), 9);
while(i <= length(normpoints1))
    x1 = normpoints1(1,i);
    y1 = normpoints1(2,i);
    w1 = normpoints1(3,i);

    x2 = normpoints2(1,i);
    y2 = normpoints2(2,i);
    w2 = normpoints2(3,i);

    a1 = - w2 * (normpoints1(:,i)');
    a2 =   y2 * (normpoints1(:,i)');
    a3 =   w2 * (normpoints1(:,i)');
    a4 = - x2 * (normpoints1(:,i)');
    z = zeros(1,3);

    a = [z a1 a2; a3 z a4];

    A(2*i-1, :) = a(1, :);

    A(2*i, :) = a(2, :);

    i = i + 1;
end

% getting H

[U D V] = svd(A);

h = V(:, end);

normH = reshape(h, 3, 3)';

% denormalization

T1 = scaling1 * translation1;

T2 = scaling2 * translation2;

H = inv(T2) * normH * T1;

The normalizing process works as intended - I've double checked, performs the translation and scaling (none on Z) as intended. But for clarity sake here that one also:

function [ originalpoints, normpoints, translation, scaling ] = pointprocessing( image, flag )

% point selection

figure(1);
imshow(image);
axis on;
if (~flag)
    title('select n >= 4 reference points, double click last');
else
    title('mark the same reference points (same order), double click last');
end

[x, y] = getpts();

close(1);

if(length (x) < 4 || length (y) < 4)
    error('selected too few points');
end

% coordinate conversion  - centroid calculation

center_x = mean(x);

center_y = mean(y);

originalpoints  = [x'; y'; ones(1, length (x))];

% coordinate conversion - apply the translation

translation = [1 0 -center_x; 0 1 -center_y; 0 0 1];

translatedpoints = translation * originalpoints;

% coordinate conversion - apply isotropic scaling

goal = sqrt(2);

heuristic = meandistance(translatedpoints);

if(heuristic < goal)
    error('average distance between points too (1 pixel) small? double check')
end

scaling_factor = (length(translatedpoints) * goal) / adaptedmeandistance(translatedpoints);

scaling = [scaling_factor 0 0; 0 scaling_factor 0; 0 0 1];

normpoints = scaling * translatedpoints;

end

function [sum] = adaptedmeandistance(point_matrix)

%assumes > 3 points
i = 1;
sum = 0;
while (i <= length(point_matrix))
    sum = sum + abs(realsqrt( point_matrix(1,i)^2 + point_matrix(2,i)^2));
    i = i + 1;
end
end

function [distance] = meandistance(point_matrix)

%assumes > 3 points
i = 1;
sum = 0;
while (i <= length(point_matrix))
    sum = sum + euclidian(point_matrix(1,i), point_matrix(2,i), 0, 0);
    i = i + 1;
end
distance = sum / length(point_matrix);

end

function [distance] = euclidian(point1_x, point1_y, point2_x, point2_y)

    distance = abs(realsqrt( (point1_x - point2_x)^2 + (point1_y - point2_y)^2));

end

The issue is that I get, for instance, an H of

0.4787    0.0051  186.8102
0.0702    0.4762  144.1276
0.0000    0.0000    0.4514

Transforming originalpoints1

   1.0e+03 *

    0.1465    2.3465    2.3465    0.1985
    0.1545    0.1345    3.0545    3.0945
    0.0010    0.0010    0.0010    0.0010

into something,

   1.0e+03 *

    0.2577    1.3108    1.3257    0.2976
    0.2280    0.3729    1.7634    1.6316
    0.0005    0.0006    0.0007    0.0006

that's not originalpoints2

   1.0e+03 *

    0.5545    2.2865    1.9265    0.5105
    0.4905    0.6505    2.5625    2.7985
    0.0010    0.0010    0.0010    0.0010

The problem might be on the scale, should I manually enforce the scale of each point to 1? Doesn't that mean there's something wrong with H?

Just realized that this is an equation involving homogeneous vectors; thus the 3-vectors should not equal, they have the same direction but may differ in magnitude by a nonzero scale factor.


How do I swap the buffers when using a Win32 window with Vulkan? I was recently told here how to manage the integration between Vulkan and Win32.

I am writing a windowing library like "GLFW", that is only going to support Vulkan; I have now come to the point where I need to implement SwapBuffers(Window) type of function; I have tried to reverse engineer Glfw's glfwSwapBuffers(GLFWWindow*) function to see how they implemented it in their framework, but I wasn't able to understand it fully.

How do I "swap the buffers" in a Win32 window?

P.S: The library is written in C

I have now come to the point where I need to implement SwapBuffers(Window) type of function

Then stop doing that. You need that function in OpenGL because the windowing code is explicitly not part of OpenGL, so to write cross-platform GL code, you need an encapsulation layer.

That is not the case for Vulkan. While you still need a layer to create the window, handling the display, swap-chain images, and so forth is all part of the Vulkan API. The user's code should have complete and total control over this stuff; your abstraction layer need not be involved.

Indeed, since Vulkan has no concept of a "default framebuffer", there's no way for you to abstract away presentable images without user support. Just do what GLFW3 does in its Vulkan support: provide a cross-platform way for the user to create a Vulkan surface.


Some questions about working in 0-255 integers instead of single precision floating point colors I have many objects that store their color values. At the moment I'm storing them as vec4 values, that is four 4-byte values for RGBA, mainly because this is how the shader reads them. I was thinking of storing these colors as a vector or 4 unsigned single-byte integers. Anyway, with storing color values as one byte integers I can have a combination of 255^3, which is 16,581,375.

I have a couple of question about this.

If I store the red color as 128, this ends up being a floating point value of about 0.502, and a color of 129 ends up being a floating point value of about 0.506. My question is whether there is any discernible difference to speak of, either consciously or unconsciously between the intensity of a color being 0.502 or 0.506, or shades in between. Does this depend on the limitations of your graphics adaptor or video display? Or is there simply no discernible difference that one could tell with the eye for a value in between those two?

Further, I have a similar inquiry about things like color changes over time. If for example we use a 0 to 255 value to do a fade to black over a significant period of time, would one integer jumps on the 0-255 scale in terms of brightness, which would result in about 0.004 in floating point on each change, be detectable as opposed to calculating the change in floating point to begin with?

First, a quick correction:

  storing color values as one byte integers I can have a combination of 255^3

You mean $256^3$. There are 256 eight-bit integers (including zero), not 255. But here comes the more fundamental thing you need to understand.

  If I store the red color as 128, this ends up being a floating point value of about 0.502

It doesn't end up being a floating-point value at all: it ends up as an 8 bit (or possibly 10 bit) integer on your display. Whatever extra precision you had in the floats to start with is thrown away when the final colour goes into the framebuffer. So the rest of what I'm going to say shouldn't influence your choice of what to store, because everything will end up as 8 bit colour channels.

Now that doesn't mean there's never any value in using a higher-precision number earlier on. More precision in the intermediate data means you'll get the full precision in your output format. e.g. if you start out with these normalized integers, and you multiply them all by 2 in the shader, you'll only end up with the even-numbered integers in your output: you lost half of your precision.

As for the question you really asked, about visible differences, it's easy to test that in live conditions. Just make a grey colour gradient from $(0, 0, 0)$ to $(1, 1, 1)$ in whatever representation you like (floats or normalized integers) and show it in place of your real computation. If you see Mach bands on your gradient, then yes, you can tell the difference between adjacent intensities. In general, for adults with normal sight and 8 bit-per-channel RGB in common lighting conditions, there is a visible difference in a grey gradient. That doesn't mean there's a visible difference between every adjacent pair: the eye is less sensitive to contrast in red and blue than in green, and less sensitive to changes in hue, so it strongly depends on the colours involved.

However, there are techniques to make the differences invisible most of the time. Check out my answer explaining what sRGB is for on SO for some examples.


MTLTexture getBytes returning blank image Inside of my application I am trying to convert a MTLTexture into a CGImage or CIimage for use in other parts of the app. I am having a difficult time doing so because none of the example code I find seems to work properly.

For some reason something breaks on each and every one of them. Here are my attempts and why they dont work. If you could give me ideas on why that would be appreciated.

All of my textures are in brga8Unorm format.

Method 1 is to copy the pixels into an unsafe mutable pointer and use that to directly create a CGTexture. It creates a textures of all 0s

func toImage() -> NSImage? {
    let texture = self
    let width = texture.width
    let height = texture.height
    let bytesPerRow = width * 4
    let data = UnsafeMutableRawPointer.allocate(bytes: bytesPerRow * height, alignedTo: 4)
    defer {
        data.deallocate(bytes: bytesPerRow * height, alignedTo: 4)
    }

    let region = MTLRegionMake2D(0, 0, width, height)
    texture.getBytes(data, bytesPerRow: bytesPerRow, from: region, mipmapLevel: 0)
    var buffer = vImage_Buffer(data: data, height: UInt(height), width: UInt(width), rowBytes: bytesPerRow)

    let map: [UInt8] = [2, 1, 0, 3]
    vImagePermuteChannels_ARGB8888(&buffer, &buffer, map, 0)

    guard let colorSpace = CGColorSpace(name: CGColorSpace.genericRGBLinear) else { return nil }
    guard let context = CGContext(data: data, width: width, height: height, bitsPerComponent: 8, bytesPerRow: bytesPerRow,
            space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue) else { return nil }
    guard let cgImage = context.makeImage() else { return nil }

    return NSImage(cgImage: cgImage, size: NSSize(width: width, height: height))
}

Method 2 tries to use CImage to do the job however CImage reports Cannot render image (with an input Metal texture) using a metal context. and returns nil

extension MTLTexture {
    func toImage() -> CGImage? {
        let context = CIContext()
        let texture = self
        let cImg = CIImage(mtlTexture: texture, options: nil)!
        let cgImg = context.createCGImage(cImg, from: cImg.extent)!
        return cgImg
    }
}

This last method doesnt even try to convert the image it is just trying to determine if the image is blank (all 0s) or not however even on fully green or even textured images it says they are blank.

func anythingHere(_ texture: MTLTexture) -> Bool {
    let width = texture.width
    let height = texture.height
    let bytesPerRow = width * 4

    let data = UnsafeMutableRawPointer.allocate(bytes: bytesPerRow * height, alignedTo: 4)
    defer {
        data.deallocate(bytes: bytesPerRow * height, alignedTo: 4)
    }

    let region = MTLRegionMake2D(0, 0, width, height)
    texture.getBytes(data, bytesPerRow: bytesPerRow, from: region, mipmapLevel: 0)
    var bind = data.assumingMemoryBound(to: UInt8.self)

    var sum:UInt8 = 0;
    for i in 0..<width*height {
        sum += bind.pointee
        bind = bind.advanced(by: 1)
    }
    return sum != 0
}

Clearly I am doing something fundamentally wrong as none of these methods are actually producing results. What am I doing wrong? What is the right way to do this?

maybe you forget to call MTLBlitCommandEncoder.synchronizeTexture to sync VRAM to system memory.

please check:
https://forums.developer.apple.com/thread/30488

and document:
https://developer.apple.com/documentation/metal/mtlblitcommandencoder/1400757-synchronize


Colorblending shader recompilation scenarios Context
As I was trying to learn more about Vulkan, I learned that certain fixed-function steps could be implemented with shader code from this video (at around the spot where he talks about the following problem):
https://youtu.be/X8Ob_b3ANzs?t=24m
In this video, the presenter says "hitching" can occur in OpenGL because for some GPUs the colorblend stage apparently needs to be recompiled at certain times. He gives an example of when a player takes out a torch and causes hitching due to bloom.
Questions
Why would this shader need to be recompiled in this scenario? In what scenarios would this shader need to be recompiled? Is this common practice/an issue across vendors?

In GL's model of the GPU, the colour blending step is performed by special-purpose hardware. This idea dates back to the fixed-function origins of GL, before programmable shaders were even a thing.

But not all GPUs today implement all colour blend options with special-purpose hardware. The hardware vendor can reduce gate count and power use by getting rid of some of the less-used blend modes (or even by getting rid of the blender hardware completely and doing everything in shaders). Instead, the GL driver has to add some code to the shader to do the blending. This code is added transparently to the driver, and isn't visible to the application, because GL doesn't have a way to make this visible or to control it from the application.

Now, the shader object (the compiled shader) doesn't just depend on the shader source code, but also on the blend mode in effect when it was compiled. If the application subsequently changes the blend mode, the driver needs to work out which shader objects are affected, retrieve the stored source of those shader objects, add the code for the new blend mode, and recompile the shader. Again, it has to do this behind the application's back, because GL doesn't have a way to control it: GL assumes the shader object is independent of the blend mode.

Normally, the application would compile shaders during a loading screen, or in a thread, but because the shader compilation isn't triggered by the application, it can't control when it happens. It just happens by magic when the application changed something that should be unrelated (the blend mode), which might be in the middle of a frame, so it causes hitching. And of course because it's hidden, the app can't simply ask to go back to the old state: changing the blend mode again might trigger another shader compilation (though it might come from a cache in the shader compiler).

Blend mode isn't the only parameter that causes this. Some unusual texture modes are also supported by adding code to the shader. (YUV textures commonly work this way, on implementations that support them.) Exactly which parameters cause a problem depends on the implementation: it's different for different hardware architectures. I think it's typically a bigger problem on mobile, because the GPU vendors are more sensitive to gate count and power use and more keen to replace special-purpose hardware with shader code.

The reason Vulkan makes this less of a problem is that it creates a new kind of state, which depends on the blend mode, the shaders, and other parameters. Changing any of these requires the app to perform an explicit "prepare" operation, which might recompile shaders. This gives the app control over when it happens, including the ability to prepare all the combinations of parameters up-front in a loading screen or in a background thread. It means that shader compilation never happens without the app asking for it explicitly. And it means the app can just go back to the old state by switching to it, instead of changing the blend mode and waiting for another shader recompilation.


A way to save 360° panorama video What is the best way to save video from faces of a cube? The render can produce 6 squares (90°x90°) of 1024x1024 resolution. Faces oriented along axies and only position is changed continuously. I want to save them into the video stream and then play back them into 360° panorama viewver by means of projection on the cube or using cubemap (OpenGL ES).

What is the best codec suitable and what is the best way to save video for above purposes? Is it better to use six separate synchronized video streams or, instead, to enclose all six faces into the one?

If cube is unfolded into six square bitmaps and composed into a 3x2 (or 2x3) rectangle, then it is possible to get artifacts on glued inner edges and on the outer edge of the resulting rectangle. I sure it may depend on codec used to compress/decompress resulting video.

In case of 6 separate video streams it is possible to render only 5 of 6 (or 3 of 6 in case of only pitch and yaw (but not roll) rotations are possible) faces of cube, i.e. only visible ones. But on the other hand it may be waste of resources to support six separate streams due to necessity of attendant metainformation handling.

What are expected performance issues inherent to different approaches?

The relevant platform is average home desktop PC.

I would be tempted to use a solution which involves decoding a single rectangular video stream so you don't need any special video decoder. As you have said there will be performance penalties for decoding multiple seperate streams.

Something like octohedran cube maps might be a good offline way to convert 6 video streams into a single video stream while minimizing distortion. A relatively cheap pixel shader can be used to project the 'cube' map back to camera space.

Further reading here > http://www.vis.uni-stuttgart.de/~engelhts/paper/vmvOctaMaps.pdf