Worsening sampling quality of raytracing with increasing distance from world origin I am using RT shader (DXR) to generate g-buffers with code below. There was no problems with smaller scene (pink room from cwyman tutorial - http://cwyman.org/code/dxrTutors/tutors/Tutor5/tutorial05.md.html ). However, when I moved to bigger scene (Sun Temple by Epic Games - https://developer.nvidia.com/ue4-sun-temple ), I've started noticing problems with texture sampling. Please compare textures at the end of this post. The greater the distance to world origin (0, 0, 0), the more jaggy artifacts appear and quality becomes worse.
At the beginning, I was suspecting acceleration structures. But skybox quality is becoming worse, so it's probably unrelated to AS.
inline void GenerateCameraRay(uint2 index, uint2 dimensions, float4x4 projectionToWorld, inout  float3 origin, out float3 direction)
{
    float2 xy = index + float2(0.5f, 0.5f); // center in the middle of the pixel.
    float2 screenPos = (xy / (float2) dimensions) * 2.0 - 1.0;

    // Invert Y for DirectX-style coordinates.
    screenPos.y = -screenPos.y;

    // Unproject the pixel coordinate into a ray.
    float4 world = mul(float4(screenPos, 0, 1), projectionToWorld);

    world.xyz /= world.w;
    direction = normalize(world.xyz - origin);
}

[shader("raygeneration")]
void RayGen()
{
    float3 rayDir;
    float3 origin = g_sceneCB.cameraPosition.xyz;

    // Generate a ray for a camera pixel corresponding to an index from the dispatched 2D grid.
    GenerateCameraRay(DispatchRaysIndex().xy, DispatchRaysDimensions().xy, g_sceneCB.projectionToWorld, origin, rayDir);

    RayDesc ray = { origin, 1e-4f, rayDir, 1e+38f };
    RayPayload payload = { 0 };
    TraceRay(SceneBVH, RAY_FLAG_NONE, 0xFF, 0, 1, 0, ray, payload);
}

[shader("miss")]
void Miss(inout RayPayload payload)
{
    float3 rayDir = WorldRayDirection();
    rayDir.z = -rayDir.z;

    RTOutputAlbedo[DispatchRaysIndex().xy] = float4(skyboxTexture.SampleLevel(g_sampler, rayDir, 0).rgb, 0);
    RTOutputNormal[DispatchRaysIndex().xy] = float4(0, 0, 0, 0);
    RTOutputSpecRoughness[DispatchRaysIndex().xy] = float4(0, 0, 0, 0);
}

Problem was that perspective matrix had very short Z_NEAR and Z_FAR values (0.01f and 200.0f). By changing it to much higher range (10.0f, 20000.0f), problem with jaggies and artifacts disappears.


What is the effect of the BRDF denominator? I was reading a paper on BRDF. I've come across this formula :
$$ f(\omega_i, \omega_o ) = \frac{FDG}{4(N.V)(N.L)}$$

The (N.L) term can be cancelled by the cosine term which appears in the rendering equation :

$$ L_o = \int f(\omega_i, \omega_o) L_i cos(\theta) d\omega_i $$

What about the (N.V) term ? What happens if (N.V) = 0 ?

Firstly, I highly suggest reading Eric Heitz's paper "Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs", which covers the full derivation of microfacet-based BRDFs.

The $\frac{1}{4(N \cdot V)(N \cdot L)}$ term is a side effect of the derivation of the BRDF for specular microfacets. Specifically, it comes from the Jacobian of the reflection transformation. See the paper and/or Walter et. al's 2007 paper for more details.

As for your concern for a divide by zero, the definition of the rendering equation prevents it. Let me explain:

For $(N \cdot V)$ to equal zero, they must be orthogonal. In this case, the visibility term of the rendering equation will cull the cases at / below the horizon.


Does WebGL 2 support linear depth texture sampling? With WebGL 1 we can use WEBGL_draw_buffers extension which lets us use a texture as the depth attachment on the frame buffer.

And we can chose to use LINEAR or NEAREST as the filtering method.

gl.getExtension('WEBGL_draw_buffers');
gl.texImage2D(gl.TEXTURE_2D, 0, gl.DEPTH_COMPONENT, width, height, 0, gl.DEPTH_COMPONENT, gl.UNSIGNED_SHORT, null);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);

WebGL 2 doesn't require an extension for this, but as soon as I switched to WebGL 2 I noticed that nothing happens unless I use NEAREST as the filtering method.

gl.texImage2D(gl.TEXTURE_2D, 0, gl.DEPTH_COMPONENT24, width, height, 0, gl.DEPTH_COMPONENT, gl.UNSIGNED_INT, null);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR); // wont work
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR); // has to be NEAREST

Is there something I have to do to make this possible?

The only workaround for this that I know of would be to draw this depth texture onto another texture, but that seems like a waste of time.

The spec says depth textures are not filterable.

  3.8.13

  ...

  Using the preceding definitions, a texture is complete unless any of the following conditions hold true:

  The effective internal format specified for the texture arrays is a sized internal depth or depth and stencil format (see table 3.14), the value of TEXTURE_COMPARE_MODE is NONE, and either the magnification filter is not NEAREST or the minification filter is neither NEAREST nor NEAREST_-
  MIPMAP_NEAREST


Fast antialiased line drawing Bresenham's line algorithm is a way of drawing straight lines using only fast integer operations (addition, subtraction, and multiplication by 2).  However, it generates aliased lines.  Is there a similarly fast way to draw antialiased lines?

Is there a similarly fast way to draw antialiased lines?

No, because by definition an anti-aliased line touches more pixels.  Such algorithms will be slower.

In a software rasterizer, the ubiquitous way to draw anti-aliased lines is Xiaolin Wu's line algorithm.  It's not hard to implement, and anyway there's unusually high-quality pseudocode at that link.

In a hardware raster pipe, the line primitive is expanded to a screen-space quad by the default (or user-provided) geometry shader, and then drawn as two triangles, which can then be anti-aliased in the usual ways.

In a raytracer, there are a variety of options.  It's worth thinking about how you actually want to draw a 1D object.  Maybe as a cylinder (woo shadows!).  Note that this introduces issues of perspective/foreshortening which may (or may not) be what you want.  There isn't a clear generalization.  Then, obviously, whatever you do, you just supersample it.


Sensor size of Sony HDR-HC7e I want to motion track a shot from my Sony HDR-HC7e camcorder and I am not quite sure about the horizontal sensor size(width).
I set the focal length to minimum and the recording to HDV 16:9
This manual (HC7e and HC7 are basically the same) states it is a 1/2.9" sensor with a diagonal of 6.3mm (page 101). But if I do my calculations and multiply (1/2.9) with 25.4 to convert it to mm I get 8.75862069?
The camcorder outputs a 1440*1080 interlaced .m2t file. The image in this file is squeezed by a factor of 1.33. as seen in this example.
So I suppose that the sensor has an aspect ratio of 4/3(1.33) So I did some math:

$$a^2 = b^2 +c^2$$
$$
\begin{matrix}
a_1 &=& 6.3\\
a_2 &=& 8.75862069\\
b &=& x\\
c &=& 1.33x\\
\end{matrix}
$$

$$6.3^2 = x^2 + (1.33x)^2$$
$$
\begin{matrix}
x &=& 3.78605\\
1.33x &=& 5.0354465\\
\textrm{horizontal 1} &=& 5.0354465
\end{matrix}
$$

$$8.75862069^2 = x^2 + (1.33x)^2$$
$$
\begin{matrix}
x &=& 5.26359\\
1.33x &=& 7.0005747\\
\textrm{horizontal 2} &=& 7.0005747\\
\end{matrix}
$$
So I thought because there always is a relation between sensor size and focal length, I just take the sensor size horizontal 35mm and the corresponding 35mm focal length equivalent of 49mm (manual page 101) and get the ration: 35 / 49 = 0.714285714 then multiply the true focal length of 5.4mm by that and get a horizontal sensor size of 3.857142856
So now I have got 3 possible horizontal sensor sizes. Now I want to calibrate my lens with a set of images like the linked one. Should I leave them with the 1.33 aspect ratio and where would I specify that in the OpenCV camera calibration or do I unsqueze them and then calibrate? And do I even need the sensor size and focal length for OpenCV camera calibration?

Answer over in Photography:
https://photo.stackexchange.com/questions/119761/sensor-size-of-sony-hdr-hc7e/119764#119764
It turns out to be Vertikal: 3.74 horizontal 4.98 and diagonal 6.23


Why is $pdf(\Psi) = \frac{1}{2\pi}$ when picking from hemisphere uniformly? I'm trying to figure out why the probability density function for picking a direction uniformly on the hemisphere is $\frac{1}{2\pi}$.

Something tells me this is related to the number of steradians in a hemisphere being $2\pi$, but I can't see how this is connected. We could come up with another measure for solid angle ("blibs" for example) where the number of blibs in the hemisphere would be $8000\pi$, but I'd expect the probability for picking a direction uniformly to be unchanged.

EDIT: I understand this now. My confusion arose because I wasn't thinking clearly about the units of the probability density function, i.e. $\frac{1}{sr}$. I got thrown off by thinking of area, and confusingly thought the pdf units were inverse area (or something)!

It's like this because the surface area of a unit sphere is $4\pi$. As ratchet freak points out, the integral of a probability distribution over its domain has to be 1. Put another way, the probability of choosing some direction is 1. The surface area of the hemisphere is $2\pi$. The constant that you integrate over a $2\pi$ domain to get 1 is $\frac{1}{2\pi}$.

The connection to steradians is that 1 steradian is defined as the solid angle that subtends a surface of area 1 on a unit sphere. That is, a unit sphere has $4\pi$ steradians and a surface area of $4\pi$.

It's just the same in two dimensions. A radian is chosen so that a 1 radian arc of a unit circle has length 1, so there are $2\pi$ radians in a circle, and $\pi$ radians in a semicircle. If you choose 2D directions uniformly on a semicircle, the probability distribution is $\frac{1}{\pi}$.


Calculate aspect ratio from 2D shape in 3D space Given the 4 coordinates of a 2D shape in a 3D space I want to calculate its aspect ratio.

The 3D space is created with 2 vanishing points.

The 4 coordinates - marked blue - are the 2D coordinates on the display.
in the example they should be roughly (14, 5.5), (19, 5), (20.3, 7.3), (25.3, 6).

I'm not sure if this is possible at all, if someone could find proof that for 2 different aspect ratios the 2D coordinates are the same this problem would be unsolvable.

My exapmle:

The ratio is with a quick and dirty visual measurement $665:501$ which is approximately $5:4$. You can measure it by taking the ratio of the vanishing angles $\alpha/\beta$ (see picture 1) because we are so close to the center.

Image 1: Ratio of the inbound angles

We can check the situation visually by drawing a 2 point perspective grid. For this we need the center line between the vanishing points.

Image 2: Seems about right.


UV Coordinate Interpolation in Compute Shader I am trying to get the screen space position of each pixel in a texture on a mesh and write it into a texture. I was hoping to use a compute shader to do this:

#pragma kernel CSMain

struct VertexData
{
    float4 v_point;
    float2 uv;
    float4x4 MVP;
};

RWTexture2D<float4> Result;
RWStructuredBuffer<VertexData> vBuf;

[numthreads(16,1,1)]
void CSMain (uint3 id : SV_DispatchThreadID)
{
    // one thread per pixel in texture

    //Get pixels position in relation to vertices in UV space, using models UV layout

    //multiply pixels object space position against MVP matrix to bring it into clip space and do some stuff to it

    //finally, output this pixels colour back into the texture, storing screen space related data.
}

In the above code (lump of comments) I can tell that I am going to run into some problems. The VertexData struct contains mesh data per vertex, but I want to run a thread for each pixel in a texture and then figure out what my pixels position is on a triangle. In a regular shader, this would be interpolated from the vertex shader when using data in the fragment shader, but from my little knowledge of compute shaders, this would require something extra. In the back of my mind I feel like its going to end up being something to do with Barycentric coords, but I just can't put my finger on it!

A picture is worth a thousand words:

The desired end result is a texture mapped to the UV coords of a mesh, storing the screen space positions of each pixel on the mesh. If there is a better way of tackling this, please let me know!

You seem to do things more complicatedly than you need to, as adressed in the comments. What you actually want to do is simple the opposite of the normal render flow. Rather than drawing your mesh in world space and mapping the texture into it, you draw it in texture space and map its vertices onto it and let your graphics hardware worry about interpolation. I'll give the general outline here, as I honestly don't know the actual DirectX terms for the stuff here, but it should be easy to map onto real HLSL if you know your way around DirectX (which I'm sure you do when messing with Compute Shaders).

You basically render into your output texture using an offscreen render target. You then render your mesh using its texture coordinates as positions rather than its model space vertices, which you just pass through to the pixel shader, where you finally transform them into screen space and write out as the pixel's colour:

[Vertex Shader]

PER VERTEX float2 uv;
PER VERTEX float4 v_point;

TO PS float4 modelPos;

void VSMain() {
    CLIP_POSITION = uv;
    modelPos = v_point;
}

[Pixel Shader]

CONSTANT float4x4 MVP;
CONSTANT float2 viewportSize;

FROM VS float4 modelPos;

void PSMain() {
    float4 clipPos = MVP * modelPos;
    float3 ndcPos = clipPos.xyz / clipPos.w;
    float3 screenPos = float3(viewportSize * (screePos.xy * 0.5 + float2(0.5)), ndcPos.z);
    OUT_COLOR = screenPos;
}

Feel free to adjust this to the realities of proper DirectX screen-space transformations and matrix multiplication orders, but the general idea should be clear.


Tiled Image Palettization Algorithm? In most cases today, image palettization consists of reducing the overall number of colors in an image to some fixed number globally. This is typically solved as a straightforward but computationally difficult clustering problem.

A more difficult version of this problem involves older graphics hardware where the palette is broken down into N subpalettes (e.g. of 4 colors each) and the image is broken down into XxY-pixel tiles where each tile can only use colors from a single subpalette and N << the number of tiles. Here both the exact colors, the arrangement of colors into subpalettes, and the subpalette for each tile of the image are all unknowns that must be solved together.

How could this more difficult problem be solved, algorithmically?

An interesting problem. I've done a bit of work in texture compression and this sounds something like a generalisation of Campbell et al's "Color Cell Compression".  It's also a little like a feature we were asked to include in the Dreamcast VQ compressor so that sub-palettes could be swapped to create different colour schemes on textures.

I was thus thinking that maybe you could adapt a standard VQ algorithm, applied in stages, to the problem.

You say that

  "the exact colors, the arrangement of colors into subpalettes, and the subpalette for each tile of the image are all unknowns"

but can we assume the X*Y tile size is predetermined?

Assuming that is the case, perhaps the following might be worth trying:

1) Guess an initial palette size for the entire image, perhaps,
 $$Psize=(M * SizeOfEachSubpalette)$$
where $1 < M < N$.

Use standard VQ quantisation to generate this palette and map the pixels to it.

2) Now consider each $X*Y$ tile to be a "super pixel" with $Psize$ channels. Set each channel value in the tile's "super pixel" to the number of pixels that mapped to the corresponding colour in our initial palette.

For example, if in our initial palette, entry $j$ is [0x11,0x21,0x31] and 5 pixels in the tile mapped to that colour, set the $j'th$ component in the tile's superpixel to 5.

Run a second VQ operation on the set of all "super pixels" using '$Psize$-dimensional space' to generate $N$ "palette" entries. (Actually, you don't need the 'palette' values - the VQ process is just meant as a means of partitioning the tiles into $N$ sets)

Hopefully, this will now identify which tiles should share the same sub-palette.

3) For each tile set, collect all the pixels in the tiles, and apply a 3rd VQ operation to identify the $SizeOfEachSubpalette$ colours needed for all tiles in that set.

I would think some sort of clever GLA/k-means process might be needed for step 2 to juggle the assignments of tiles to sub-palettes. The choice of M might also be interesting.


Gaussian blur with transparency I made a convolution testing program for experimenting with stuff. I've created a 5x5 Gaussian kernel for blurring. Everything looked fine, then I tried to blur a transparent png with a white circle in the middle. On the blurred image, the edge looks like a gradient from white to black.
This is mathematically correct, because the transparent points are treated as black, transparent pixels, so the convolution should gradually give darker colors near the edge.
But if I do a gaussian blur on the same image e.g. in GIMP, the circle will be fully white, with gradually more transparency toward the edges. I'd like to get this effect. What am I missing?

If I set the transparent pixels to transparent white, it'll work, but only if the blurred color is white.

This is the original image:

This is the result I get if I left the transparent pixels on default:

And if I set the transparent pixels to #00FFFFFF, I'll get this (this is the intended IF the blurred color is white)

I apologise for only skimming through your question/answer and the links you gave but I thought it would be a good idea to post what needs to be done to correctly do any blending and/or filtering on images with an alpha channel.
The first thing to note is that all calculations should be done in a linear colour space, but the image data in PNGs and JPEGs is most likely to be in sRGB which is non-linear. You should convert this data to a linear form before doing any calculations. For example, the average of black (0,0,0) and white (255,255,255) in sRGB is not (127,127,127) but (off the top of my head) something like (186,186,186). The filtering of images, especially those with neighbouring, complementary, saturated colours (e.g. Yellow&Blue), can look quite odd if you don't take gamma into account.
The second part of this theory dates back to Porter and Duff's seminal 1984 paper, "Compositing Digital Images", which introduces the concept of pre-multiplied alpha. Pre-multiplied alpha basically means that you multiply the RGBs by the alpha value before doing calculations.
The Porter-Duff paper states...

There are a few reasons to dismiss this proposal,
the most severe being that all compositing operations
will involve multiplying the 1 in the red channel by
the .5 in the alpha channel to compute the red contribution
of this object at this pixel. The desire to avoid this
multiplication points up a better solution, storing the
pre-multiplied value in the color component, so that
(.5,0,0,.5) will indicate a full red object half covering a
pixel.

... which suggests this was initially done just to reduce computation costs, but in Jim Blinn's article "Compositing-Theory", he demonstrates that the correct way to do any filtering or compositing operations is to first convert everything to premultipled.
Once you have linearised and then pre-multiplied your pixel data, do the filtering/blending and, finally, undo the pre-multiply and reapply the gamma function to map back to sRGB.


What is the algorithm used for phong shading normal interpolation? Let's suppose we have an object consisting of only 3d points, and triangle faces that each take a subset of these 3d points. How can I interpolate the normal vectors to get that Phong smooth shading? The actual algorithm seems impossible to find yet everybody is doing it.

There is one post on here, but I still have no idea how to implement it: Normal Interpolation for Phong shading

Also, it seems wrong to me that this supposed formula only takes in account two normals. What if you have a triangle surrounded by 3 other triangles? Then how do you blend the normals?

EDIT: I misunderstood your question, so please disregard this answer.

The Phong shading model assumes a surface normal $\mathbf{n}$ at the shading point $\mathbf{x}$ but it is independent of its type, so either the shading normal or the geometric normal will work. Moreover, this reflectance model requires an incoming and outgoing directions. In that sense, you can't just "interpolate the normals" to get the desired effect since you need to take into account the positions of the camera and the light, as well as the roughness parameters.

The (normalized) Phong BRDF is a sum of a diffuse and specular term:

$$
f_r(\mathbf{x},\omega_i,\omega_o) = \frac{\rho_d}{\pi} + \rho_s \frac{n+2}{2\pi}\max(0,\cos^n\alpha),
$$
where

$\rho_d$ is the diffuse albedo, i.e. the fraction of the incoming energy that is reflected diffusely,
$\rho_s$ is the specular albedo, i.e. the fraction of the incoming energy that is reflected specularly,
$n$ is the Phong exponent; higher values yield more mirror-like specular reflection, and
$\alpha$ is the angle between the perfect specular reflection direction $\omega_r$ and the outgoing direction $\omega_o$.

You can compute $\omega_r$ from the surface normal $\mathbf{n}$ and the lighting direction $\omega_i$, which is just

$$
\omega_r = 2\mathbf{n}(\mathbf{n}\cdot\omega_i) - \omega_i.
$$

Typically you pick the albedos such that $\rho_d + \rho_s \leqslant 1$ to conserve energy.


The variance for the Monte Carlo estimator I was wondering about the variance for Monte Carlo.
The book (Advanced Global Illumination 2ed, p.60) writes the equation as follows:

$\sigma^2 = \frac{1}{N} \int(\frac{f(x)}{p(x)} - I)^2p(x)dx$

$\space\space\space\space=\color{red}{\frac{1}{N} \int(\frac{f(x)}{p(x)})^2p(x)dx  -I^2}$

$\space\space\space\space=\frac{1}{N} \int\frac{f(x)^2}{p(x)}dx  -I^2$

However, the equation appears incorrect. I think the red area is wrong and the equation should be changed as follows:

$\sigma^2 = \frac{1}{N} \int(\frac{f(x)}{p(x)} - I)^2p(x)dx$

$\space\space\space\space=\color{red}{\frac{1}{N} \int((\frac{f(x)}{p(x)})^2 -2\frac{f(x)}{p(x)}I + I^2)p(x)dx}$

$\space\space\space\space=\frac{1}{N} \int\frac{f(x)^2}{p(x)} - 2f(x)I + I^2p(x)dx$

$\space\space\space\space=\frac{1}{N} \int\frac{f(x)^2}{p(x)} - 2f(x)I dx \space + \frac{1}{N}I^2$

$\space\space\space\space=\frac{1}{N} \int\frac{f(x)^2}{p(x)} dx - \frac{2}{N}I^2 \space + \frac{1}{N}I^2$

$\space\space\space\space=\frac{1}{N} \int\frac{f(x)^2}{p(x)} dx - \frac{1}{N}I^2$

Is it right?

I don't have that book to check the context of this, but from the equations you posted, yes, it looks like you're right. The $1/N$ factor should be applied to both terms. That agrees with the formula for variance from statistics, which is $E[X^2] - E[X]^2$.


NURBS topology classification In what sense is it true that a NURBS surface can only have the topology of a plane, cylinder or torus?

For example I can do a NURBS sphere.

Is the sphere homeomorphic to one of the above surfaces?

Before we begin, let us differentiate between two things:

The shape of a sphere,
and the topology of a sphere.

A NURBS surface can make the shape of a sphere. In a typical configuration, it will be 'open' at the poles. That is the mathematical function of the surface does not wrap over the pole in that it is not a true sphere (it has the shape of a sphere). Topologically the sphere is in this typical configuration a cylinder.

It is possible to leave the other direction open too in which case it is topologically a plane. Several CAD applications choose this approach. A torus is also possible if you allow a thin no volume sliver at the center of your sphere.

Image 1: Turning cylinder to sphere. Note this is not a topological sphere it is still a cylinder as the top is open (even though infinitesimally small).

  Is the sphere homeomorphic to one of the above surfaces?

No, but you can still have a spherical shape even if it does not satisfy the topology condition of mathematics.

Why only 3 topological families?

Simply, a NURBS surface has only 4 possible configurations of wrapping around the parameter space:

It does not wrap at all. Topology: Plane.
It wraps around the U direction. Topology: Cylinder.
It wraps around V direction. Same as above. Topology: Cylinder.
It wraps around both U and V. Topology: Torus.

A wrap is always periodic so it goes from - direction to + direction. It can not arbitrarily connect (on a mathematical level).


How can you interpolate over an array of say 5 colors? I have an array of color values. I have an interpolating parameter, t, which varies from 0 to 1.

I can't for the life of me figure out how to smoothly interpolate between these color values based on t, such that I get a smooth blended color for any t value.

Any ideas? Its probably something simple that I am blanking out on..

Say you have colors

$[c_1, c_2, c_3, c_4, c_5]$

And $t$ values at which each color should be purely displayed.

$[t_1, t_2, t_3, t_4, t_5]$

Now your problem is given $t$ which color do I have to display?

Find the t-values $t_i$ and $t_{(i+1)}$ between which $t$ lies.
Calculate a 0 to 1 ratio where $t$ lies between them

$ param = (t - t_i) / (t_{(i+1)} - t_i) $

Interpolate the color via

$ c = param * c_i + (1-param) * c_{(i+1)}$

If I haven't messed up something in my mind, this should work for any piecwise linear function, and not only if your $t_i$ values lie between 0 and 1.


Why map Hammersley 2D set's (u,v) to sphere's (θ, φ) coordinates (and not to (φ, θ) )? As explained in Holger Dammetz page Hammersley Points on the Hemisphere, the 2D coordinates (u,v) are usually mapped to the spherical coordinates (θ, φ).

As a result, the coordinate "uniformly distributed" - u in the 2D case - is the polar angle θ. This looks counter-intuitive to me, I would have mapped regularly spaced u values to regularly spaced azimuth φ values. To avoid "empty areas" like in this example:

which is generated by the mapping of the following 2D set:

What is the reason not to map (u, v) to (φ, θ) ?

You can of course, as you suggested, map (u, v) to (φ, θ). Unfortunately, it does not solve the problem for 5 points:

I've changed Holger Dammertz' code a bit (switched u and v), and you see that the problem still persists. For a higher number of points, it really doesn't seem to make a (visual) difference.

The Hammersley point set is a way to very quickly determine points on a sphere that have good, but don't need to have a perfect distribution. In the problematic case of very few points, Hammersley might just not be the tool for the job. Pre-calculating and storing points sounds like a better option to me.


Artifacts when gamma correcting Doing post-process gamma correction is showing texture artifacts, here is a small example on a skybox:

The first image is gamma correcting on the skybox shader and seems correct, the lower is doing it in post-process and has terrible artifacts near the edges of the clouds, however both versions do it the same usual way:

Color.rgb = pow(Color.rgb, vec3(1.0/2.2));
Color.a = 1.0;

So that makes me believe there's something wrong with the way I'm creating and using frame buffers, here's sort of how I'm doing it:

// Generate FirstFBO, specify this as the color attachment
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, FramebufferSize.x, FramebufferSize.y, 0, GL_RGB, GL_UNSIGNED_BYTE, nullptr);

// ...

while (true) {
    glBindFramebuffer(GL_FRAMEBUFFER, FirstFBO.ID);
    DrawStuff();

    glBindFramebuffer(GL_FRAMEBUFFER, 0);
    glBindTexture(GL_TEXTURE_2D, FirstFBO.ColorTexture);
    DrawFullscreenQuad();
}

Short answer, set the precision of the image to a higher value.

Long answer,

When looking at a gamma correction curve, you can see that the lower values get changed much more, this means that the difference between lower values will get greater and that causes this effect. You have a limited amount of values for a color channel and this means that when it rounds the number down to the 8 bit value, it makes some neighboring pixels the same color. You don't notice this in a normal image, but after gamma correcting it this effect comes out, because you lose that bit of change in the color that later on can become a much bigger difference in color.

To fix this, you just give the gamma correction more precision to also get that small color change it would've else rounded away.

In the actual shader, it has floating point color information and that means that it has more precision and thus not the weird artifact.

So when making the frame buffer, you can use for example GL_RGBA16F or GL_RGBA32F. This would increase the precision and remove the artifact.

I hope that this can help you fix the problem, good luck and have a great day!


Inverse-Transpose of Vertex Normals For the past month, I have been learning on my own and writing a little graphics library and am currently working on the transformation of vertex normals to soon support lighting/shading. I understand how if I transform the vertices by a matrix M that I need to apply the inverse-transpose of M (i.e., (M^-1 )^T ) to the surface normals. However, I am struggling to understand how this "rule" can be applied to all of the basic transformations (translate, scale, rotate) ... particularly, translation.

Using a homogeneous (w) coordinate in a 4D vector class, a Translate(x, y, z) would have no effect on a normal since its w-coordinate is 0, indicating that it is a vector, not a point. However, when we transpose the inverse of the Translate then it would have an effect on the normal (which feels incorrect).

For example, let's say that we have created a 3D triangle and want to apply a basic Translate(1, 2, 3) to the three vertices' positions. Based upon the "rule" for transforming vertex normals (i.e., inverse-transpose), we should be multiplying the three vertices' normals by Transpose(Translate(-1, -2, -3)). However, this feels incorrect since a vertex normal should remain unaffected by Translation and only be affected by Rotation and Scaling.

Any help in understanding how the inverse-transpose can be applied to vertex normals would be greatly appreciated!

You are correct, when they are talking about taking the inverse-transpose they are talking about the 3x3 part of the affine transformation leaving the translation since it doesn't affect the normal vector.

Check : Normal / Transforming Normals | Wikipedia

About how or why is it the inverse transpose, a simple explanation, that is also given in the book "Essential Mathematics for Games" is

Suppose we have a Point $P$ and a normal vector $n$ both represented by column matrix. Then we know that
$$
n^T P = 0
$$
This is the same as the dot product.
Now if we apply a transformation matrix $M$ to point $P$, we'll have to apply some sort of transformation matrix $Q$ to $n$ also.
$$
\begin{eqnarray}
(Q n)^T M P & = & 0 \\
n^T Q^T M P & = & 0
\end{eqnarray}
$$

One solution is
$$
\begin{eqnarray}
Q^T M & = & I \\
Q & = & {M^{-1}}^T
\end{eqnarray}
$$
Hence the inverse transpose. This stands for any sort of transformation matrix $M$ be it scaling, rotation etc. Also for pure rotation matrix, since they are orthogonal, the matrix transforming the normals is the same as that of the vertices. as ${M^T}^T = M$


OpenGL Framebuffer with multiple Depthbuffers inside I am trying to put multiple depthbuffers into one Framebuffer.
I want to use VR and render both eyes at the same time:
that means, in the geometry stage I want to clone the incoming triangle to two outgoing triangles with a shift of the eye offset for layered rendering. Therefore I would like to have two RGB textures / maybe one RGB texture 2d array and two depthbuffers / a depthbuffer array
here is my program code so far, the error occurred at the second last line of code and it is a GL_INVALID_ENUM error.
GLError();
glGenFramebuffers(1, &m_sFrameBufferInfo.m_nFrameBuffer);
GLError();
glBindFramebuffer(GL_FRAMEBUFFER, m_sFrameBufferInfo.m_nFrameBuffer);

//Left eye
GLuint textureLeft;
glGenTextures(1, &textureLeft);
GLError();
glBindTexture(GL_TEXTURE_2D, textureLeft);
GLError();

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, m_nWindowWidth, m_nWindowHeight, 0, GL_RGB, GL_UNSIGNED_BYTE, NULL);
GLError();

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
GLError();
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
GLError();
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureLeft, 0);
GLError();

//right eye
GLuint textureRight;
glGenTextures(1, &textureRight);
GLError();
glBindTexture(GL_TEXTURE_2D, textureRight);
GLError();

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, m_nWindowWidth, m_nWindowHeight, 0, GL_RGB, GL_UNSIGNED_BYTE, NULL);
GLError();

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
GLError();
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
GLError();
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_TEXTURE_2D, textureRight, 0);
GLError();

//depth buffer
GLuint textureDepth;

glGenTextures(1, &textureDepth);
GLError();
glBindTexture(GL_TEXTURE_2D_ARRAY, textureDepth);
GLError();

glTexImage3D(GL_TEXTURE_2D_ARRAY, 0, GL_DEPTH24_STENCIL8, m_nWindowWidth, m_nWindowHeight,2 , 0, GL_DEPTH_STENCIL, GL_UNSIGNED_INT_24_8, NULL);
GLError();

glFramebufferTexture3D(GL_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_TEXTURE_2D_ARRAY, textureDepth, 2, 0);
GLError();

glBindFramebuffer(GL_FRAMEBUFFER, 0);
GLError();

Thanks for your help!

FBOs can only use a single depth image, but that image can be layered. Broadly speaking, VR multi-eye rendering ought to be done through layered FBO images for both the color and depth buffers.
The problem with your code is that you can't use glFramebufferTexture3D to attach a layered image. You must use glFramebufferTexture.
Also, you need to use layers for the color image as well.


In what scenarios and how does GL_ARB_buffer_storage help performance? The Dolphin emulator for GameCube/Wii has the ability to use the ARB_buffer_storage (or EXT_buffer_storage for GLES) to improve rendering performance.

From the extension's description, a GPU driver implementation provides a means to allocate storage that can be marked as immutable, i.e. cannot be deallocated or resized while in use. What is required of the driver implementation for this to actually be useful? At one extreme, a naive implementation could allocate immutable memory buffers exactly the same way it allocates typical global memory buffers, functionally correct but not beneficial for performance.

At least one user has suggested buffer_storage only matters in the case of discrete GPUs which carry their own dedicated RAM storage.

On the other hand, the Dolphin team continually requested buffer_storage for Mali's driver over the years, which suggests there is a performance improvement even in SoCs with integrated graphics and no dedicated GPU DRAM. If so, is the benefit at all comparable and how does the driver achieve it?

Does Vulkan have an equivalent or somehow avoid the need for such an extension?

The intent here is to understand the value in implementing EXT_buffer_storage in the driver for Broadcom's VideoCore VI.

[1] "Dolphin on the pi 4"

[2] "Will there be EXT_buffer_storage at all?"

[3] "EXT_buffer_storage is now supported by ARM"

[4] "
Possible to add GLES EXT_buffer_storage driver support to Videocore VI on the Pi4?"

The marquee feature of buffer storage is not immutability of the allocation itself, but instead is a feature you couldn't have without immutable allocation: persistently mapped buffers.

Pre-buffer_storage, you could not use a buffer while it was mapped. This is done to allow implementations the freedom to play games with mapping behind your back. For example, it's legal for glMapBufferRange to not actually return a mapped pointer; it could just return some CPU-accessible memory which, at unmap time, will get copied into the actual buffer. To permit such freedoms, mapping and unmapping a buffer is something you have to do semi-frequently.

Furthermore, pre-buffer_storage, it is 100% legal to call glBufferData (note the lack of Sub) on that buffer again. This will cause the old data storage to be discarded and new storage allocated. So you couldn't keep a buffer mapped if you tried to reallocate its storage.

Also, implementations couldn't really know where to allocate storage for GPUs with multiple kinds of memory, so they would tend to move the buffer's allocation around based on how you use it. If you frequently upload to it, they'll eventually put it into CPU accessible memory to speed up such operations. And so forth. But this is all done after allocating storage, and it's done based on how you use the buffer. So having the buffer be mapped would prevent such shuffling around of data.

Making a buffer's storage immutable solves these problems. You cannot reallocate an immutable buffer's storage, so there's no problem with keeping it mapped forever. glBufferStorage takes usage flags which unlike glBufferData's hints, explicitly forbid you from using the memory in certain ways. So the implementation knows exactly where to put the storage and will not need to move it around after the fact. So there's no problem with persistently mapped storage.

And I imagine that, for a console emulator, having persistently mapped storage would be very useful. After all, programs written for such consoles touch memory directly at any time, the same memory that the GPU can address. So storing it in a buffer and keeping that memory mapped all the time probably works out better than uploading pieces of it, or trying to figure out which pieces should be uploaded, or whatever.


How to deform some mesh so that it fits along a spline curve? Given a set of vertices, transforming them to fit onto some spline curve.
For example 3D modelling software has extrude along curve and Unreal Engine has a spline mesh component that takes some mesh and stretches it along a defined curve.

Is there some branch of mathematics or theory that looks into this?

I have no knowledge of the literature on the topic, but I did something very similar to what you're asking some time ago: I wanted to generate lathe meshes and bend them according to a spline. I think the same technique could be adapted to your case quite easily.

First you would need to define what your default axis is: if the input mesh corresponds to the case when the spline curve is a straight line, where is that line in 3D? It could be defined arbitrarily as the Y axis, the PCA of the mesh, or some manually defined axis.

The algorithm is then:

for each vertex v
{
    matrix4 M = ComputeTransformFromAxisToSpline(v);
    v = ApplyTransform(v, M);
}

The tricky part is obviously to define a transform $M$. You could do something like:

matrix4 ComputeTransformFromAxisToSpline(vector3 v)
{
    float h = GetPositionOnAxis(v, originalAxis);
    vector3 p1 = GetPointOnSpline(h);
    vector3 p2 = GetPointOnSpline(h + dh); // dh is a small delta
    vector3 u = normalize(p2 - p1);

    // Define an orthogonal basis
    vector3 v = v0;
    vector3 w = normalize(cross(u, v));
    vector3 v = normalize(cross(w, u));

    return MakeTransform(u, v, w, p1) * MakeTranslation(originalAxis, -h);
}

$v_0$ is going to be the problem. I have tried two approaches:

Use an absolute value, like an axis orthogonal to the default axis. This will work only if the spline never gets parallel to that axis. It may also lead to undesired shapes.
Start with an absolute value for the points at $h=0$, then reuse the previous $v$ for bigger $h$ values. This means you have to treat vertices ordered by $h$. In my case this was trivial because I was generating the vertices, but in your case that would probably mean to sort them. The result is a mesh that will tend to twist in curves, which may or may not be desirable.

Another possibility would be to define the orientation by having more information in the curve, but I haven't tried that approach as I was already satisfied with the result.


smallpt's early clamping Kevin Beason's smallpt estimates the pixel radiance by accumulating $2\times2$ subpixel radiance estimates using the following expression:

c[i] = c[i] + Vec(clamp(r.x),clamp(r.y),clamp(r.z))*.25;

Here, c[i] represents the radiance estimate of the pixel with flattened index i, r represents the radiance estimate of one of its four subpixels. As one notices a clamp operation is used to clamp the radiance estimate of each subpixel separately to the $[0,1]^3$ range before accumulation. But this seems wrong from an unbiased path tracing perspective, since this effectively introduces bias to the radiance estimate of each pixel. So I wonder if I am missing something (e.g., clever trick at the cost of bias), given that Kevin Beason's smallpt is already around and known for nearly ten years?

Apparently, Kevin Beason also added a presentation by David Cline explaining every line. The aforementioned expression is explained as follows:

  Add the gamma-corrected subpixel color estimate to the Pixel color
  c[i].

This seems, however, completely unrelated since no gamma correction is involved yet at this stage in the code. And since exponential functions are non-linear transformations. (Obviously clamp does not involve any exponential functions at all. ;-) )

Reference code:

for (int y=0; y<h; y++){                       // Loop over image rows
    fprintf(stderr,"\rRendering (%d spp) %5.2f%%",samps*4,100.*y/(h-1));
    for (unsigned short x=0, Xi[3]={0,0,y*y*y}; x<w; x++)   // Loop cols
        for (int sy=0, i=(h-y-1)*w+x; sy<2; sy++)     // 2x2 subpixel rows
            for (int sx=0; sx<2; sx++, r=Vec()){        // 2x2 subpixel cols
                for (int s=0; s<samps; s++){
                    double r1=2*erand48(Xi), dx=r1<1 ? sqrt(r1)-1: 1-sqrt(2-r1);
                    double r2=2*erand48(Xi), dy=r2<1 ? sqrt(r2)-1: 1-sqrt(2-r2);
                    Vec d = cx*( ( (sx+.5 + dx)/2 + x)/w - .5) +
                            cy*( ( (sy+.5 + dy)/2 + y)/h - .5) + cam.d;
                    r = r + radiance(Ray(cam.o+d*140,d.norm()),0,Xi)*(1./samps);
                } // Camera rays are pushed ^^^^^ forward to start in interior
                c[i] = c[i] + Vec(clamp(r.x),clamp(r.y),clamp(r.z))*.25;
            }
}

This seems, however, completely unrelated since no gamma correction is involved yet at this stage in the code.

The values are indeed not gamma corrected. Earlier in the presentation (On page 32) you have the toInt function which does the gamma correction. This function is called when the image is written to a file.

Then the clamping itself.
If we have a list of 5 samples.

0.1; 0.4; 0.2; 0.8; 0.6

It averages to 0.42. However, if one samples becomes very bright.

0.1; 0.4; 8.0; 0.8; 0.6

The average would become 1.98. That is a lot brighter. Because we take random samples, it could be that these two averages are neighboring pixels. This would cause a few pixels in the entire image to be much brighter than their neighbors. This is called a firefly. These fireflies are often not wanted, as they stick out in the image and make it look a lot more noisy. Therefore we want to try and remove these fireflies.

An example image of this can be found in blender's cycles documentation here. The bright pixels are fireflies and are unwanted.

There are three ways to remove a firefly.

Add more samples. By adding more samples, you get a truer image and that one bright sample would end up with less influence. This does however take longer to render.
Some sort of denoising algorithm. The algorithm could find these fireflies and remove them. However, it will almost always change other parts of the image which is often unwanted, and there is also no room for it in smallpt.
Discarding the bright sample or similar. This bright sample is the culprit of the problem and we could always decide to just ignore it. However, that bright sample is there for a reason and removing it would most of the times not be the greatest idea (it also makes things more complicated). Clamping however would still allow us to somewhat discard the bright sample without actually discarding it and is very simple and easy to implement. We just make it not have such a big influence on the image.

Clamping is often the choice for most render engines. If you would look at any commercial render engine, or even hobby path tracers, they allow you to clamp the samples. Normally you clamp the individual samples instead of the sub-pixels and more specifically you clamp indirect samples with its own clamping value, but they have about the same effect.

This clamping does change the image as it makes some areas or all darker since it removes energy (by clamping the color values) from the scene. This would become unbiased, yet it is often done (and almost always available in unbiased render engines) simply because it produces visually nicer images, because there are no fireflies and fireflies are generally worse than noise. Often you would not notice any artifacts when clamping, other than the removed fireflies.

In the case of smallpt, clamping it between 0.0 and 1.0 seems to work well for that one scene. However, I do recommend to just tinker with the code and render two images out for yourself, one with clamping and one without.


Can (and how so) shaders paint screen pixels beyond those that occupied by the shaded mesh? I have some experience programming geometry and compute shaders - but never adventured myself in really playing with the fragment shaders. Currently, I am trying to better understand how they work and their potential. One of the things I have read in multiple places is that a fragment (i.e. a screen pixel) cannot extend beyond itself within the fragment shader. Which means, a given fragment being iterated can only affect itself.

Therefore, and for the sake of learning, I would like to know if the following is possible (and if it is, how, in general lines, can it be achieved). Suppose we have, for the sake of simplicity, a point-mesh consisting of only two vertices (located in the 3D World space). Can we program a shader in a way such that each of these two vertices would be painted on screen at their exact WorldToViewport position, but also that a circle around each of them of radius=R is also painted in the surrounding pixels even if they extend beyond the original mesh to which the shader is attached? Like in the figure below (where the red square inthe center of the circles represents the vertices painted on screen):

If that is possible, can a shader also be programmed in such a way that these circles that extend beyond the vertices influence the color (RGBA) of each other? Like in the figure below:

As I said, if these are possible things, I would love to hear a bit of how to achieve such a thing - either in conceptual or practical terms. Is it done in the fragment shader, or has to be calculated before at the vertex or geometry shaders? How to calculate and pass "additional fragments" that extend beyond those fragments occupied by the mesh body?

When you use linewidth or line antialiasing or pointwidth or pointsprites, OpenGL creates for you a small rectangle instead of the line or point, with texture coordinates. Nowadays you can even program this yourself using geometry shaders ot even the tesselator.

A totally different approach is to use defered shading, using one geometric pass just to store information in the RGBAZ buffer, then a second pass than you run on all the pixels of the screen to do some process. (to act on all pixels, simply draw one full-screen rectangle).
Nowadays you can even do the first pass as one or several "render to texture", then MIPmap these textures, so that the final pass can easily access to less local values.


How to include OpenGL extension loader in my project I have a modern OpenGL project hosted publicly.  I want to create the easiest experience so someone can just clone the repo and have it just work.

So far, I have included (as git submodules) the window creation library GLFW and the image loading library which is header only.

The only other part I have to handle is the OpenGL function loading part.  During development I have been using the webservice offered by glad

The two files I need from GLAD are roughly ~10k lines each.

Other than asking a user to downloading these files  and place them in the correct folder themselves, is there any way to get around including these files directly in my project?

Are there any other drawbacks to included the function loading code directly in my project?

GLAD is designed around you simply including it directly in your project. That's the point of it; it is easy to use because you don't have to deal with any external dependencies. No messy libraries or DLLs to generate. It's just another source/header file in your build process.

Furthermore, your code will (presumably) be written to expect GLAD generation with a specific OpenGL version and set of extensions. If you require the user to use GLAD to generate the file, you also have to tell them what version+extensions to use, or else your code won't compile. If you just ship the GLAD-generated files with your program, there's no chance of breakage (or at least, you could detect the breakage because your own build doesn't work anymore).


How is motion blur implemented in modern realtime renderers? When realtime renderers (such as 3d games) have motion blur, how is it usually implemented?

I think I've heard something about needing to have motion vectors for each pixel but I'm not sure about that and not sure how it would be used.

There are several techniques used.

A simple, but limited,  post-process approach that is not really used any more consists in reconstructing the world space position of a pixel using both the view projection matrix from current and previous frame. Using these two values you can compute the velocity at a pixel and blur accordingly, sampling along the velocity vector with how many samples you want ( the more sample the blurrier, but the more expensive as well).

This method unfortunately takes into account just the camera movement and therefore is not accurate enough if you have a dynamic scene with fast moving objects.

As you mentioned, other more modern techniques use a velocity buffer.

This velocity buffer can be created during the base pass in the deferred pipeline transforming each vertex using the world-view-projection matrix of the previous frame and then computing in the pixel shader the velocity vector using the screen space position of both frames.
With this velocity buffer you can, in a post-process step, sample across the per-pixel velocity direction like in the approach I described above.  The motion blur that you will get is obviously not camera-only; however you get an overhead in terms of memory (you need to keep an extra buffer) and time (you need to render additional info per pixel).

One technique based on velocity buffer that I particularly like is McGuire et al. that IIRC, is used with some variations in Unreal Engine 4 and many other games.

From [McGuire et al. 2012]

They divide the scene in tiles, assuming that each of these have a dominant velocity. Then, starting from the full res velocity buffer, they locate the velocity vector with higher magnitude for each tile, producing a downsampled version of the velocity buffer where 1 pixel = 1 tile.
Using this buffer, in another pass, they can generate a buffer that stores for each pixel/tile the dominant velocity among the neighbourhood pixels.
Note that the tile size is based on the max blur radius.

Using these buffers they then proceed with the gathering step sampling in the neighbourhood dominant velocity direction as this is the most plausible direction of the blur for each sample (as I said, the tile size is the max-blur radius). The weighting used for the blur takes into account the full res velocity and depth informations. Note that the paper propose three different weighting for three different scenarios (blurry over sharp, sharp over blurry and blurry over blurry).

Another important advantage of the method is that you can early-out if the dominant velocity in the neighbourhood is smaller than a certain threshold.


Actual vs Perceived Brightness of RGB Colour In this video from about 1:15, it is stated that if you have an RGB value of (0.5,0.5,0.5) it is only 22% as bright as (1,1,1) rather than the expected 50% as bright.

Does this mean that RGB is adjusted for a logarithmic scale so that we perceive the (0.5,0.5,0.5) to be half as bright, or have a misunderstood the video?

Despite searching around a lot I can not find any other source of information that discusses this.

Two different effects are causing the observation mentioned in the video.

On one side, the vast majority of screens have a non linear response: if the RGB value is half as much, the emitted light intensity is not half as much. This behavior originally comes from cathode ray tube (CRT) displays, which produced different light intensities by varying the voltage used to generate the electron beam. From the article:

  CRTs have a pronounced triode characteristic, which results in significant gamma (a nonlinear relationship in an electron gun between applied video voltage and beam intensity).

As CRT became ubiquitous, systems assumed this behavior to be present and took it into account. Nowadays CRT have mostly been replaced by LCD technology, which emulate this behavior for compatibility.

Gamma is usually considered to be around $2.2$, which means the light intensity follows a function $I=x^{2.2}$. This is where the 22% mentioned in the video comes from: ($0.5^{2.2} \approx 0.218$).

On the other side, the human vision is not linear either. According to Wikipedia, a typical sunset on a clear day is about 400 lux, and clear sky daylight is about 10000 to 25000 lux. Yet the latter doesn't feel over 25 times as bright as the former.

In the end, the non linear response of screens is a good thing, because it gives more precision where it is needed. Note that it is not logarithmic though, but closer to a square function. If you want to read more on the topic, you can look for articles on "gamma correction".


Am I passing the correct matrices for my shader? openGL/C++ On page 89 of David Wolff - OpenGL 4.0 Shading Language Cookbook book, it suggests a way to set up up a Phong shader in the fragment stage. It lists some different matrices to pass via uniform, and I suspect I might be passing slightly incorrect matrices, as my results are a bit off. This could be from somewhere else in the code of course, but I'd rather know for sure.

These are the matrices used that I'm curious about:

uniform mat4 ModelViewMatrix;
uniform mat3 NormalMatrix;
uniform mat4 MVP;

What exactly are these? At the moment I'm sending these matrices:

view*model //ModelViewMatrix;
transpose(inverse(model))) //top 3x3 row of modelview
projection*view*model //MVP;

The shader "works", but I can't get the lighting direction right.

I wasn't placing the lightposition into view space before passing it in.

vec4 LightPosition_view=view*LightPosition; //put lp into view space with the rest

When it comes to these things, you have to think in terms of spaces, and I was multiplying the view matrix and the model matrix, into view space, but not the lightposition itself.

I hope that helps anyone else facing the same/similar problem.


Why are depth buffer values clamped to (0, 1)? This is a simple question.

I know that in OpenGL or other graphics API, the values stored in the depth buffer are automatically clamped to (0,1).

I just want to know why people do this instead of just keeping the actual depth values as float. Is that just because of the non-linear depth equation which makes the closer depth values more precise and the further depth values less precies?

Thank you!

(Just converting comment into an answer)
It seems you might be asking one of two possible questions.

If you meant "Why doesn't the Depth buffer contain camera space Z"
then it's perhaps worth reading Why do GPUs divide clip space Z by W, for position?

If, OTOH, you just meant "why is it restricted to [0,1)?", then that's probably because old systems represented the Z-buffer depth with fixed point numbers, hence a natural [0,1) range.
 Of course, later hardware arrived that could use floating-point depths; IIRC Dreamcast's depth representation was an arbitrary positive float representing $\frac{1}{w}$.


Interpreting Image Processing Math Equation I am reading "Image Processing and Analysis" by Chan and Shen, c 2005 SIAM. They introduce some notation I'm not 100% sure how to interpret:

$$ u_0(x)=u(x)+n(x),  x=(x_1,x_2) \in \Omega $$

They state $u_0$ represents a noisy image, $u$ is the clean image and $n$ is Gaussian white noise. I assume that $ x=(x_1,x_2) $ is meant to convey that x is a two-dimensional vector indexing both image height and width. I don't know how to interpret $\Omega$. Is it meant to denote the entire image index range?

It sounds exactly like that!

$\Omega$ is the image domain, it can be either discrete or continuous. No idea how it is defined in that particular book.
$x=(x_1, x_2)$ are the 2D image coordinates
$u_0, u, n$ are functions that return intensity values at the given location $x$.


Main techniques to draw a point at non integer coordinates? I've difficulties in looking for an answer by myself because I don't know the jargon. I need somebody who'll points me in the right direction.

Assume I have 100000 points with coordinates in the unit square, so points like (0.55313, 0.236831).

I want to create a B/W picture with a given side in pixels (say 1000 x 1000) which shows these points.

The naive approach is to multiply by the side the coordinates and round, so point (0.55313, 0.236831) becomes pixel (553, 237) and to turn pixel (553,237) on.

This is rather unsatisfactory. If the points are many, say millions, I could easily fill all the area with a black blob that gives me 0 information.

So, what I thought was to have an array 1000x1000 of floating point numbers that I can update somehow when I add a point, and at the end I can normalize the values and obtain a grayscale image, which will be "blacker" where more points are.

Now the question: which algorithms are used to add a "blurred" point? I.e., I would like to take into account the real coordinates, so that, if a point is near the border of a "pixel" I would like its addition to my array to "smudge" a little the nearby pixels. Like when you draw a point in a drawing program and it actually adds a little disk with decreasing (linear? Gaussian?) intensity.

Surely it is a problem that has been already been studied and solved, just I don't know how it is called...

thanks

Instead of having a binary on/off per pixel you can instead make a heat map. The more points fall within the pixel the brighter the pixel is. Essentially create a 2D histogram which you then renormalize to get the grayscale for the output.

If you want to blur the points then you can move a portion of the brightness to the next pixel based on the remaining fraction after multiplying to the resolution. If it the coordinate is (0.55313, 0.236831) and your resolution is 1000x1000 then 0.87*0.17 goes to 553, 236, 0.87*0.83 goes to 553, 237, 0.13*0.17 goes to 554, 236 and 0.13*0.83 goes to 554, 237. This is distributed linearly but there are other options.


Are there tricks for getting proper sort ordering on particle systems? If you have a bunch of particles to render, using different shaders and/or render states, that have some level of transparency, the naive solution of sorting all particles by depth can be extremely inefficient since it is likely to turn a few large batch draw calls into a multitude of smaller batch calls which have expensive render state switches between those draw calls.

I know of a few ways to help fake things such as sorting particle batches by their emitter location, or using something like additive blending, which gives the same result regardless of draw order.

I also know there are some techniques out there to attempt order independent transparency, often being an approximation, requiring a (sometimes unbounded) amount of memory, or some combination thereof.

Does anyone know any pragmatic solutions (fast/low extra memory requirements/absolute correctness not required) for dealing with this situation?

First option should be to make all particles able to go through the same pipeline. Perhaps with an uber shader. That way you can batch them all.

Positive ieee floating point numbers can be sorted like unsigned integer. And there are O(n) algorithms to sort those for example radix sort.


gamma transform in webGLSL: when already done or yet to be done? Color spaces issues are such a pain, especially when you create color data at the middle of the chain, and a pile of libs and apps separate this to the display... Here I only worry about gamma transform.

Shadertoy addicts are coding in this web site webGLSL shaders, which in my case are rendered through google chrome on linux ubuntu and displayed on a calibrated monitor.

Is one layer already doing the gamma transform, or should I do pow(color,1./2.2) at the end of my rendering ?
For me (linux,etc), it seems I must do it. Is it also true on windows, mac OS, whether Angle or Native OpenGL is used (windows), whatever the browser ?
The point is that often shaders look ok on windows and very dark on linux, for instance, both people being sure of their settings and qualibration.

This is a hard problem since the systems do not in fact tell you how it should operate. The problem is that there are several ways to address this issue:

Apply a LUT to all output
Let the system color manage
Let each application color manage

Nearly all of our color management systems are borderline braindead. There is really no sane way to ensure that you are doing the proper thing without extensively testing the system. You can not ask or specify your intent. In case of multi operating system situations. If there is reason for me to color manage something myself im at the mercy of each operating system peculiarities.

For a alternate discussion on this see this stackoverflow post. But its not really a robust enough talk to cover all cases.


how to update my scene graph? I am updating my scene graph in response to user input and the network. What is the best way to do this? The updating threads are separate. Some approaches I am thinking about:

double buffer. There are 2 copies of the scene graph, one is used for rendering and the other for updates. The rendering graph is updated at the end of the render loop. This update can be expensive for a large graph,
synchronization objects, like mutices, probably unsuitable,
dumping updating functors into a FIFO queue while rendering, then executing the functors at the end of the rendering loop. Updating the queue may block.

Is there something better and which approach is best?

A few more options:

don't use a scenegraph. They don't map well to the data a gpu needs for rendering. A flat array of object tends to work much better for the cache as well
separate the scenegraph into render-related data and non-render-related data. Then only double buffer the render-related data.

The render related data would end up in a flat array containing per rendered object: reference to the mesh, the final model matrix, a reference to the texture

Flat arrays are much easier to double buffer than node-based data structures.


How do I make sure my cosine-weighted sampling works fine? I'm working on a pathtracer and while it's working mostly fine, I'm getting different outputs with cosine-weighted sampling and uniform sampling :

Uniform sampling is on the left, cosine-weighted is on the right. The difference is somewhat subtle, but it does not go away with more samples so it's not some kind of start up bias.

I've checked my code against PBRT and Wikipedia but I cannot figure out what is wrong, because all the factors seem to be there (brdf, pdf, Li, and cos(theta)). I'm hoping someone here could take a fresh look at it :

// Sampling functions :
Vec3 sample_uniform_hemisphere(Sampler* sampler, float* pdf) {
    float cosTheta, v;
    sample_unit_square(sampler, &cosTheta, &v, NULL);
    float sinTheta = sqrtf(1 - cosTheta * cosTheta);
    float cosPhi = cosf(2 * M_PI * v);
    float sinPhi = sinf(2 * M_PI * v);

    if (pdf)
        *pdf = .5f / M_PI;
    return (Vec3) {
        sinTheta * cosPhi,
        sinTheta * sinPhi,
        cosTheta
    };
}

Vec3 sample_cosine_weighted_hemisphere(Sampler* sampler, float* pdf) {
    Vec3 sample;
    sample_unit_disc(sampler, &sample.x, &sample.y, NULL);

    float z2 = 1.0f - sample.x * sample.x - sample.y * sample.y;
    sample.z = sqrtf(z2 > 0.0f ? z2 : 0.0f);
    if (pdf)
        *pdf = sample.z / M_PI; // p(theta, phi) = cos(theta) / pi
    return sample;
}

// Snippet that does the actual shading :
        float pdf;
        Vec3 bounce_sample = it.material->bsdf->sampler(it.sampler, local_out, &pdf);
        Ray bounce = {
        .o = surface_point,
        .d = basis2world(bounce_sample, u, v, it.normal)
        };

        Intersect bounce_it = trace_ray(sc, bounce, it.depth + 1, it.sampler);
        Color f = it.material->bsdf->f(it.material->params,
                       bounce_sample, local_out);
        Color dc = cmul(
            bounce_it.outgoing_radiance,
            f);
        float inv_pdf = pdf > EPSILON ? (1.0f / pdf) : 1.0f;
        rval = cadd(rval,
            cscale(dc, inv_pdf * bounce_sample.z));

Thanks in advance!

The code you've shown here looks right to me. In order to verify the implementations, you could try writing some code that just integrates a few test functions over the hemisphere using both methods and verify that you get the same result (within some sampling error). Also if you haven't already, verify that the outputs from sample_unit_square and sample_unit_disc are doing what they're supposed to.


Using Software Rendering to make an External GPU I was looking at the Raspberry Pi 4 specs and realized that running a software renderer on the pi could get some decent graphics. By using USB ports and Ethernet ports, I think I might be able to send graphics data to the pi, using it as an external graphics card.

While the graphics of the external pi card would obviously not approach an RTX, would this work?
Is there a software renderer that I could run on all 4 cores of the Rpi 4?

I realize that I may have to create this myself, and I would like to make a custom, real-time Vulkan driver that uses the pi's cpu to do most of the rendering. It would only use the on-board graphics card to do simple, repetitive tasks that it can do efficiently.

By installing a special driver, I could output graphics data from a computer into the pi via USB or Ethernet; the pi would then solve it and output it through its own display ports.

Does anyone have info that could help me?
I'm a graphics noob and the only help I can get right now is from my dad, but he is quite busy.

Theoraticaly this can work, but I am not sure if it would be very efficient. Modern GPU's are tailored for doing low level graphics operations and the drivers that manage them are also tailored for efficiently overseeing these operations to render your scene.
Now you can ask all the hardware questions related to using Raspberry Pi, how to access its cpu etc to Raspberry Pi SO.
As per software side, I think P. Shirley's books on
ray tracing should get you started on rendering on cpu's rather than gpus. Using multiple threads is extensively discussed in the following issue of the source code of the book. So once you had written your single threaded cpu ray tracer, you can adapt it easily to execute in multiple threads.

Update 21-01-2021
I noticed that there is already a driver for Raspberry Pi 4 provided by mesa, which gives you access to OpenGLES 3 functionality. I think it is a decent trade off, I was even surprised to see that there is actually a graphics driver for Pi.
Unless you insist on ray tracing your scenes, I would suggest to use that for your experiments.


Label Placing for thousands of objects with OpenGL Question

How to manage data (GPU,CPU) for dynamic label placement in 3d for thousands of objects (OpenGL)?

Details

objects:

count: 2k-10k
type: points/quads

labels:

bilboarding
data: text and colors

placing:

labels are not overlaping

more on this in my another question

Elaboration

To calculate label placing I need screen space coordinates. There are lots of algorithms for the purpose, but I do not even know if they are implementable (w/o some hard hacking) on shaders since Vertex Shader has access to only one vertex. But with many objects calculating matrix multiplication (MVP * vertex position) for all vertexes so as to get screen-space coordinates on CPU-side and sending them each frame seems to be fairly slow and badly designed, imo.

So, should I:

compute screen-space coordinates on CPU and send them to GPU so as to draw objects, compute label placing on CPU and send data to GPU
send only vertice's word coordinates and calculate screen space coordinates on GPU, then somehow implement label placing on GPU
Read the GPU-side calculated screen-side coordinates and compute on CPU
something different

I know that it depends on the load on CPU/GPU from other things, but what do you think?

Since you are new to computer graphics, you may be better off avoiding the complications of SIMD and sticking with the traditional 'create a CPU thread + collect results' approach, or even run in the main thread if the task is lightweight enough.

But if that approach fails to be performant and/or you are willing to dip into OpenCL / GLSL compute shaders, read on...

This is a per-object calculation, not a per-vertex or fragment calculation, so the traditional graphics pipeline is not the best choice. That said, you could lead a vertex buffer with vertex positions at each object world position, then transform the vertices to screen space (see below) in the vertex shader, but considering you are dealing with OpenGL 3.3 you will probably be limited to writing the output label positions to textures via render to texture or transform feedback buffers which may be slow depending on the hardware. If you can use the shader image load store extension, it could help, but bending the graphics pipeline to GPGPU is all probably more work than it is worth.

You have not detailed your overlap requirements (i.e., whether it must be perfect or approximate, overlap probability, whether you can guarantee sufficient screen room to achieve perfect non-overlap, whether you need all labels to be on visible screen at all times, etc.), but generally pairwise problems which can give non-convergent iterations when multiple particles are adjusted simultaneously are best done on a single thread on the CPU. It could be a dedicated CPU thread.

To understand what I mean by non-convergent iterations, imagine a grid of labels whose lower left and upper right corners touch. If they all are adjusted simultaneously in opposite directions of one neighbor, they may end up in a situation where they simply overlap their other neighbor. This cycle may repeat indefinitely since the corner overlaps are opposite. Whether this actually occurs depends on the specifics of the algorithm and the object positions / dimensions, but it is possible unless you can make some object spacing guarantees or some other assumption I am unaware of.

If you have access to GLSL computer shaders or (even better) OpenCL, and you  do not demand perfect overlap, or predict that overlap will be relatively seldom, the following simple SIMD approach may outperform a single-threaded CPU approach.

SIMD algorithm: At the beginning of each frame (or whatever label update freq. makes sense), pass the object world positions, label dimensions, and camera projection + view transform to a per-object + label pair compute shader / kernel to calculate the screen space positions:

vec4 objScreenSpacePosition = camera.projectionViewTransform * object.worldPosition;

Now place the corresponding label somewhere nearby.

Then place a sync fence in the shader / kernel to ensure all screen space positions have been calculated before proceeding. Finally, perform a pair-wise loop with all other labels and adjust the label position / depth to correct for overlap. You may need to repeat the loop multiple times until all labels have no overlap if you demand absolute assurance of zero overlap, or you could cutoff overlap adjustment after a certain number of iterations to ensure it doesn't take too long or run indefinitely (see above about non-convergence).

Note that the adjustment loop is only adjusting the label its compute shader was assigned. To avoid position read/write race conditions, it is best to use a temporary position and fence every time you decide whether to continue the loop. Here is an example:

vec4 objectScreenPos = camera.projViewTrans * object[kernelIndex].worldPos;

//...transform objectScreenPos to desired coordinate
// system if needed; either [0,1] or [-1,1] or whatever
// depending on your projViewTrans and needs of your
// overlap correction algorithm...

label[kernelIndex].pos = objectScreenPos.xyz + someOffset;

fence();

bool overlap = true;
uint maxIters = 5;
vec3 tempPos = label[kernelIndex].pos;
for (uint numIters = 0; overlap && numIters < maxIters; numIters++)
{
    for (each labelIndex)
    {
        // Of course a label overlaps itself!
        if (labelIndex == kernelIndex)
            continue;
        // ...check overlap and adjust position of tempPos...
    }
    label[kernelIndex].pos = tempPos;
    fence();
}

Of course avoiding fencing as much as possible and avoiding cross-kernel talk of any kind is ideal, but for this problem you can't completely avoid it. Still, I'd try turning off the fence in the loop and see what happens to perf and output. =)

There are more clever overlap correction algorithms, most of which are approximate, but this one can in principle perfectly correct overlap and is simple to implement and understand.

A GPGPU or even threaded CPU approach could actually be slower than performing a single-threaded calculation on the CPU, depending on the number of actual objects. This is why OpenCL is ideal for these sorts of problems: it gives you the option of running single or multi-threaded on CPU, or moving to the GPU. You could even switch between CPU and GPU on the fly if the number of objects radically changes over time and you can predict the change ahead of time.

I would probably write this in OpenCL since if all else fails, I could run it in a single thread on the CPU and come as close as possible to guaranteeing perfect non-overlap. Then I could experiment with multiple threads / GPU calculations and profile, profile, profile to my heart's content.

Update - OpenGL 3.3 graphics pipeline approach:

While OpenCL or the OpenGL compute pipeline would be ideal, you should be able modify the above SIMD alg to use the OpenGL 3.3 core graphics pipeline with a bit of extra effort and likely perf loss. Example: During a point draw, pass object positions and IDs as vertex data to a vertex shader that calculates the object screen position. Set glPosition to the object ID and pass the resulting screen position to the fragment shader using the flat modifier to prevent interpolation. Render the result to a 1D framebuffer with length equal to number of labels so that there is a 1:1 correspondence between 'pixel' (i.e. label) and vertex (i.e. object). You now have a 1D texture filled with label positions. You could then read this back to the CPU to perform overlap elimination (with possible guarantee of no overlap), then upload the updated label positions to OpenGL in a uniform buffer array, texture, or whatever works best in your situation. Alternatively you could launch a second vertex shader with label IDs as vertex attributes and perform a non-guaranteed overlap elimination which does not depend on the updated positions of the other labels, then follow a similar pattern from before to render the updated positions to a second 1D framebuffer, and finally use the related framebuffer texture as a label ID indexed lookup table during label rendering to determine the final positions of each of the four billboard corner vertices. The label ID could either be passed in using a uniform or vertex attribute.

Hopefully you are beginning to see how much the GL 3.3 core graphics pipeline limits your options for GPGPU and how much more complicated it makes things. What was before a single compute shader or kernel gets broken into multiple shaders (and possibly multiple draws) where you must abuse the concepts of 'vertex' and 'pixel' to achieve your ends. Achieving good perf with this approach often requires a decent understanding of the graphics pipeline and hardware you are working with. Depending on the hardware, driver, and number of objects, the above graphics pipeline algorithm may not be able to beat simply calculating screen space coordinates and eliminating overlap on the CPU then uploading results to GPU. Hence the reason I suggest you either opt for OpenCL or OpenGL compute, or else stick with traditional CPU threading.


Given two convex patches, how to find points/faces that have distance < x? The title says it all, given two convex patches, how to find points/faces that have distance < x?  Can I use GJK for this case?

Convex patches are obtained from convex decomposition, see here. If I read it correctly, I don't think methods used in that paper can give points/faces that satisfy above condition.

Yes GJK is appropriate for this.

The overall algorithm of GJK goes something like

simplex = empty
direction n = arbitrary

while(not finished){

    find furthest point p1 in mesh 1 in direction n
    find furthest point p2 in mesh 2 in direction -n

    supplement simplex with point p1-p2

    if(simplex is tetrahedron) simplify simplex

    direction n = pick direction perpendicular to simplex pointing to origin

}

you can then use the simplex at the end of the loop to calculate the distance from it to the origin and set finished to true and you will get a collection of points on each mesh which make up a pair of faces that are closer than x.


Change of speed but no change of size after OpenGL Perspective Projection I am new to OpenGL and I am trying to code un optic flow with dots appearing from the background and moving towards the viewer. As a consequence of a perspective projection, the dots are faster and bigger when they move towards the viewer, but they are smaller and slower when they are farther away from the viewer.

As a result of OpenGL perspective projection, I got the dots moving faster when approaching the viewer, but their size does not change with the distance. Isn't the change of size a consequence of the projection -- just as the change of speed? If not, shall I compute it separately?

Thanks in advance for your help]1

When you draw a point with GL_POINTS, it's not a 3D object with a size in world-space: its size is a fixed number of pixels. Because of that, the size of the point is independent of the distance from the camera.

If you want to draw dots whose size depends on distance, you either need to compute the perspective yourself and change the point size for each point (not recommended, because state changes like that are expensive); or use another method to draw them. For instance, you might draw a quad and use the fragment shader to make it a circle (e.g. by discarding the fragment if the world co-ordinate was more than a particular distance from the centre of the quad, or by using a transparent texture with a circle in it).


Understanding lighting for physically based rendering i wanted to ask if you know the best books or resources to understand light physics. I have heard that Subrahmanyan Chandrasekhar's book "Radiance Transfer" is very good. Any other books or suggestions will be appreciated

Here you have a full repository with a lot of papers about light transport. There's also a folder named 'theory' where you can get both intro and advanced refs regard light physics. And much more. I'm personally curating this and keeping up to date with latest advancements.

https://drive.google.com/drive/folders/16A6Yj2esRTk0FP-ALr6XfxkGQvTb_1dD?usp=sharing

However here are the main books/papers I suggest to read :

1985 - Physically Based Lighting Calculations For Computer Graphics

1986 - The Rendering Equation

1997 - A Framework for Realistic Image Synthesis

1997 - Robust MonteCarlo Methods For Light Transport Simulation

2011 - The State of the Art in Interactive Global Illumination

2014 - The Path to Path-Traced Movies


How to determine RGB values given a display spectral response curve? I am reading Physically Based Rendering section 2.2.2 on RGB color.
I am trying to understand how, given a display spectral response curves for rgb intensities, we can choose the intensities so that the display will emit an equivalent spectrum (a CIE xyz metamer)
It is my understanding that the following section tries to explain exactly that.

Given an (x,y,z) representation of an SPD, we can convert it to corresponding
RGB coefficients, given the choice of a particular set of SPDs that
define red, green, and blue for a display of interest. Given the
spectral response curves R(lambda), G(lambda), and B(lambda), for a particular display, RGB
coefficients can be computed by integrating the response curves with
the SPD S(lambda) and using the tristimulus theory of color perception:

I can't understand this formula. Why it is considered correct? Why choosing r this way makes the display emit the correct / equivalent SPD?
Can someone break it down for me?

I'll try my best to give an overview of the subject, and english is not my first language so feel free to ask more detailed information in comments etc.

Here are some terms, light is an electro magnetic radiation. When the power of this radiation falls between a certain wave length, it becomes visible for human beings. S is the function that takes wave length value as input and gives a power value as output and is called a s(pectral) p(ower) d(istribution).
Color on the other hand means different things to different people. We can define color based on our perception of it, or based on the measured spectral power distribution, based on the average physical capacity of human eye etc.
The thing is spectral distributions change over time constantly, but the color does not change at the same speed. This means that variations in the spectral distribution does not map to color one to one, which shows that there must be a transformation happening between what we perceive as color and what we measure as spectral distribution.
Now the needs for an accurate and standardised representation of color was quite important for daily life (think of colors around road signs, lamps, etc), the CIE defined its trichomatic system for this purpose. The main idea behind the trichromatic system was that given a carefully chosen three reference spectral distribution, we can represent all of the perceivable colors by mixing these three spds.
Notice that I did two things over there. I passed from mixing spds, which should produce an spd at the end, to color values which are generally represented with trichromatic system, and I added perceived. The last one is to signal that this is an experimental procedure, that is people did a lot of experiments to come up to this idea of mixing reference spds.
How do we arrive from mixing spds which should produce an spd to a color value in rgb ? Simple by summing all the power values of the spd resulting from the mix.
The mix is $R(\lambda)S(\lambda)$ part where R is the reference spd and S is any given spd. $\int$ indicates that we are summing the resulting multiplication.
$\lambda$ is the wave length.
A great reference on the subject is McCluney 1994, Introduction to Radiometry, chapter 11. Read through 11.2 and then 11.4, that should clarify anything and everything with respect to spds and their transformation to trichromatic values


Confusion about notation in a paper I have been trying to understand "Fast Simulation of Mass-Spring Systems" by Liu et al., which can be found here. There is one part in the methods section that is confusing me. Just after equation 12, it states:

Similarly, Si ∈ Rs is the i-th spring indicator, i.e., Si,j = δi,j

The subscript i is for each spring in the system. But I am not sure what δ is supposed to be - it isn't mentioned anywhere else in the paper. I also don't know what the subscript j is for, or what a spring indicator is. My guess was that S is a diagonal matrix, with the deflection of each spring along the diagonal. But this doesn't work in my code. Having S as just the identity matrix works better.

Does anyone know what the S matrix is supposed to be?

The notation $\delta_{i,j}$ is the Kronecker delta, a notation commonly used in physics. It's defined as:
$$\delta_{i,j} \equiv \begin{cases}1,&i=j\\0,&i\neq j\end{cases}$$
So, as you suspected, it's essentially a shorthand for the identity matrix.

The notation $\mathbf{S}_i \in \mathrm{R}^s$ means that each $\mathbf{S}_i$ is an $s$-dimensional vector, so the index $j$ labels the components of this vector. The term "indicator" probably refers to the concept of an indicator function, which is a function that labels some set or point with the value 1 while mapping everything else to 0. So, the "$i$-th spring indicator", in this context, is a vector that has a 1 in the $i$-th component and 0 everywhere else. That's exactly the Kronecker delta, i.e. the identity matrix.

The way this "indicator vector" gets used in equation 12 in the paper is in the form of an outer product with another vector, $\mathbf{A}_i \in \mathrm{R}^m$. The two vectors are multiplied together in the form $\mathbf{A}_i \mathbf{S}_i^{\mathrm{T}}$, which is an outer product. The result is an $m \times s$ matrix with the contents of $\mathbf{A}_i$ in the $i$-th column, and zeroes everywhere else.


Texture units parameters in OpenGL I am writing a procedural terrain renderer for OpenGL. To shade the terrain I use multitexturing, that is 4 material textures and a blend map which distributes them across the terrain (encoded in RGBA).

Should I decide to split my terrain patch into two, blend map, consequently gets split too. Now there is a seam problem. Because of texture filtering, on the edges, the color "bleeds" from the opposite side of the texture. I recognize this can be fixed by setting texture repeat parameter to GL_MIRRORED_REPEAT or GL_CLAMP_TO_EDGE.

I believe that glTexParameteri function sets parameters for currently bound texture unit, still I'm unsure if these parameters get saved in texture unit's state, e.g. each texture unit "remembers" its parameters. Moreover, I know these parameters can be set per texture object. In that case whose parameters should take precedence?

On a more concrete note, I sample blend map from GL_TEXTURE0. So I had an idea to set repeat parameters for GL_TEXTURE0 once before rendering all terrains, but it doesn't work.

However, if I set this for every terrain rendered (absolutely the same code), it works. I'm inclined to believe that texture units don't "remember" their parameters. Is there something else I'm missing here?

Your understanding of the behavior of texture units and state is confused. That's to be expected: OpenGL's pre-DSA API is not making it easy to understand this.

As far as core profile OpenGL is concerned, the only texture unit state is what textures are bound to which targets. That's all. Therefore, all glTexParameter calls affect the state of the texture object bound to the currently active texture unit, not of the texture unit itself.

So there is no "precedence". Texture objects have state, which are modified by glTexParameter calls. This is made more clear in direct state access APIs, because glTextureParameter directly takes a texture object, rather than a bound texture target.

If you call glTexParameter when texture object 0 is bound to the given texture target, then you're setting the parameter of the default texture object for that unit/target. And you shouldn't do that.


Real 3D interior room from 360 images If I have 360 images of an interior room from all different angles, would it be possible with current 3D software technology to create a real 3D representation of that room, where the camera could freely navigate to any part of it?
So similar to Photoshop’s stitch, where you stitch a bunch of photos to create a single 360 photo, would you be able to stitch a bunch of 360 photos to create a 3D environment?
I’m not looking to create graphics or photorealism out of the room, but rather the actual real room. Most of my research points either to one of the above or VR videos, which have limitations on camera direction.
If it is possible, could you please point me to the right direction/software and/or examples of work that have achieved this?
If it’s not possible, I would be very interested in knowing what the limitations are.
Thanks!

A website devoted to prescanned photogrammetry is :Quixel.com They did (at least some of) the assets for the version 5 unreal engine demo.
I have gotten excellent results with software from Agisoft but it costs real money.
For good free software try 3DF Zephyr
This software will just generate the mesh, they will provide only minimal navigation of the result which is outside the scope of generation.
To navigate the mesh you can use something like windows 3D viewer.
Or better yet, and way more fun, write your own!


Finding geodesics on a mesh? What are some state of the art methods/algorithms to find geodesics on arbitrary manifold meshes?

There can be much dispute on which work is considered best or state-of-the-art. It will depend on your application. However, I'd like to recommend to give a look at this paper by Keenan Crane et al. It is impressive in terms of computational speed. Also, It is simple to implement.
Crane et al. Geodesics in heat: A new approach to computing distance based on heat flow. ACM TOG. 2013. (https://dl.acm.org/doi/10.1145/2516971.2516977)


Correctness of logical steps for create the matrix of viewing transformation In general, I understand the creation of viewing transformation matrix, but I have my doubts.

For example, let's say that a world coordinate (called $A$) has a center at $(1,1,1)$, and the view reference point (VRP) (called $B$) has a center at $(3,3,2)$.

Each vertex of the world must be "mapped" to $B$. The axes of $A$ and $B$ aren't aligned so a translation isn't enough, but there are two necessary rotations (around $x$ and $y$ of two certain angles).

I want to know if I am proceeding in a correct way. For overlapping the axes from $A$ to $B$ these are the following steps:

translate (transformation $T$) with translation parameters $(2,2,1)$, namely $(3-1,3-1,2-1)$
rotation around $x$ (transformation $R_x$)
rotation around $y$ (transformation $R_y$)

Thus, the transformation is:
$R_yR_xT$ when $T$ is the first matrix which must be applied. Thus, the transformation to "map" a point of $A$ into the the corresponding point in $B$ coordinates is the inverse namely:
$(T^{-1})(Rx^{-1})(Ry^{-1})$

My main doubt is if it's correct, for converting $A$ and $B$, to do a translation first.

In general, is this process correct?

I'm assuming you mean $A$ is object$\rightarrow$world matrix and $B$ is your camera$\rightarrow$world matrix? If that's the case, to calculate object object$\rightarrow$camera matrix you calculate:

$$C=B^{-1}*A$$

In general your math is correct, i.e. the order of matrix multiplication is reversed for the inverse calculation. It's just that normally these transformations are combined to a single 4x4 transformation matrix which does rotation, scaling and translation, and then you just work with that single matrix instead of it being decomposed to separate rotation & translation matrices. Anyway, if you want to decompose the matrix multiplication, based on the above you would do:

$$A=T_A*Ry_A*Rx_A$$
$$B=T_B*Ry_B*Rx_B$$
$$C=Rx_B^{-1}*Ry_B^{-1}*T_B^{-1}*T_A*Ry_A*Rx_A$$


Bevel/Chamfer algorithm for arbitrary 3D mesh I can't find any paper about a bevel/chamfer algorithm for 3D mesh. Bevel has a lot of special cases (corners, different bevel profile, bevel larger than a face). I am looking for robustness.
Do you know any algorithm for this ?

Thanks

You should look into Medial Axis Transform or Straight Skeletons. They are often used to generate the information you need to bevel things properly. They give you a distance from the bounds of your object at any point in space, which allows you to generate bevels correctly. They are difficult to calculate, though.

You may also want to look into this paper from Valve Software for an alternative way to generate the distance field.


Confusion about light PDF I have a doubt regarding shape lights pdfs, specifically various sources point at the fact that the pdf is a constant:

$$
lightpdf = 1/A
$$

Where $A$ is the area of the light surface. Expressed in solid angle terms, this becomes:
$$
lightpdf(\omega) = \frac{r^2}{cos\theta A}
$$

or $= 0$ for directions on the hemisphere which do not hit the light.

Where $r^2$ is the squared distance from the hit point o the light surface and the point we are sampling from, and $\theta$ the angle from the light surface normal and our ray direction (source).

What isn't clear to me is knowing that the pdf should satisfy this condition:

$$\int_0^{2\pi} lightpdf(\omega)d\omega = 1$$

Assuming for example a sphere light source (but a cylinder would also do), if in the second formula above we use the entire sphere area as $A$, aren't we using a solid angle for the sphere which is double of what it actually should be? (by including the half surface we cannot see from $p$:

Most of the answer is in the text you linked:

  Shapes almost always sample uniformly by area on their surface.
  Therefore, we will provide a default implementation of the
  Shape::Pdf() method corresponding to this sampling approach that
  returns the corresponding PDF: 1 over the surface area.

So it is not always 1. This is just the most basic way to sample a light. It is simple, fast and provides a good baseline to check other algorithms for correctness. As for:

  Assuming for example a sphere light source (but a cylinder would also
  do), if in the second formula above we use the entire sphere area as
  $A$, aren't we using a solid angle for the sphere which is double of
  what it actually should be? (by including the half surface we cannot
  see from $p$:

The half you cannot see will simply be excluded by the visibiliy test. Or because it is back facing, if you choose to emit light only on one side of the object. A better sampling algorithm will take that into consideration and adjust the PDF accordingly. Indeed, if you read a little further on that page, you will find:

  The second shape sampling method takes the point from which the
  surface of the shape is being integrated over as a parameter. This
  method is particularly useful for lighting, since the caller can pass
  in the point to be lit and allow shape implementations to ensure that
  they only sample the portion of the shape that is potentially visible
  from that point.

Which is exactly what you are asking about. But doing this is not about having correct sampling, it's about having efficient sampling. Any sampling method is correct as long as:

It can generate samples everywhere they are potentially needed.
Its PDF correctly represents the probability of generating each sample.


Rasterization: How does hardware disambiguate identical Z values at silhouette edges? I have written a software rasterizer, and have encountered a (literal) edge case.
At the silhouette edge of a typical object, you have two triangles—the front face and the back face.  They are not coplanar, but they of course share that single edge.  Say the pixel center / subpixel sample happens to lie very close to this edge—so close that the depth value computed for both triangles is actually exactly, bit-for-bit, identical.
Without special handling for this case, sometimes the "wrong" fragment will get through.  For example, with the < depth function, if the backface renders first, then the depth test for the frontface will fail and the backface will show through.   If the frontface renders first, then the depth test for the backface will fail and the frontface will show.  The former case produces the wrong image—sometimes quite objectionably so.
I have thought about hacks to fix this like trying to handle the "equals" case explicitly, depending upon the depth function and whether the fragment came from a front face, but it's not clear, especially for the ≤ and ≥ depth functions, what the "right" answer is.  Still, it's about the only thing I can think of trying.
This is basically Z-fighting, but unlike the usual Z-fighting graphics newbies encounter when they don't set the clipping planes right or whatever (and which makes this issue almost ungoogleable), existing hardware definitely does provide some kind of solution to handle this particular case, because this problem does not occur on real GPUs.
So my question is—how is this handled by real GPUs?  If that's unanswerable, then how can it be handled consistently?

If you are taking about this occurring within a single draw call for a single object then you are talking about rasterization not Z fighting.  So the real question is, "how are two triangles rasterized that are right next to each other". And the answer to that question boils down to rasterization rules. Which must be enforced, on GPU's that happens in hardware, for your software rasterizer you need to pick a rule set, then enforce it in code. Generically a rasterizer can leave no holes, and each pixel is only drawn once. You may want to always add colors when drawing your triangles, start with a color of zero and just add new colors drawn to that pixel, this will make your overdraw stand out like a sore thumb.
The most widely know rule set for handling this is the "top left rule" which states: The center of every pixel must lie completely inside the triangle, (you are computing pixel centers right?) or it center lies exactly on a triangle edge or on multiple edges in the case of corners, and a left edge is a non-horizontal edge that is on the left side of the triangle.
This last bit is directly from Wikipedia.