Why is the half vector not used in diffuse BRDF Fresnel calculations? I am currently working with Earl Hammon Jr's Presentation PBR Diffuse Lighting for GGX+Smith Microsurfaces (now mentioned as [PBR, p.XYZ] and have read through (among others) Brent Burley's Physically-Based Shading at Disney (now mentioned as [DIS, p. XYZ] to get a good diffuse BRDF component. I am stuck at combining the two with the Fresnel Term.

Short introduction for vectors and angles as I use them:

$\omega_i$ is the light vector
$\omega_o$ is the view vector
$\omega_n$ is the macro geometry normal
$\theta_i$ is the angle between $\omega_i$ and $\omega_n$
$\theta_o$ is the angle between $\omega_o$ and $\omega_n$
$\theta_h$ is the angle between $\omega_n$ and $\omega_h$
$\alpha_{hi}$ is the angle between $\omega_i$ and $\omega_h$
$\alpha_{ho}$ is the angle between $\omega_o$ and $\omega_h$ (this distinction is for clarification)
$\alpha_h$ is any of the angles $\alpha_{hi}$, $\alpha_{ho}$, since they are equal

now given that $r_s$ is the BRDF term of the specular component without the fresnel factor and $r_d$ accordingly the term of the diffuse component without the fresnel stuff, the fresnel factor is written as $F(angle)$.
[PBR, p.105] mentions that the diffuse light is transmitted two times, once in and once out. Thus, the Fresnel component has to be multiplied two times.
[PBR, p. 106] goes on to say Fresnel's laws are symmetrix, meaning entering and leaving is direction independent (i.e. it does not matter that once we go into the material from air and once we leave into air). Now I would assume (for $F_1$ is Fresnel for entering and $F_2$ is for Fresnel leaving the material) to use

$(1-F_1(\alpha_{hi}))*(1-F_2(\alpha_{ho}))$

$F_1$ and $F_2$ are the same function, and $\alpha_{hi}$ and $\alpha_{ho}$ are the same angle, therefore

$(1-F(\alpha_h))^2$

This would lead to a brdf $f$:

$f = F(\alpha_h) * r_s + (1-F(\alpha_h))^2 * r_d$

But both [PBR, p.113] and [DIS, p.14] use

$f = F(\alpha_h) * r_s + (1-F(\theta_i))*(1-F(\theta_o)) * r_d$

as does the original paper to use this kind of calculation by Shirely et al. 1997.
I just don't get this, why do they change from the microfacet angles to the macro angles? The microfacet angles lead to energy conversation

$F \in [0, 1]$
$\Rightarrow(1-F) \in [0, 1] $
$\Rightarrow(1-F)^2 \in [0, 1]$ and $(1-F(\alpha_h)) >= (1-F(\alpha_h))^2)$

it should be reciprocal

$f(\theta_i, \theta_o)
= F(\alpha_{hi}) * r_s + (1-F(\alpha_{hi}))*(1-F(\alpha_{ho}) * r_d
= F(\alpha_h) * r_s + (1-F(\alpha_h))^2 * r_d
= F(\alpha_{ho}) * r_s + (1-F(\alpha_{ho}))*(1-F(\alpha_{hi})) * r_d
= f(\theta_o, \theta_i)$

and thus fulfill BRDF requirements. The microfacet angle is used for the specular term, therefore it is the more sensible thing to interpolate between specular and diffuse component (ignoring the fact of two transmissions for this argument).
Instead, [PBR, p.113] and [DIS, p. 14] put the $\theta_h$ into a roughness calculation and leave that rather unexplained.

Additionally to my confusion about this, in the explanation slides [PBR, p.187] uses the dot product $\omega_h * \omega_o$ (and therefore the $\alpha_{ho}$ angle) and later on [PBR, p. 191] also the dot product $\omega_h *  \omega_i$ ($\Rightarrow\alpha_{hi}$).

I finally figured out a flaw in my argumentation to use the half vector for the diffuse part.
tl;dr version:
$\alpha_{hi}$ and $\alpha_{ho}$ are not equal, this assumption only works for the specular part. Therefore the energyconservation is not given.
More correct: Per definition $\alpha_{hi} = \alpha_{ho}$, but you are not allowed to use them in the equation, you need to use the microfacet normals instead of the half vectors. The microfacet normal and the half vector are however not generally equal.
long version:
The diffuse BRDF (like the specular one) is an integration over all possible microfacet normals. The generalized BRDF form is:
(1)
$f = \int_{\Omega} \frac{\rho_m * D * G * \cos(\alpha_{hi}) * \cos(\alpha_{ho})}{ \cos(\theta_i) * \cos(\theta_o)}$
where $\rho_m$ is the behaviour of light when it reaches a microfacet, $D$ is the normal distribution function, $G$ is the geometry function and the rest is normalization [PBR, p.16-25][HEI, p.61f.]
Now for the specular BRDF, the behaviour is perfect reflection. This means, only microfacets where the microfacet normal $\omega_m$ equals the half vector $\omega_h$ between light ($\omega_i$) and view ($\omega_o$) direction reflect light from the light direction to the view direction. Mathematically, this translates to a Dirac Delta function $\delta(\omega_h)$. The amount of light reflected is determined by Fresnel $F$. With accounting for integration domain changes[PBR, p.31-41], this leads to ($\rho_{m,s}$ being the specular specific reaction of microfacets for the specular part):
(2)
$\rho_{m,s} = \frac{\delta(\omega_h) * F }{4*\alpha_{hi}*\alpha_{ho}}$
Plug this into (1) and you integrate over any $\omega_m$, but the dirac delta function makes you ignore anything where $\omega_m \neq \omega_h$. This is the nice thing about those dirac delta functions, the "cancel" the integral. Therefore, you come to the well known BRDF [PBR, p.43]
(3)
$f_s = \frac{F * D * G}{4 * \cos(\theta_i) * \cos(\theta_o)}$
and here you can savely assume $\alpha_{hi} = \alpha_{ho}$. The Fresnel part makes this one side of a linear interpolation (seeing as this is the ratio of light being specularly reflected).
On the other side though, you still have an integral and you can't "cancel" it with anything. Let $\theta_{mi}$ be the angle between the microfacet normal and the light dir, $\theta_{mo}$ between the microfacet normal and the view dir (as opposed to $\alpha_{hi}$ and $\alpha_{ho}$, which specifically are defined for the half angle!).That means 2 important things:
(4)
$\theta_{mi} \neq \alpha_{hi} = \alpha_h = \alpha_{ho} \neq \theta_{mo} \forall \omega_m \neq \omega_h$
and $\rho_{m,d}$ (as the diffuse reaction of light when reaching microfacets) is depended upon Fresnel, it can only use non reflected parts, but you cannot just assume EVERY microfacet normal diffusley reflects all the light that was not specularly reflected (i.e. $(1-F(\alpha_h))$).
So the problem now is, you have one side of that linear interpolation being the analytically solvable specular BRDF without an integral. On the other side, you have the interpolation factor within the integral, and thus you would use it for any microfacet normal. This is not energy conserving anymore, if you model $\rho_{m,d}$ with a lambertian term.
If you manage to pull the $1-F$ part out of the integral, then and only then can you use the half vector for both specular and diffuse BRDFs, otherwise you need to handle it differently.
If you still want to use the half vector for the "Fresnelinterpolation", then you need to normalize.
$f = F(\alpha_h) * r_s + (1-F(\alpha_h)) * r_d * k$
With $k$ being chosen, such that
$r_d * k = 1$
Note that $r_d$ still includes an integral over all possible microfacet normals.
Using $\theta_i$ and $\theta_o$
I guess I should add a sentence about the actually used angles, because the question I originally posed was not just "why not $\alpha_h$" but also "why $\theta_i$ and $\theta_o$". So for the poeple wondering if this is correct: No. It is just an approximation.

Instead, we can modify the matte term to be a simple approximation that captures the important qualitative angular behavior[...].
[SHI, p.46]

Discussion on why this is so hard to figure out
Now [DIS, p.14] and [PBR, p.100,184] quoted [SHI] and [ASH] on this, where [ASH] uses [SHI], so it boils down to the model of the latter. And seeing as [DIS] and [PBR] are non-scientific (though pretty well thought through) publications, I can't fault them too much, but really, a better way of quoting papers or an extra explanation might have helped and saved me about 2 weeks of thinking, reading and almost starting a paper on why everybody is (supposedly) wrong.
Furthermore, neither of them really explains their diffuse approximations, [PBR, p.193] just mentions trying different functions until he saw one he liked.
Additionally, [SHI, p.46] actually quoted [SCH, p.10f.] who used the linear interpolation between $F(\alpha_h)$ and $1-F(\alpha_h)$. However, [SHI, p.46] quoted him incorrectly, they said he used $F(\theta_i)$ and $1-F(\alpha_h)$. Assuming this, they are of course right in thinking [SCH] is not energy conserving, but this lead me to believe that $F(\alpha_h)$ and $1-F(\alpha_h)$ would be. Their mistake might've come from them using $F(\theta_i)$ themselves, I can't say.
Literature

[ASH] An anisotropic phong BRDF model, in Journal of Graphics Tools Vol. 5, No. 2, Michael Ashikhmin and Peter Shirley, 2000
[SCH] An Inexpsensive BRDF Model for Physically-based Rendering, in Computer Graphics Forum Vol. 13, No. 3, Christophe Schlick, 1994
[SHI] A Practitioners' Assessment of Light Reflection Models, in The Fifth Pacific Conference on Computer Graphics and Applications, 1997. Proceedings, Peter Shirley,  Helen Hu, Brian Smits and Eric Lafortune, 1997
[DIS] Physically-Based Shading at Disney, at SIGGRAPH 2012 Course: Physically Based Shading in Film and Game Production, Brent Burley, 2012
[HEI] Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs, in Journal of Computer Graphics Techniques Vol. 3, No. 2, Eric Heitz, 2014
[PBR] PBR Diffuse Lighting for GGX+Smith Microsurfaces, at GDC2017, Earl Hammon Jr., 2017


2D array texture with different filtering setting per texture? Currently I have textures which need GL_NEAREST and some other need GL_LINEAR in the same texture2dArray. In the shader I perform a manual GL_NEAREST lookup via textureFetch().

To avoid this my question is, is it possible to set the value of GL_TEXTURE_MIN/MAG_FILTER for each texture in a texture array to different values?

I know of the same size restrictions, but could not find anything regarding mipmapping.

Array textures do not contain textures; they are a single texture that contains a number of images, with each image being of the same size, format, mipmap depth, etc.

As such, the sampling parameters that are part of the texture apply to sample fetches for all images within the texture.

That having been said:

  In the shader I perform a manual GL_NEAREST lookup via textureFetch().

This means your FS knows when to look the texture up with linear filtering and when not to. As such, you could have two different array texture samplers in the shader, one of which you expect to use linear sampling and one which uses nearest.

Both of these samplers can be using the same texture object, just paired with a different sampler object (which contains all of the filtering parameters for the texture). One sampler would use nearest filtering, the other would use linear.


How can we convert a probability density according to solid angle to a density according to area? In Chapter 14.5.3 of Physically Based Rendering - From Theory to Implementation (3rd edition), the authors claim that a probability density $p_\omega$ according to solid angle can be converted to a density $p_A$ according to area by $$p_A=p_\omega\frac{\left|\cos\theta_i\right|}{\left|p_i-p_{i+1}\right|^2},\tag1$$ where $\theta_i$ is the angle between the surface normal at $p_{i+1}$ and the vector to $p_i$.

  The problem is: $(1)$ doesn't make sense. If $p_\omega$ is a density with respect to the surface measure on the upper unit hemisphere $\Omega_{p_i}$ over $p_i$, then $p_\omega$ is by definition a function $\Omega_{p_i}\to[0,\infty]$. Analogously, if $p_A$ is supposed to be a density with respect to the surface measure on the manifold $M$ build by all surfaces in the scene, then $p_A$ is by definition a function $M\to[0,\infty]$.

  So, there's at least an argument transformation missing. I would really appreciate, if anybody could clarify.

I don't understand what's the problem. We want to make a change of variables so that instead of integrating over the range of directions $d\omega_i$ in the upper hemisphere we integrate over patches of area $dA$ of a region $R$.

This whole thing is given in section 5.5 as Hubble said but let me post it again.

We know by definition that,

$\omega = A *  cos(\theta)/r^2$

Now if you differentiate this, you get

$d\omega = dA * cos(\theta) / r^2$

where,

$Jacobian = cos(\theta) / r^2$

as Hubble said. I don't know what are you trying to do by calculating Jacobian through determinant but as far as I know, that Jacobian determinant is for multi-variables and involves partial derivatives. For change of variables involving a single variable the Jacobian can be gotten by simply derivating as I did.

Now this might seem the inverse but wait a moment. What you posted in the question is a relation between the PDFs.

Consider a single sample of the rendering equation integrated over the domain of $\omega_i$

$L_s = BRDF * L_f * (\omega_i . n) / p(\omega)  $

After changing the domain by above equations this would become

$L_s = BRDF * L_f * (\omega_i . n) (\omega_i . n_r)\, / \,  (p(A) * r^2)  $

$L_s = BRDF * L_f * (\omega_i . n) * JN\, / \,  p(A) $
where $Jacobian = JN$

This means,
$ 1 / p(\omega) =  JN/p(A)$

$ p(A) = p(\omega) * JN$

$ p(A) = p(\omega) * cos(\theta)/r^2$

This is what you see above. Simple as ****.


What modern real-time rendering applications still use fragment shader depth writes / late-Z? It is pretty well known that writing depth in your fragment shader generally disables "early Z" test/write optimizations that modern GPUs rely on for fastness. Nonetheless, APIs still allow it and GPUs still provide support for correct visibility when shaders write arbitrary depth.

The question is: are there any state of the art real-time effects in today's game engines that require writing Z in the fragment shader?

Any techniques that involve raytracing in the fragment shader might want to write Z in order that the depth buffer contain an accurate representation of the raytraced surface. For example:

Distance-field ray marching, as seen in many shadertoys and demoscene productions. Here, only a full-screen quad gets rasterized, and all the geometry is generated by the fragment shader. Writing Z would be necessary if you want to use the depth buffer for deferred lighting or post-effects, or to composite distance fields with ordinary triangle geometry and get correct occlusion.
Voxel ray marching, similar to the above. Even when the voxels are translucent (as in a smoke cloud or suchlike), it may be useful to write Z when the voxel density is high enough to become opaque; for instance, that can be useful for later motion blur / depth-of-field calculations.
Parallax occlusion mapping techniques, in which a flat surface is given the appearance of depth by ray-marching against a heightfield.

There are probably other similar cases where the shape of a surface is defined by the fragment shader rather than by the rasterized triangles.

By the way, newer APIs and GPUs include support for a conservative depth output mode, in which the shader-written Z is only allowed to be greater-equal to the geometric Z, but not less. This allows early-Z / hierarchical-Z optimizations to still work based on the geometric Z: if the rasterized geometry gets culled for being behind something, then you know the shader-written Z would have been culled as well. This is a good fit for cases like the above, as you can rasterize some bounding geometry for the distance-field / voxel / heightfield object; then the actual depth will be greater-equal to that of the bounding geometry.


glBindImageTexture returning INVALID_VALUE I am currently having an issue with OpenGL as I am trying to load some vector information into a 3D texture. Currently, I am developing in OpenGL 4.6 which should support Image bindings.

this my code to bind the texture to an image:

glGenTextures(1, &field_sbo);
glBindTexture(GL_TEXTURE_3D, field_sbo);
glTextureStorage3D(field_sbo, 1, GL_RGB32F, 10,10,10);
glTextureSubImage3D(field_sbo, 0, 0, 0, 0, 10, 10, 10, GL_RGB, GL_FLOAT, vector_data);
glBindImageTexture(0, field_sbo, 0, GL_TRUE, 0, GL_READ_ONLY, GL_RGB32F);

Additionally, this is the declaration in the compute shader:

layout(binding = 0, RGB32F) uniform image3D field;

From my understanding, the glTextureStorage3D should make the texture immutable, however I keep getting an INVALID_VALUE at the glBindImageTexture stage. I have looked at the documentation for the errors however I cannot seem to fix the issue.

Outside of GL_R11F_G11F_B10F, 3-channel formats are not allowed for image load/store. And you'll note that rgb32f is not in the list of valid format qualifiers for GLSL variables, nor are any other 3-channel formats (except r11f_g11f_b10f).


Is all grid based noise inevitably anisotropic? I'm interested in how this applies to higher numbers of dimensions too, but for this question I will focus solely on 2D grids.

I know that Perlin noise is not isotropic (direction invariant), and that the underlying square grid shows up enough to be able to identify its orientation. Simplex noise is an improvement on this but its underlying equilateral triangle grid is still not completely obscured.

My intuition is that any attempt to make noise of a particular frequency on a grid will result in a lower frequency in directions not aligned to the grid. So while attempts can be made to disguise this, the noise cannot in principle be isotropic unless it is generated without reference to a grid, allowing the average frequency to be the same in all directions.

For example, with a square grid without noise, with square side length $n$, the frequency of vertices horizontally or vertically is $\frac1n$, whereas the frequency of vertices at 45 degrees (through opposite corners of the squares) is $\frac1{\sqrt{2}n}$.

Is there a random distribution that could be applied to offset the vertex positions that would result in the frequency becoming identical in all directions? My suspicion is that there is no such distribution, but I don't have a way of proving either way.

In short, is there a way of making perfect grid based noise of a given frequency, or should I be focused on other approaches (non-grid based noise or ways of disguising artifacts)?

As usual with numerical methods and samplings, it also depends of your quality threshold of what you consider "isotropic". And of what you would consider as a being or not a "grid-based noise algorithm".
For instance Gabor Noise reproduces a target spectrum, for instance blue noise, which in Fourier domain is a simple isotropic ring.
Now if you consider that this ring is not analytic but rasterized, as such it is not perfectly symmetrical. Also if the ring radius (i.e., frequency) get too close to the window size (i.e., maximum frequency), it will be truncated (and thus no longer symmetrical). Up to you to accept or not these as anisotropic ;-)

"This is not a circle" - Magritte . . . . . . . . . . . . . . . .   "This is not a circle" - Nyquist
You might or might not accept a rasterized ring in Fourier space to be "isotropic". Still, in the extreme cases where the ring get thinner than the resolution, or larger than the window, isotropy is objectively lost.


Determining Vanishing Points of edges of Triangle using given Point Due to perspective foreshortening, the projections of parallel lines (1D geometric objects) meet in their vanishing point (a 0D geometric object). Higher dimensional geometric objects also has higher dimensional vanishing objects. I am interested in vanishing points of the edges of the triangle

∆(p0, p1, p2)

if p0T,p1T and p2T are given ?

Any idea ?

Vanishing points are "points at infinity" in projective geometry, which are represented by $w = 0$ in homogeneous coordinates. You can construct the vanishing point of a ray or line by taking its $xyz$ direction vector and adjoining $w = 0$. (For a line, use the direction vector and its negation to get vanishing points in both directions.)

Then you can transform this homogeneous vector by your view/projection matrices as usual, and divide out $w$ from the result to find where the point lies in screen space. (If the result has $w \leq 0$ then it's behind the camera or off to the side.)


How does MSAA performance scale? I'm finding myself increasingly interested in rendering techniques utilizing multisampling for anti-aliasing (obviously), out-of-order transparency, volumes etc.

Searching the web for performance numbers of MSAA in the range only gets me to gamer verdicts for this and that game, and even then MSAA is only used for AA, and being compared to screen based post process AA which can hardly be used for anything else than AA.

Is there any rule of thumb (other than memory consumption) of how much it costs having x2, x4, x8, x16 MSAA? Linear? Logarithmic?

Roughly speaking, what the Supersampling Anti-Aliasing method does is:

For each pixel in the image to render

"Split" the pixel of a target image, usually in 2, 4, 8 or 16.
Actually this step is more: pick the 2, 4, 8 or 16 locations ("samples") inside the pixel
Compute the color that the image will have at each of those locations (sample or "split part").
So instead of only computing the color that the image will have in that single pixel, we compute 2x, 4x, 8x or 16x times a similar value
Compute the average of all the 2, 4, 8 or 16 sample colors to decide the final color or the initial pixel

You can see that Super-sampling cost is proportional to the cost of computing one pixel's color in the image. Therefore it depends on the cost of shading.
It also depends on the number of pixels, i.e the resolution of the image.

In the context of Ray-Tracing applications, this means that 2, 4, 8 or 16 times more rays will have to be computed.

In the context of GPU-accelerated graphics computation, this means that 2, 4, 8 or 16 times more pixel shaders ("fragment programs") will be executed.

Now roughly speaking, what the Multisampling Anti-Aliasing method does is:

For each pixel in the image to render

Pick 2, 4, 8 or 16 locations ("samples") inside the pixel

For each primitive (triangle) in the scene (actually, only for those that overlap the pixel):

Determine which of these samples are covered by the current triangle
If there is at least one such sample, compute a single color of the contribution of that triangle to the pixel, and store it for each sample covered
The worse case happens if each sample is overlapped by a different triangle ;  in that case the color must be computed for each triangle

Compute the average of all the gathered sample colors to decide the final color or the initial pixel

You can see that evaluating the performance of Multisampling Anti-Aliasing is more complicated than that of Supersampling, since it also depends on the topology of the scene (the size and shape of the triangles).


Closed form solution to a problem of tangent circles I have two circles, that I know are both tangential to a third (unknown) circle. I also know what quadrants of the circles the tangential points are in. It is as pictured below. I also know the arc angle that the third circle spans between the two tangential points (beta).

I would like to know the location of the two tangential points, as well as one of the tangential directions, so I can draw the unknown (red) circle.

I currently have a working solution for this involving a nonlinear equation solver. If at all possible I would like to have a solution that involves no non-linear equation solving, because I need to run very many of these problems and I need precise results.

First let us solve the radius of your circle. You can see that the centers of the circles form a triangle. There are 8 possible triangles, of which 4 are mirror solutions across the line that connects the centers of your original circles.  The other solutions are permutations of inside tangent and outside tangent connections.

Image 1: All possible solutions

So knowing the quadrant is hardly enough for the choice of solution.

Solving the triangles

So for eaxmple let the circle be on the inner outside of both circles (the sign changes for the other solutions). Now you need to solve the equation:

$$
 D^2=(x+r_1)^2+(x+r_2)^2-2(x+r_1)(x+r_2)\cos(\beta)
$$

Now this can be solved in a computer algebra system and gives 2 solutions with numerous corner constraints (do it yourself it's important for robust code and obviously you need 4 solutions to check.)

$$
x=\frac{1}{2} \left(\pm\sqrt{\frac{\cos (\beta) (r_1-r_2)^2-2 D^2+(r_1-r_2)^2}{\cos (\beta)-1}}-r_1-r_2\right)
$$

Once you have x solved you can solve the center. This can be done in many ways but I would proceed by solving angle $\alpha$ and some vector rotation, normalization and scalar multiplication. Same calculation can be used to find the intersection point and the tangent is just this vector rotated by 90 degrees.

You could also use an alternate formulation with 2 angles and a vector sum of $a+b-c=0$, with 2 unknowns and 2 equations.


Do straight lines always remain straight when projected with a perspective camera? If I have a straight line in 3D space and I use a camera matrix to do a perspective projection into a 2D plane/screen will the resulting line always be straight too? If I project the start point and end point and join the dots is that equivalent to having projected the line?

I suspect this is true, but have no basis for this assumption.

Is this still true for off-centre perspective projection or a cylindrical projection? I assume this isn't true for real cameras as they have all kinds of non-ideal optical effects right?

"A not so simple approach". I may have messed a little bit too much with grouping the terms, do forgive my elementary math skills, it's a side effect of using tools like wolfram and mathematica too much.

Without loss of generality we can assume that the near plane is given as $z=1$ (that is having normal $(0,0,1)$ and a point on it $(0,0,1)$) and the camera center (the reflected pinhole camera aperture point) is at $(0,0,0)$. Let $p,q \in \mathbb{R}^3$ be two points on a line (non-degenerate, meaning $p\ne q$, otherwise it's just a point and the projection is also a point - unless it's at $(0,0,0)$ then it's undefined). If the line defined by $p,q$ passes through $(0,0,0)$ the projection is a point and the requirement is trivially satisfied.
Now let this not be the case. A perspective projection onto the plane through $(0,0,0)$ is given by $p' = \frac{p}{p_z}$, and $q' = \frac{q}{q_z}$. The line defined by $p,q$ is given as $r(\lambda) = p + \lambda(q-p), \lambda \in  \mathbb{R}$. Its projection is given by $r'(\lambda) = \frac{r(\lambda)}{r_z(\lambda)}$. Since $z=1$ for all projected points, then we need only study the $x,y$ coordinates. We want to show that any projected point $r'(\lambda)$ can be written as an affine combination of $p',q'$ (this defines a line). Taking that into account:
$$p'+\mu(q'-p') = \frac{r(\lambda)}{r_z(\lambda)}$$
$$(1-\mu)\frac{p}{p_z} + \mu\frac{q}{q_z} = \frac{p + \lambda(q-p)}{p_z + \lambda(q_z-p_z)}$$
$$(1-\mu)q_z(p_z+\lambda(q_z-p_z))p + \mu p_z(p_z+\lambda(q_z-p_z))q = q_zp_z(p + \lambda(q-p))$$
$$q_zp_z(p - \lambda p) + q^2_z\lambda p - \mu q_z(p_z+\lambda(q_z-p_z))p + \mu p_z(p_z+\lambda(q_z-p_z))q - q_zp_z(p - \lambda p) - q_zp_z\lambda q = 0$$
$$(1-\mu)\lambda q^2_z p -\mu(1-\lambda)q_zp_z p + \mu(1-\lambda)p^2_zq - (1-\mu)\lambda p_zq_z q = 0$$
$$(1-\mu)\lambda q_z(q_zp - p_zq) - \mu(1-\lambda)p_z(q_zp-p_zq) = 0$$
$$((1-\mu)\lambda q_z - \mu(1-\lambda)p_z)(q_zp - p_zq) = 0$$

For this to be satisfied one of the terms must be $0$. The first case concerns the second term: $p' = q'$, which we already considered as the trivial case. Then we must show that we can always pick a $\mu$ such that $(1-\mu)\lambda q_z - \mu(1-\lambda)p_z = 0$. Solve for $\mu$ and you get: $\mu = \frac{\lambda q_z}{(1-\lambda)p_z + \lambda q_z}$. Thus I have not only found that it is a straight line, but also the relationship between the parameterizations.

In general a projection onto a different 2-manifold (not a plane) will not result in the lines remaining straight (think of a sphere or a cylinder).


glCompressedTextureSubImage3D Usage When using an array texture I can call:

glTextureStorage3D(m_texture_array, 1, GL_COMPRESSED_RGB_S3TC_DXT1_EXT, width, height, m_num_billboards);
for (int i{}; i < 4; ==i) {
    glTextureSubImage3D(m_texture_array, 0, 0, 0, i, width, height, 1, GL_RGB, GL_UNSIGNED_BYTE, data);
}

Which results in a compressed image. So when do I need to use glCompressedTextureSubImage3D ?

Furthermore, the docs say:

  internalformat must be a known compressed image format (such as GL_RGTC) or an extension-specified compressed-texture format

Why can't it be a generic format?

OK, when you call the regular SubImage functions, you are passing pixel data. You are provoking a pixel unpack operation, where the implementation will read the pixels from your data in the format specified and convert them to the internal format of the image.

However, if already have compressed data, you don't want that conversion. You just want to pass the compressed data to the GL, and have it use it directly. That's what the Compressed functions are for.

If you want to compress uncompressed data, you use the non-Compressed functions. But you really, really, shouldn't. Indeed, with ASTC, you can't compress the data online. And even for other formats, the quality of the result will be rather sketchy.

Do you really want to rely on the quality of a compression algorithm written to be fast rather than good?

The format and type transfer parameters describe a single pixel's worth of data, and compressed formats don't generally store data in units of "pixels". Compressed formats usually use blocks of some size. Regular pixel transfers expect a 1:1 correspondence between a pixel in your data and a pixel in the GL-owned image. That simply isn't the case when dealing with compressed data.

So you need a new function, which just takes a block of bits of a known size, whose format is defined by the image's format, and jams it into the image.

The reason why you can't use a generic compressed format with the Compressed transfer functions should be obvious by now. These functions are for providing compressed data in a specific format; by definition, the generic formats are implementation defined. So there's no way you could possibly know what data to provide for generic compressed transfers.


Is this the correct way to implement Beer's Law? When I implement Beer's law (color absorption over distance through an object), it never looks very good for some reason.

When i have the color behind the object, I calculate the adjusted color like this:

const vec3 c_absorb = vec3(0.2,1.8,1.8);
vec3 absorb = exp(-c_absorb * (distanceInObject));
behindColor *= absorb;

That will give me something that looks like this (note a little bit of refraction applied):

And here it is without refraction:

Note that this is implemented as a shader toy here.

That fulfills the description of what Beer's law does, but it doesn't look very good, not when compared to shots like this:

Specular highlights aside, I'm trying to figure out the difference.  Could it just be that my geometry is too simple to really show it off very well? Or am I implementing it incorrectly?

Your image definitely does not look correct, and it appears that you are not correctly computing the internal path of light rays as they travel through your mesh. From the looks of it, I would say that you are computing the distance between the point where the view ray first enters the cube and where it first hits the interior wall, and using that as your absorption distance. This essentially assumes that light will always exit the glass the first time it hits a wall, which is a poor assumption.

In reality, When light enters glass from air, it often does not immediately exit the glass. This is because when the light strikes the glass/air interface, a phenomenon known as total internal reflection (TIR) can occur. TIR occurs when light travels from a medium with a higher index of refraction (IOR) to one with a lower IOR, which is precisely what happens in the case of light hitting the interior wall of a glass object. This image from Wikipedia is a good visual demonstration of what it looks like when it occurs:

In basic terms, what it means is that if the light hits at a shallow angle, the light will completely reflect off of the interior of the medium. To account for this, you need to evaluate the Fresnel equations every time that your light ray hits a glass/air interface (AKA the interior surface of your mesh). The Fresnel equations will tell you the ratio of reflected light to the amount of refracted light, while will be 1 in the case of TIR. You can then compute the appropriate reflected and refracted light directions, and continue to trace the light's path either through the medium or outside of it. If you assume a simple convex mesh with a uniform scattering coefficient, then the the distance to use for Beer's law will be the sum of all internal path lengths before exiting the medium. Here is what a cube looks like with your scattering coefficients and IOR of 1.526 (soda lime glass), rendered using my own path tracer that accounts for both internal and external reflections and refractions:

Ultimately the internal reflections and refractions are a major part of what makes glass look like glass. Simple approximations really don't cut it, as you've already found out. It gets even worse if you add multiple meshes and/or non-convex meshes, as you not not only have to account for internal reflections but must also account for rays that leave the medium and enter it at a different point.


Adding noise to gradient I found this picture

When you zoom in to any part of the image, the background looks a bit noisy/grainy. See pixels around "violent violet" label:

I'm trying to understand how to achieve the same effect programmatically?

I was trying to change pixel color in rgb space by adding tiny gaussian offset, but it still doesn't look nearly as natural as in the image above:

I feel like I'm missing something very simple, but I don't even know where to start? How is this done in general?

Maybe you can play with this jsbin to show me how this is done: https://jsbin.com/fatecojolo/edit?js,output ?

It's generally a good idea to add noise like this when you're using a gradient, to avoid visible banding in the gradient, especially on smartphones. Often smartphone screens claim to be 24-bit colour but the panel itself is actually only 16 or 20 bit.

The chief difference between the example and your attempt to reproduce it is that your noise is in RGB space, while the example only adds luminance noise (i.e. brightness). You could achieve this by using HSV colours and only adding to the V channel, or with RGB colours and adding the same amount to each channel.


Translate vtkCellLocator I would need to translate a vtkCellLocator, without rebuilding it entirely. I suppose that it should be possible to apply an offset to all the buckets in the octree. Do you know if is it possible to do this (or to apply a translation to any other cell locator structure) with the tools already provided by vtk?

In order to translate a vtkCellLocator, you have only to translate a bounding box of root octant. However this bounding box cannot be publicly accessible. So it is necessary to create a new class inheriting from a vtkCellLocator and provide access functions to get and set it directly. The following is a simple example.

#include <vtkTransform.h>
#include <vtkTransformPolyDataFilter.h>
#include <vtkSmartPointer.h>
#include <vtkSphereSource.h>

#include <vtkCellLocator.h>
#include <vtkObjectFactory.h>

class vtkMyCellLocator : public vtkCellLocator
{
public:
  static vtkMyCellLocator *New();

  vtkSetVectorMacro(Bounds,double,6);
  vtkGetVectorMacro(Bounds,double,6);
};
vtkStandardNewMacro(vtkMyCellLocator);

int main(int, char *[])
{
  auto sphereSource = vtkSmartPointer<vtkSphereSource>::New();
  sphereSource->Update();

  // Create the tree
  auto cellLocator = vtkSmartPointer<vtkMyCellLocator>::New();
  cellLocator->SetDataSet(sphereSource->GetOutput());
  cellLocator->BuildLocator();

  double translate[3] = {10.0, 0, 0};
  auto transform = vtkSmartPointer<vtkTransform>::New();
  transform->Translate(translate);

  auto transformFilter = vtkSmartPointer<vtkTransformPolyDataFilter>::New();
  transformFilter->SetTransform(transform);
  transformFilter->SetInputConnection(sphereSource->GetOutputPort());
  transformFilter->Update();
  cellLocator->SetDataSet(transformFilter->GetOutput());

  double bounds[6];
  cellLocator->GetBounds(bounds);
  for (int i = 0; i < 3; ++i)
  {
    bounds[2*i] += translate[i];
    bounds[2*i+1] += translate[i];
  }
  cellLocator->SetBounds(bounds);

  double testPoint[3] = {2.0, 0.0, 0.0};

  //Find the closest points to TestPoint
  double closestPoint[3];//the coordinates of the closest point will be returned here
  double closestPointDist2; //the squared distance to the closest point will be returned here
  vtkIdType cellId; //the cell id of the cell containing the closest point will be returned here
  int subId; //this is rarely used (in triangle strips only, I believe)
  cellLocator->FindClosestPoint(testPoint, closestPoint, cellId, subId, closestPointDist2);

  std::cout << "Coordinates of closest point: " << closestPoint[0] << " " << closestPoint[1] << " " << closestPoint[2] << std::endl;
  std::cout << "Squared distance to closest point: " << closestPointDist2 << std::endl;
  std::cout << "CellId: " << cellId << std::endl;

  return EXIT_SUCCESS;
}


Error with tube lighting I have recently been working on area lights for a small open source game engine called Urho3D but have run into an issues when rendering tube lights where the lights look the same as sphere lights despite the length value being changed.

I am working from Epic's paper on PBR by Brian Karis. I have tried looking at other sources to see if I could find anything but none of the results fixed the issue I was having.

The method I am using for tube lighting is as follows:

float3 TubeLight(float3 worldPos, float3 lightVec, float3 normal, float3 toCamera, float roughness, float3 specColor, out float ndl)
{
    float3 pos   = (cLightPosPS.xyz - worldPos);
    float3 reflectVec  = reflect(-toCamera, normal);

    float3 L01 = lightVec * LightLengh;
    float3 L0 = pos - 0.5 * L01;
    float3 L1 = pos + 0.5 * L01;
    float3 ld = L1 - L0;

    float distL0    = length( L0 );
    float distL1    = length( L1 );

    float NoL0      = dot( L0, normal ) / ( 2.0 * distL0 );
    float NoL1      = dot( L1, normal ) / ( 2.0 * distL1 );
    ndl             = ( 2.0 * clamp( NoL0 + NoL1, 0.0, 1.0 ) )
                    / ( distL0 * distL1 + dot( L0, L1 ) + 2.0 );

    float RoL0      = dot( reflectVec, L0 );
    float RoLd      = dot( reflectVec, ld );
    float L0oLd     = dot( L0, ld );
    float distLd    = length( ld );
    float t         = ( RoL0 * RoLd - L0oLd )
                / ( distLd * distLd - RoLd * RoLd );

    float3 closestPoint   = L0 + ld * saturate( t);
    float3 centreToRay    = dot( closestPoint, reflectVec ) * reflectVec - closestPoint;
    closestPoint = closestPoint + centreToRay * saturate(LightRad / length(centreToRay));

    float3 l = normalize(closestPoint);
    float3 h = normalize(toCamera + l);

    ndl       = saturate(dot(normal, l));
    float hdn = saturate(dot(h, normal));
    float hdv = dot(h, toCamera);
    float ndv = saturate(dot(normal, toCamera));

    float distL      = length(closestPoint);
    float alpha      = roughness * roughness;
    float alphaPrime = saturate(LightRad / (distL * 2.0) + alpha);

    const float3 fresnelTerm = Fresnel(specColor, hdv) ;
    const float distTerm     = Distribution(hdn, alphaPrime);
    const float visTerm      = Visibility(ndl, ndv, roughness);

    return distTerm * visTerm * fresnelTerm ;
}

If the code snippet is not enough to help solve this issue you can download the full area lighting branch from GitHub. Build instructions can be found on the Urho3D website (cant link due to reputation), note currently area lighting is only tested in DX11. Shader source files can be found under

[build location]/bin/CoreData/Shaders/HLSL

You can see that both images look identical here.

If pos in multiplied into line 40 then it creates more of a tube effect but is still far from correct results, As you can see here.

Found the solution, it turns out the lightVec is not the vector of light from the tube but rather the direction the tube will point. Therefore i will need to pass it a light rotation value to be used there.

Results:


Rotation of a random unit vector at a point in 3D space by an angle I have a scenario where I need rotate a vector ( unit vector) at a point in space by an angle along Z-axis. Is it possible to help me to understand a procedure to achieve this. I'm using vb.net programming to do this programattically.

Here is the schematics for your understanding:

Rotation matrix | wikipedia.org in section that says "in three dimensions". It's the first search result when searching about rotating a vector about an axis. Let me know if you have any questions.

For your specific problem: bring the point P to the origin, then rotate the vector about the Z axis, then bring the point P back to its original location.


Ask for help on understand an algorithm which combines stratified sampling and importance sampling of Monte Carlo I cannot understand an algorithm which combines stratified sampling and importance sampling of Monte Carlo. It is introduced in Page 73 of a textbook "Advanced Global Illumination", 2nd edition, written by Philip Dutre, Philippe Bekaert and Kavita Bala. The text gives almost no explanation on the algorithm so I got stuck at the line $N_i=\lfloor Pn+u\rfloor-N_{sum}$. I tried my best but I think I'm at my wit's end to figure out how this line or the whole psudo-code works. I just don't have any understanding. I read the text again and again and stared at Fig 3.9 above the algorithm for a long time but could not find any clue. Could you please give me an explanation of this algorithm in a way a common human can follow? For your convenience, I copied this section as follows. I know there is another question thread but that question has not been answered and I think my post is more detailed so please do not merge mine to his.

Thanks a lot for your help.

It's difficult to parse, but here's my best understanding.

In this case the goal is determine number of $N_i$ out of $n$ samples to distribute to each strata with probability given by the discrete pdf in $\{p_0, ..., p_n\}$

The summation of $P$ += $p_i$ is computing the CDF for $i$.

I believe "sample the ith term $N_i$ times" means to take a jittered sample within that strata using new random variables.

Let's take a small example where we have the pdf with 4 elements.
$$
PDF = \{0.2, 0.1, 0.6, 0.1\}
$$

And it's corresponding cdf, each element corresponds to $P$ in the ith iteration in the pseudo code.
$$
CDF = \{0.2, 0.3, 0.9, 1.0\}
$$

Now given $\left \lfloor{P*n + u}\right \rfloor$ we can illustrate for a few example cases.

With $u = 0$
$$
\left \lfloor{P*n + u}\right \rfloor = \{0, 1, 3, 4\}
$$
$$
N_i = {0, 1, 2, 1}
$$

With $u = 0.5$
$$
\left \lfloor{P*n + u}\right \rfloor = \{1, 1, 4, 4\}
$$
$$
N_i = {1, 0, 3, 0}
$$

With $u = 0.99$
$$
\left \lfloor{P*n + u}\right \rfloor = \{1, 2, 4, 4\}
$$
$$
N_i = {1, 1, 2, 0}
$$

Notice how for example $N_3$ corresponds to a higher number of samples. When we now sample the 3rd element $N_3$ times we are weighting that strata in proportion to its pdf.

Also, I don't see any reason that the sample count has to be the same $n$ as the # of discrete elements in the pdf.


How to calculate the view vector in Phong Lighting Model? All the books and reference I have read say that the view vector is calculated by subtracting the point where eye is at, from the point where we want to calculate light. But since, eye is at (0, 0, 0) the view vector would be just the negative of the point where eye is at? Is my understanding correct?

EDIT

While the basic principle of vector calculation below is correct, as Sebastián Mestre pointed out, in lighting you would use the vector from the lighted fragment to the eye position. Therefore, you would in fact use $v = -(point - eye) = eye - point = -point$, so you were actually correct in your "negative" assumption. Still (what might have been more a typo than a misunderstanding), it is the negative position of the lighted fragment, rather than the negative eye (which would still be the origin).

As Sebastián also said in a comment to you, you would usually normalize the view vector after you have calculated it. Sebastián is the MVP.

Calculating a vector $v$ from point $A$ to point $B$ is subtracting the values of $B$ from $A$:

$v = \left(\begin{array}{c}x\\y\\z\end{array}\right) = B - A = \left(\begin{array}{c}b_x - a_x\\b_y-a_y\\b_z-a_z\end{array}\right)$

So in your case, you need to subtract the $eye$ point from the point you're calculating light ($pos$) for:

$
v = point -eye
$

Your $eye$ is the origin, so remaining is only the $point$. Therefore: no, it is not the negative position of the lighted point, but the positive position.


Avoiding CPU race condition/cache invalidation of RNG on multi thread path tracing I'm implementing my path tracer using C++ and OpenMP partially based on the design of Small Paint (using a globally available RND function to generate random floats) and PBRTv2 (uses a single RNG across all threads). I notice that due to the fact there can can be multiple threads accessing the same RNG. Race condition and CPU cache invalidation will occur caused by the RNG writing to memory to store/update it's current state. Thus dragging the performance down and mess up the RNG(makes it unpredictable).

How come this is in one of the top grade educational renderer? Or it's just a minor issue that isn't important?
If it's important, is there a known good way to fix this without breaking PBRT's rendering/threading architecture?

You need to implement your own RNG class and not use the clib RNG implementation that shares the state across threads. What I'm using is a simple & efficient multiply-with-carry RNG by George Marsaglia, that I initialize with a seed based on pixel coordinates (e.g. simple x+y*width) that I'm calculating (state per pixel instead of global state)


Energy conservation of BRDF I'm reading Advanced Global Illumination.

Here is the part confusing me:

What do the second equation and $\delta$-function mean?
Why the third equation is a sufficient condition even though a reason is given?

You are right to be confused. What I think they should have written:
$$
L( x \leftarrow \Psi ) = L_{in} \delta(\Psi - \alpha)
$$
using $ \alpha $ instead of $ \Theta $, which is already used as a dummy variable in the integral.

You should look up the Dirac delta function to learn its meaning and properties. In this context, you can imagine the $ L$ above as representing a very concentrated beam (a laser) coming from the angle $ \alpha $. Practically, to do the integral over $ \Psi $ with $ \delta (\Psi - \alpha) $ present in the integrand, remove the integration and replace all occurrences of $ \Psi $ with $ \alpha $. Then it should be clear how they arrive at the next line, which should read, for all $ \alpha $: $$ \int f_r(x, \alpha \rightarrow \Theta) \; \cos(N_x, \Theta)\; d \omega_\Theta \leq 1 $$.

The fact that this condition is sufficient follows from two facts: 1. that any function (e.g. $ L $) can be approximated by a sum of many $ \delta $ functions, and 2. that everything is linear. In other words, if I write $ N(L) $ and $ D(L) $ for the numerator and denominator of the left hand side of (2.21), then you can see that if $ L = L_1 + L_2 $, then $ N(L_1+L_2) = N(L_1) + N(L_2) $ and $ D(L_1+L_2) = D(L_1) + D(L_2) $. So if I know $ N(L_1) \leq D(L_1) $ and $ N(L_2) \leq D(L_2) $, then I know $ N(L) \leq D(L) $ since $$ N(L) = N(L_1) + N(L_2) \leq D(L_1) + D(L_2) = D(L) $$.


Shader uniform performance I'd like to know if there's any performance impact based on how I struct my shader uniforms. For instance, is passing 4 floats worse than passing a single vec4?

In any modern desktop hardware, there shouldn't be: uniform buffers will just be memory blobs read by the shader. (See also: Uniform Buffer Objects)


Is placing z value of vertex in w enough to achieve perspective projection in OpenGL? I have been learning about projection matrix in OpenGL and I am finding it really hard to understand. All I know is that there are other configurations in the projection matrix besides placing z in w for perspective projection. What are those other numbers and why are they there?
Thanks.

The projection matrix distorts the view frustum (the volume the camera can see) into a unit cube. So everything with all coordinates in the range -1 to 1 after projection is potentially visible, and everything else can be clipped.

To make things further away look smaller, we need to divide by their Z distance. We can't put this value directly into the matrix since it is different for each point we want to project, so instead we place each point's Z into its projected W, giving (X, Y ,Z ,1) -> (X', Y', Z', Z), known as clip coordinates. This is then divided by the new W to give (X'/Z, Y'/Z, Z'/Z, 1),  known as normalized device coordinates. On the GPU this division happens automatically after the vertex shader and before clipping and rasterization.

The general form of a projection matrix is

A 0 0 0
0 B 0 0
0 0 C D
0 0 1 0

The A and B terms relate to the field of view, higher values give a narrower view. Specifically, B is the cotangent of half the vertical view angle, and A is B/aspect ratio.

The C and D terms are bias and scale terms which map the near plane to -1 (0 in DirectX), and the far plane to 1. These are used for writes to the depth buffer to determine which objects occlude others. The depth mapping is actually a function of 1/Z, rather than Z, since this gives better precision close to the near plane, and allows perspective-correct interpolation.

Also note that in OpenGL, W' is -Z rather than Z, since we are using a right-handed coordinate system, with negative Z pointing into the screen.


What is ambient lighting? Wikipedia says:

  An ambient light source represents a fixed-intensity and fixed-color light source that affects all objects in the scene equally.

By saying "affects all objects equally" they mean all objects get the same amount of light? So if you have three houses in your scene, you would need to calculate a specific position for the ambient lighting source, in order that every object gets the same amount of light? If not, you would not have an ambient light source?

What is the difference between "normal" lighting from the sun and ambient lighting?

Traditional rendering solutions do not do account for secondary light bounces (called indirect light). Even with strategically placed fill lights you still have areas where none of the direct light hits.

Ambient light tries to solve this problem by shining by a constant amount in all directions. In practice this means that light position or surface normal has no meaning, one just adds some of the shaders color multiplied by ambient light color to the shading result.

Ambient light has a tendency to look artificial when overused. But the opposite problem is that surfaces look like they are on outer space. Ambient light also makes the difference between dark materials and light materials more apparent.

Image 1: Image without ambient light (left) looks like it was shot in space. Image with ambient (right) looks more natural, although possibly a bit flat if overused.

The real problem is that ambient light does not really exist. Even if youd argue that its useful model, it certainly is not uniform. Its just a quick fix. Therefore all kinds of tricks, like ambient occlusion, have been proposed to enhance the quality of ambient light.


Shader that transforms a mercator projection to equirectangular? I am new in the OpenGL - GLSL and trying to make a shader that taking a mercator projection texture as a source and converting it to an equirectangular projection texture.

Sample input: (1024x1024)

Desired output (1024x512)

I found this question, that have a similar problem in unity3d. But i stuck to understand it.

I have a C# code that do this task, using WindowsForms, but it's not efficient.

static class Program
{
    const double TWO_PI = Math.PI * 2;
    const double PI180 = Math.PI / 180;

    private static double Clip(double value, double min, double max)
    {
         return Math.Min(Math.Max(value, min), max);
    }

    public static PointF PixelXYToLatLong(int X, int Y, int mapSize)
    {
         PointF coordinate = new PointF();
         double x = (Clip(X, 0, mapSize - 1) / mapSize) - 0.5;
         double y = 0.5 - (Clip(Y, 0, mapSize - 1) / mapSize);
         coordinate.X = (float)(360 * x);
         coordinate.Y = (float)(90 - 360 * Math.Atan(Math.Exp(0 - y * 2 * Math.PI)) / Math.PI);
         return coordinate;
    }

    static void Main()
    {
        Bitmap source = new Bitmap("INPUT_PATH");
        int w = source.Width;
        int h = source.Height;
        int dh = h >> 1;
        Bitmap des = new Bitmap(w, dh);
        dh = dh >> 1;
        int dw = w >> 1;
        double r = w / TWO_PI;
        for (int i = 0; i < w; i++)
        {
            for (int j = 0; j < h; j++)
            {
                Color color = source.GetPixel(i, j);
                PointF point = PixelXYToLatLong(i, j, w);
                double x = point.X * PI180;
                double y = point.Y * PI180;
                x = r * x;
                y = r * y;
                x = Math.Round(dw + x);
                y = Math.Round(dh - y);
                des.SetPixel((int)x, (int)y, color);
            }
        }
        des.Save("OUTPUT_PATH");
        source.Dispose();
        des.Dispose();
    }
}

So i try to implement it in c++ and OpenGL.

My vertex and fragment shaders

const GLchar* vertexSource = R"glsl(
    #version 330 core

    layout (location = 0) in vec3 texPos;
    layout (location = 1) in vec2 texCoord;

    out vec2 TexCoord;

    void main()
    {
        gl_Position.xyz = texPos.xyz;
        gl_Position.w = 1.0;

        TexCoord = texCoord;
    }
)glsl";

const GLchar* fragmentSource = R"glsl(
    #version 330 core

    in vec2 TexCoord;

    uniform sampler2D ourTexture;

    void main(){
        float pi180 = 3.14159265 / 180;
        float r = 1 / (2 * 3.14159265);
        float x = min(max(TexCoord.x, 0), 1) - 0.5;
        float y = 0.5 - min(max(TexCoord.y, 0), 1);
        x = 360 * x;
        y = 90 - (360 * atan(exp(0 - y * 2 * 3.14159265)) / 3.14159265);
        x = x * pi180;
        y = y * pi180;
        x = 0.5 + (r * x);
        y = 0.5 - (r * y);

        gl_FragColor = texture(ourTexture, vec2(x,y));
    }
)glsl";

And my vertex buffer:

static const GLfloat g_vertex_buffer_data[] = {
   1.0f,  1.0f, 0.0f, 1.0f, 0.0f,
   1.0f, -1.0f, 0.0f, 1.0f, 1.0f,
  -1.0f, -1.0f, 0.0f, 0.0f, 1.0f,
   1.0f,  1.0f, 0.0f, 1.0f, 0.0f,
  -1.0f,  1.0f, 0.0f, 0.0f, 0.0f,
  -1.0f, -1.0f, 0.0f, 0.0f, 1.0f
}

Output:

As you can see there are a lot of problems.

Also i try to change texture wrapping to GL_REPEAT, GL_MIRRORED_REPEAT, GL_CLAMP_TO_EDGE and GL_CLAMP_TO_BORDER, as below:

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

And glViewport(0, 0, 512, 256) but without any success.

So where I'm wrong?

Sorry for the long question, Thanks in advance.

This question and its answer, actually answer my question. I had to read its answer carefully and find my mistake.

Now my fragment shader look like this:

const GLchar* fragmentSource = R"glsl(
    #version 330 core

    in vec2 TexCoord;

    uniform sampler2D ourTexture;

    #define PI       3.141592653589793238462f
    #define TWO_PI   6.283185307179586476924f
    #define PI4      0.7853981633974483096155f

    void main(){
        float lat = (TexCoord.y - 0.5f) * PI;
        if(lat >= -1.48442222974871 && lat <= 1.48442222974871){
            float y = log(tan(PI4 + (lat / 2.0)));
            y = (y + PI) / TWO_PI;
            gl_FragColor = texture(ourTexture, vec2(TexCoord.x, y));
            return;
        }
        gl_FragColor = vec4(1, 1, 1, 0);
    }
)glsl";

And it's works perfectly.


Is this smaller grid for Poisson disc sampling still correct? I've seen a number of 2D Poisson disc sampling algorithms online that use a grid to accelerate checking for existing points within the minimum radius [![r][r image]][r link] of a candidate point. For example:

Dev.Mag
mbostock

They use a grid of squares of side $\frac{r}{\sqrt2}$, which is the same side length that I intuitively came up with when implementing this myself.

I can see the reason - that is the largest square that cannot contain more than 1 point (assuming the minimum is not attainable - the distance between two points must be strictly greater than $r$).

However, having thought about it further, I adjusted the grid size to $\frac{r}2$ instead. This finer grid means 4 additional squares need to be checked (the 4 corner squares are now within the radius), but the total area covered by the required squares is less, so that on average fewer points will need to go through the Euclidean distance check. The difference can be visualized using the same style as the diagram in the first linked article.

For a candidate new point, existing points must be checked in all squares that are within a radius $r$ of the corners of the candidate's square. Here the two grid sizes are shown side by side, to scale, for the same radius $r$. This shows clearly that a significantly smaller area is being checked. Each square is exactly half the area of the previous approach, and even if the 4 outer corner squares are excluded in the previous approach (left image), this still gives an area $2 \cdot \frac{21}{25} = 1.68$ times larger than in the new approach.

My main question is this: Is this approach still correct, and does it give identical results?

I'm also interested to know whether there is any reason to favor the $\frac{r}{\sqrt2}$ approach. Using $\frac{r}{2}$ seems more efficient in time, which seems worth the cost in space efficiency. Is there anything I'm missing?

Images produced with this jsfiddle (in case I need to edit them later).

You approach will work, and in general any square of size <  will work fine because the invariant "at most 1 point per square" is valid. Extrapolating from your idea, it means that one should get the best reduction in area checks for infinitesimally small squares. But we don't want to do that, right?

There are two reasons why  is usually better than :

How much state do you want to manage? Each square can either be "occupied" or "unoccupied" and contains a pointer to its sample point (if any).

For very large domains with small r, storage costs can be hard to manage. The problem is much worse if you're working with many dimensions. See Gamito et al.'s paper, Figure 12. Even in 4D you can be severely storage limited.
In your proposed grid size, each candidate points checks 24 squares, as opposed to 20 in the usual method. There is a per-square cost (e.g. checking whether square is occupied and fetching sample point if it is) which increases the overall cost in your method.

Basically, your method works, but it is usually not more efficient.

However, the state of the art in unbiased Poisson-disk sampling techniques actually successively reduces the grid size as the uncovered area reduces. In addition to Gamito et al.'s paper, also see Ebeida et al.'s paper.

P.S. Sorry, I don't know how to do math formatting in Stack Exchange.


Why is precalculating a distortion different from calculating it in the shader? I have an application where I simulate a camera distortion. There are two ways to go about it: calculate the distortion per pixel in the shader or precalculate the distortion and store the distortion as a 2D texture. In either case I first render the undistorted image, then calculate different texture coordinates in a new render pass for every pixel and thus read from the original image at a different position.

In the latter case I store tex coords, meaning that for my texture coordinate $(u_d, v_d)$ in the shader I read out the texture coordinate $(u, v)$ and thus have a distortion from the original point $(u, v)$ to my distorted point $(u_d, v_d)$.

I can set up my distortion to have no distortion at all (i.e. $(u, v) = (u_d, v_d)$). This works fine when I calculate the distortion in the shader, but if I precalculate the distortion and store my new tex coords, read them out in the shader and then apply that "distortion", I get a somewhat blurry image. I have checked my texture with the new texture coordinates, but they seem to store exactly what I want them to (basically their own position).

Here is the same excerpt from my results, first with the shader calculated distortion:

And then with the precalculated distortion and blurry result:

I'm a bit at a loss as to why I get this blurry image now, especially when there is actually no offset between the original and the distorted tex coords.

The texture is using GL_FLOAT as the data type and GL_RG32F as the internal format, I don't use mip mapping and the image is usually non power of 2 and non square (e.g. in this case 1920*1079), the prerendered image has the same size as my precalculated texture coordinate texture. This effect occurs with filtering set to GL_LINEAR as well as set to GL_NEAREST.

Edit2

I have managed to reproduce the problem in a more minimal example, excluding any distortion calculations.
That is, in OpenSceneGraph I have two cameras, one (rttCamera) rendering a scene into a texture and one (displayCamera) rendering this texture to the framebuffer. I precalculate a texture which stores texture coordinates, but basically they should return the same texture coordinates at every pixel as the geometry of the screen quad does:

for (int i = 0; i < res[0]; i++)
{
    for (int j = 0; j < res[1]; j++)
    {
        reinterpret_cast<float*>(img->data(i, j))[0] = i / res[0];
        reinterpret_cast<float*>(img->data(i, j))[1] = j / res[1];
    }
}

img->data is the buffer for the texture, i, j are the pixel coordinates, res stores the resolution (1920 * 1080 in this case). The texture itself is again GL_RG32F with data type GL_FLOAT.

The fragment shader code for the displayCamerais the following:

#version 400

uniform sampler2D uRenderedImage;
uniform sampler2D uTexCoordLut;

uniform bool uIsLutUsed;

in vec2 vTexCoord;
out vec4 gl_FragColor;

void main()
{
    vec2 texCoord = texture(uTexCoordLut, vTexCoord).rg;
    // gl_FragColor = vec4(texCoord, 0, 1);
    gl_FragColor = texture(uRenderedImage, vTexCoord);
    if(uIsLutUsed)
    {
        gl_FragColor = texture(uRenderedImage, texCoord);
    }
}

where the boolean uniform is for me to switch between the geometry texcoord version and my precalculated texcoord version.

The resulting images are the following (again cropped), first the version without the precalculated texture coordinates

and then the version with precalculated texture coordinates

Edit3

I tried around a bit, and I found one solution (although that doesn't answer my question of 'why').
Currently my prerendered image and my 2D texture that stores my new texture coordinates both have the same resolution. However, in my minimal example from Edit2, if keep the prerendered image at 1920*1080 pixel and increase the texture with my precalculated coordinates to 10000*10000, the difference between precalculated texture coordinates and geoemtry texture coordinates (or the rendered image with said coordinates) is minimal, barely noticable at all.

I finally found the problem.

My precalculation was done with pixel coordinates from 0 to res[0] - 1 (i.e. the resolution width):

for (int i = 0; i < res[0]; i++)
{
    for (int j = 0; j < res[1]; j++)
    {
        reinterpret_cast<float*>(img->data(i, j))[0] = i / res[0];
        reinterpret_cast<float*>(img->data(i, j))[1] = j / res[1];
    }
}

However, pixels in OpenGL are accessed via their center, as can be read in another SO answer or in Real-Time Rendering (3rd and 4th Edition, p.21). Meaning, that if I have a pixel with 0 as its left border and 0.1 as its right border, then 0.05 is its center.
That means in my case, I have to offset the precalculated texture coordinate by half a pixel to read out of my prerendered image:

for (int i = 0; i < res[0]; i++)
{
    for (int j = 0; j < res[1]; j++)
    {
        reinterpret_cast<float*>(img->data(i, j))[0] = i / res[0] + 0.5 * pixelWidth;
        reinterpret_cast<float*>(img->data(i, j))[1] = j / res[1] + 0.5 * pixelHeight;
    }
}


What is the algorithm used in pbrt-v3 for triangle intersection? Where's this algorithm come from? Any references?

And a specific question, when the authors compute the transformation from world space to ray-triangle intersection coordinate space, what's the permutation step for?

The permutation is described in section 3.6.2, page 158-159.

A reason behind this permutation is that a shear matrix (which involves dividing z component of ray direction) is computed to transform from world space to ray triangle intersection coordinate space.
Permuting the coordinates is to avoid divide-by-0 error by mapping nonzero component to +z.


How to use PCA to reconstruct 3D Mesh？ Given a set of human body mesh {S_i}, for each S_i, the points are N*3, we can do PCA on that. We can use PCA to reduce the dimension of {S_i}, for example to k dimension. Let the main direction of PCA be [D1, D2, ..., Dk], then the body geometry can be represented by S = W1 * D1 + W2 * D2 + ... + Wk * Dk, where W = [W1, W2, ..., W3] is the weights.

But how can we get the weight W of one specific mesh S_i in {S_i}?

Given a matrix $D \in \mathbb{R}^{3N\times k}$ where each column is normalized $D_i$, You can get $W = D^TS_i$, and later it can be used to reconstruct the shape with $S_i = DW$ (With some error if you truncated some basis). The acquisition of weights is equivalent to calculating dot products between each PCA basis and $S_i$. What is done here is you project $S_i$ onto each orthogonal basis $D_i$, and calculate the length of the projected vector, so you are basically calculating the required contribution of each basis to construct the given $S_i$.

If you are using unnormalized PCA basis, you can decompose the unnormalized PCA basis matrix into a diagonal matrix an an orthonormal matrix (Say, $\Lambda D$), then use a similar approach to obtain the contribution of each basis. In this case, you'll have to use $\Lambda^{-1}D^T$ instead of $D^T$.


Understanding distinct vs. not distinct knots in B-splines In reading the wikipedia page about B-Splines https://en.wikipedia.org/wiki/B-spline, I am getting confused about the concept of knots.

If we have a spline of order $n$, with a knot vector ($t_0, \cdots, t_k$),
then if the $k$ knots are distinct, I think we have $k$ polynomials that are pieced together, but if we have that $r \leq k$ knots are distinct, then I am not sure what this means.

How many polynomials are we piecing together?
 How is the overall piecewise curve defined?

So, the number of polynomials that you piece together depends on multiple factors, including the number of control points and the degree of the curve. If the number of points is equal to the order, then you have a single polynomial, if the order is $n$ and you have $n+1$ points, then you are piecing together 2 polynomials, etc...

Distinct points are points that are different from each other.
Take the vector [0,1,2,3,4] where all knots are distinct and contrast it with [0,1,2,2,3,4] where the knot 2 has multiplicity 2 (it's not distinct since it appears twice). These knots lower the degree of the curve locally by 1 for each repetition.

The overall curve can be defined in many ways, the simplest way to look at it is as DeDoor's algorithm, which is a generalisation of DeCastlejeau's.

I am going to do a bit of self promotion and link my own notes of B-Splines. Gitlab's PDF viewer butchers them a bit so I recommend downloading them and reading them with Okular or Adobe reader.

Regardless, that document has more information than what can be put into a stack exchange answer and does indeed try to convey the intuition about how to build B-Splines, both mathematically and algorithmically.


Vulkan: Uniform Buffers versus Push Constants for static data I'm sort of struggling to understand the conceptual difference between uniform buffers and push constants. From what I can gather by reading the spec, the main differences are:

Uniform buffers can be much larger than push constants.
UBOs use std140, PCs use std430.
UBOs can be updated at any time with vkCmdUpdateBuffer (or host mapping) and persist their values otherwise, PCs have to be re-pushed for each render pass. (Which surprised me - based on the name. I thought I would literally be updating constants in the pipeline in-place, and have those changes persist)

In my scenario, I have about ~200 bytes worth of data that I expect to be mostly constant. That is, I will change them very infrequently. Would it be better to (assuming the size permits) use push constants even though I have to re-send them in every command buffer? Or would it be better to use a 200-byte UBO and only update it infrequently with vkCmdUpdatebuffer?

Also. what if I have e.g. a float random_seed that I will update every time the shader is run? Assuming I already have a UBO, would it be better to lump that in with the UBO, even though the rest of the UBO is constant, or would I gain a benefit from using push constants for specifically this variable, so I can avoid having to vkCmdUpdateBuffer before every render pass?

UBOs can be updated at any time with vkCmdUpdateBuffer

From the specification: "vkCmdUpdateBuffer is only allowed outside of a render pass." So "at any time" is not the case.

Even if it were allowed inside of a render pass, it's still a transfer operation. Which means you need to synchronize the memory transfer with the commands that use it. Which slows down performance.

For the general Push Constant vs. Uniform thing, use your judgment. By "judgment", I mean just look at how they work. Push Constants allow you to change their data at any time without doing heavy-weight processes like memory operations, synchronization, or altering the descriptor state. Clearly, they are for frequently changing data. How frequently is "frequently"? Well, that's a judgment call.

Failing that, profile the performance difference.


What does "makeContextCurrent" do exactly? I am pretty confused about that function. GLFW Context reference says as follow:

  This function makes the OpenGL or OpenGL ES context of the specified
  window current on the calling thread. A context can only be made
  current on a single thread at a time and each thread can have only a
  single current context at a time.

But it does not help me much.
Can it work like glBind- methods? (glBindBuffer, glBindVertexArray and so on)

OpenGL is a state machine. An OpenGL context holds that state. The state contains information such as which textures are bound to which texture units, which attachments the current FBO has, and things like that.

When you set the current context, you are switching all the state from the old context to the new context. Here's an example:

glfwMakeContextCurrent(window1);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, texture1);

glfwMakeContextCurrent(window2);

What this does is makes the context of window1 current. It then enables the 2D texture unit and binds texture1 to it.

Next it makes the context of window2 current. If you were to check the enabled state of GL_TEXTURE_2D it would be false (unless you had previously enabled it on the window2 context). Likewise, if you check which texture is bound to the 2D texture unit, you would likely find no texture was bound, or some texture other than texture1 was bound.

Basically all drawing happens in the current context. If you make a different context current, then all drawing will now happen in that context.

It is common to have a single context per thread in a multi-threaded OpenGL application. That way each thread is drawing into its own context and doesn't mess up the state of contexts on other threads.


What does it mean by a Constraint-based Graphics System? Scratchpad was a constraint-based graphics system.

What does it mean  by a Constraint-based Graphics System?

If you have a geometric drawing with measurements you can solve what the shape has to be. This is valuable for example in engineering as it allows you specify the requirements and the computer can then solve the constrained drawing via a solver.

Image 1: Constraint solver in action, red constraints are fixed while blue ones are free to change.

There are several ways how the solver might work, it could be a numeric nonlinear based on gradient descent. Or it can be fully algebraic. Usually its a gradient descent solver as this is easier to implement, and possibly faster.


When creating a JPEG, how can I minimize the occurence of artifacts? There are 2 main factors that seem to lead to digital artifacts when creating JPEG images: Aliasing and Compression.

Example:

  Converting a PNG with characters on it to a JPEG or placing vector images over a photograph, will result in pixelization on their edges. The antialiasing generally creates a kind of blur around them, but if the image is lossy compressed, part of the details are also lost, therefore the blur and the pixelization may become less noticeable.

Is this the right thing to do? I.e. since a lossless compression generates a highly detailed image, the artifacts resulting from aliasing will be more noticeable, so a balance may be found by using the right compression, although compromising the image quality.

Edit

I just saved this JPEG in mspaint (3.46KB):

Here's the same JPEG with maximum compression (lowest quality, 0.5KB):

Here's yet the same JPEG with 50% compression (notice the difference in size, 1.29KB):

The same 50% compression but saved as "progressive JPG", kept the original EXIF and XMP data, and "tried to save with original JPG quality" (you can notice that there are no grey pixels around, 2.96KB):

And finally the same as before with chroma subsampling disabled (same file size, 2.96KB):

JPEG compression involves three main steps:

Chroma subsampling. The image is converted from RGB into YCbCr color space, in which the luma or brightness (Y) is stored separately from the chroma or color components, Cb and Cr.  The Y component is kept at full resolution, but Cb and Cr are downsampled, typically to half resolution on each axis.  This exploits the fact that the human visual system is more sensitive to fine details of brightness than of color.
Frequency quantization. The Y, Cb, and Cr images are converted into a frequency representation, by breaking them up into 8x8 blocks and applying the discrete cosine transform (a variant of the Fourier transform) to each block.  The result is a matrix of numbers that describe the amplitudes of different spatial frequencies in the block.  These numbers can then be quantized (rounded off to a chosen number of bits of precision). Different levels of quantization are used for different frequencies, exploiting our visual system's relatively lower sensitivity to high frequencies.  This is where the JPEG encoder's quality setting comes into play: lower qualities use coarser quantization.
Entropy coding. The  quantized DCT values are passed through an entropy coder, which losslessly compresses the bit stream by using fewer bits to represent the more common values, sort of like a zip file.

Steps 1 and 2 are the lossy ones, and each produces its own type of artifacts (though there is some overlap).  Chroma subsampling tends to blur sharp edges between regions of different colors.  This is particularly visible in vector art, where brightly colored shapes pick up vicious jaggies around their edges. Frequency quantization blurs fine details in general, and also creates block-shaped artifacts at low quality settings, because the DCT is done on a block-by-block basis.  It is particularly visible on text.

This is why JPEG usually isn't used for images containing vector graphics or text—its compression algorithms are poorly suited for those cases, although they work well for photographs and other images with complex texture and not-too-sharp edges.


What does it *mean* to scale in an arbitrary direction? My graphics text explains rotating along the cardinal axes, then generalizes the math to show how to make a rotation matrix for rotating around an arbitrary vector. That all makes sense and I can both follow the math and picture the operations.

Then the text discusses scaling uniformly and in cardinal directions. Again I can follow everything. But then it goes through the math for "scaling in an arbitrary direction", and although I can follow the vector operations I don't understand what scaling in a non cardinal direction even means. I can start with an object rotated "right side up", and then imagine it getting wider/taller/deeper, but I don't understand what it would mean to scale it in an arbitrary direction.

My first thought was they must just mean scaling each cardinal direction by a different amount, but that can't be right because they don't end up with a diagonal matrix (take the identity matrix, change the first row 1 to 5 and the second row 1 to 3, and now you have a matrix that scales x by 5 and y by 3). Their 2D general scaling matrix is below.

When you scale along the X-axis, the X-coordinate (parallel to the axis) gets stretched, while the Y-coordinate (perpendicular to the axis) remains the same. You can think of scaling along an arbitrary axis as stretching along some diagonal.

Here's a pic of a square being scaled along the main diagonal (the axis pointing to <1, 1> ) by factors of 2 and 0.5.

You can see that the distance along the axis (green line) gets scaled while the perpendicular distance (red line) stays the same.

Mathematically, this is the same as rotating until your chosen axis lies along the X-axis, scaling by X, then rotating back again.


Path Tracer - Model loading Libraries? so I am in the process of making my own GPU path tracer based on OpenCL as part of my bachelor thesis. I have already made a CPU and a GPU ray tracer based on OpenGL compute shaders but all I have ray-traced are simple spheres and planes.

1) Onto the main question, what do I need to path trace actual models like the teapot/dragon etc. Are there any standard, well known libraries for this purpose? Or do I have to write my own model loader?

Writing my own loader seems all nice but I won't just have to read the data but maintain a proper data structure for all the triangles (K-d tree or octree perhaps) as well and then pass that structured data onto the GPU. If there is a library that can give me the data structure it'd be nice as I'll have to write the code for traversing it on the GPU only as I have already written one to traverse octrees for my GPU raytracer.

AssImp library may be a good choice for you, assuming you are using C++ ?. It supports a decent number of model formats.


Correctness or otherwise of the procedure for generating an orbiting camera in WebGl I have to create a orbiting camera. I created the model of sun (a sphere) around (0,0,0) , triangulated and Phong shaded it. I would have a camera that goes around the sun thus an orbiting camera and I would know if my way to proceed is right or not:
In WebGl there isn't a phisical camera. Nevertheless I imagine to place my camera at the center (0,0,0) and I have first to translate it of [0 0 100] ([0 0 100] is the point where early is located the camera) and later I rotate the camera around y ([0 1 0]) of some angle (Actually the angle is animated, i.e. the angle increases over time).

Thus cameraMatrix = R(angle,[0 1 0])T([0 0 100])
(First translation and later rotation)

But in WebGl there isn't camera, then I have to move the world with inverse transformation namely:

ModelViewMatrix = inverse(cameraMatrix);

Thus
ModelViewMatrix =  inverse{R(angle,[0 1 0])T([0 0 100])} = inverse(T([0 0 100]))inverse(R(angle,[0 1 0])= T([0 0 -100]) R(-angle,[0 1 0])

(In the last first rotation and later translation)

So far is it all right?
Finally in drawScene namely the function of rendering cycle the order of call is inverted namely:

mat4.translate(mvMatrix , [0,0,-100.0]);
mat4.rotate(mvMatrix , degToRad(-angle) , [0,1,0]);

in this order does a rototranslation, namely it occurs first the rotation and later the translation (inverted respect the order of call functions).

I would confirm if this is right because I'm a neophyte of Computer Graphics and I'm not sure.
(I know that I can use lookAt() or define the values of cameraMatrix by hand but I'm not interested in it).
Thanks in advance

Your math looks correct to me. Your terminology is a little off - technically what you are creating here is just the 'view' matrix rather than the 'modelView'. If you're just drawing a single sphere at the origin then it doesn't make a difference, but normally the modelView is unique for each object in a scene - it's the object's model-to-world transform matrix multiplied by the view matrix you describe above.


What is the meaning of words 'stochastic', 'temporal' in computer graphics? Last time I see a lot of papers which operates on this words, i.e. stochastic AA, temporal coherence etc. What does that means? Can you provide examples with explanations in plain English?

A stochastic process is one where you gather your samples randomly. So stochastic AA is antialiasing where you gather multiple samples for the same pixel with small random changes.
Temporal refers to a process over time. Temporal coherence is for instance mentioned when you have no flickering artifacts, or aliasing that you see during movement. It means that frames are coherent in a certain time window.


Transforming a ray from camera space to world space I am writing a Raytracer and generating rays from the camera. I have a working program but slightly confused about the linear algebra concepts involved in transforming the ray from camera space to world space. I already have the camera position in world coordinates. So, only need to transform the direction vector. I think there are two ways to achieve this,
a) Directly transforming the direction vector:
The direction vector can be built as,
direction vector = point on film - camera position
The camera is at the origin, so the above would reduce to the coordinates of the point on the film. Now, we can write this direction vector in homogeneous form and apply the camera to world transformation. This should give us the direction vector for the ray in world space.
b) Transform the origin and point on film:
The origin's position vector is at zero and the position vector of a point on the film is the point's coordinates. We can apply the camera to world transformation to these and this would give us two position vectors in world space: the camera's position vector and the film point's position vector. Then, the direction vector in world space can be directly built as,
direction vector = point on film (world space) - camera position (world space)
Mathematically, the first approach is to multiply the direction vector in camera space by the camera to the world (4x4) matrix, and this would be my answer. However, based on the second approach, I should do the same thing (multiply by 4x4 matrix) and then subtract by the camera's position in world space.
I tried programming both strategies and the second approach seems to give the right answer. What's wrong with my first approach?

Now, we can write this direction vector in homogeneous form and apply the camera to world transformation. This should give us the direction vector for the ray in world space.

That is the problem. When transforming a direction, you must ensure the w coordinate is 0, so that it doesn't get translated by your camera-to-world matrix. What you're doing in the second method is that you're transforming the direction vector as a point, and then subtracting the camera's position, which effectively undoes the translation.


How to display edges in a 3D file on Facebook I'm trying to show a 3D model on Facebook and display its edges and lines. I found that I could export a 3D model from Sketchup using glTF Export plugin to a glb file that I can drag and drop in a Facebook post.

I tried many ways to set material, width, styles on lines in my model but once imported in Facebook the lines don't appear. It's not a Facebook problem since the gtTF viewer has the same display. I also have the same rendering problem by ubloading an stl file to Sketchfab.

Is there a missing implementation in glTF format?
Is there another freeware to export a 3D model to Facebook (showing edges)?

3D graphics are not images they are instructions for images. Now the formats that export data beween systems work on lowest common denominator. Geometry is fine, but any effects on top of geometry would require you to agree on a set of instructions to implement. (no such reffort has been made for npr rendering lines).

The lines are not really part of the model that has been exported trough the gltf pipeline. Simply because the line rendering is a special feature of the sketchup system. There is no generic format that support those, since they need to be generated frame by frame. And there is
quite possibly a infinite number of such features that could be implemented, none of these are part of the generally accepted set of things to export.

Most likely your out of luck the engine in question simply has no way to do this. So there is no solution for your problem. STL is a even more rudimentary format it only has triangles nothing else.

PS: This is why it is called an export not save. Although many applications have gone so far as to call even partial implementations save. But you can not just expect all things to go from application to application, especially in 3D


Point of view and rotation Two questions which cross over maths/3D imaging/computer science. I'm trying to describe a point of view of an item in a 3d space and subsequently its rotation in a paper I'm writing. Versions I've had read have confused readers.

The scene is as illustrated in the attached image - we are looking on as it were from the left-hand side of the scene.

How should I describe the view of the object from the eye symbol? I've termed it a "thirty degree elevated front view" - it's the 'front' that I think is most problematic.
Then if the object is rotated backwards along the x axis how do I describe that? I've said "transformation, specifically a rotation seventy degrees backwards along the x axis".

Are these the correct way of describing this point of view and rotation?

You could phrase it in terms of elevation away from various planes. So 30° above the x-z plane, and 0° from the y-z plane (or whatever it actually is). It's sort of like spherical coordinates where you'd have theta and phi.


What exactly is it about reality that is so difficult to recreate in a 3D simulation? I have to admit that, while watching footage of the enhanced 2020 version of the 2018 game Forza Horizon 4, when they are driving fast, I have to sometimes remind myself that I am watching a video game and not a video recording of reality. It's the closest I've ever seen to photo realism, but as soon as the car stops, or they go too slowly, or they do something slightly not expected, the illusion quickly falls apart.
It's just a bunch of smoke and mirrors and dead polygon objects after all. Not an actual photo-realistic 3D version of England where I can "drive anywhere and do anything". But for small moments, it can fool me. Example gameplay footage: https://youtu.be/PqwQDRersIM?t=42
What exactly is the "missing piece" at this point? It's impossible for me to not be impressed by the game, but I still don't want to play it. I can tell that I would very quickly get bored of the illusion, and just grow ever more frustrated by the limitations of the reality as presented in the game.
For me, it's largely knowing that it's all dead 3D objects. There are no detailed rooms inside all of those buildings which I could enter and go through. There are no people either, or, if there are, they don't look or behave anything like real humans would. There are many visual details which just look "off", and the more photo-realistic it looks overall, the more demanding my brain becomes and the more eager I am to point out any flaw.
If you then take a look at this random actual video, even that game looks like garbage in comparison: https://youtu.be/FMOb5-e3J7Y?t=49 (yes, I know that the settings/locations are quite different, but that doesn't really matter).
Something about reality is so much "richer" and "real" than what they can do with a 3D world, but I don't quite understand what exactly. I feel as if I should be more impressed than I actually am by that car game. It looks objectively amazing, yet at the same time, I have very little interest in actually playing it. And keep in mind that I'm a person who has been hunting for photo-realistic 3D simulations my entire life and who is endlessly fascinated by the concept.
One thing I can say for sure: it has almost nothing to do with the resolution of the game/camera. Even if you take a very old and cheap camera and go filming reality, it will look a million times better than the 8K game.
Can you explain to me why we don't have a simulation that looks and behaves just like the actual video I just linked to?

As @AnnoyinC already mentioned in his comment, we are simply lacking computational power and storage requirements (RAM).
For the visual effects, you need to calculate every ray of light that somehow ends up in the observer's eye after an arbitrary number of reflections on different surfaces. The reflections depend on the surface's microstructure. This alone is so extremely demanding that is far from possible to calculate it in real-time with current consumer hardware. The first steps into this direction are already taken with ray tracing support of modern graphic cards but it is still very limited.
And that's just the "easy part".
It gets way more complicated if you want to have objects and fluids that behave physically correct. While each ray of light is more or less independent of other rays, physical objects influence each other. Also, simulating the correct reaction of an arbitrary object to an external force can already take days and weeks for a single timestep/frame since you need to solve extremely large equation systems that are often non-linear. If you want to know why I would suggest searching for a "Finite-Element-Method" or "Computational Fluid Dynamics" tutorial. These techniques are commonly used by engineers and separate a structure into a mesh of smaller interacting substructures. The results get closer to the behavior of the real structure/fluid the denser the mesh. However, doubling the mesh density in each of the 3 spatial directions comes at the price of needing 8 times more computational power and RAM. Now think of a tree waving in the wind with thousands of leaves where each leaf alone is a complex physical object... I think you get the picture.
To get around all these problems, a lot of approximations are used. While most of them are quite good illusions they will give wrong results at some point. However, that doesn't necessarily mean that we realize it.


Why does my Perlin Noise look "blocky"? I tried to implement Perlin Noise by myself using just the theory (following flafla2.github.io/2014/08/09/perlinnoise.html). Unfortunately I was unable to achieve the look of the "original" Perlin Noise.

What's the reason the code below renders a blocky version of Perlin Noise?

What should I improve/change in the code so that it renders Perlin Noise without the artifacts?

I suspect there might be problem either in the way I interpolate or in the grads vector. The grads vector contains dot products of (random vector for lattice point) and (the size vector) – for all 4 nearby lattice points. (The random and size vectors are described in the very first link.)

GLSL Sandbox: http://glslsandbox.com/e#32663.0

float fade(float t) { return t * t * t * (t * (t * 6. - 15.) + 10.); }
vec2 smooth(vec2 x) { return vec2(fade(x.x), fade(x.y)); }

vec2 hash(vec2 co) {
    return fract (vec2(.5654654, -.65465) * dot (vec2(.654, 57.4), co));
}

float perlinNoise(vec2 uv) {
    vec2 PT  = floor(uv);
    vec2 pt  = fract(uv);
    vec2 mmpt= smooth(pt);

    vec4 grads = vec4(
        dot(hash(PT + vec2(.0, 1.)), pt-vec2(.0, 1.)),   dot(hash(PT + vec2(1., 1.)), pt-vec2(1., 1.)),
        dot(hash(PT + vec2(.0, .0)), pt-vec2(.0, .0)),   dot(hash(PT + vec2(1., .0)), pt-vec2(1., 0.))
    );

    return 5.*mix (mix (grads.z, grads.w, mmpt.x), mix (grads.x, grads.y, mmpt.x), mmpt.y);
}

float fbm(vec2 uv) {
    float finalNoise = 0.;
    finalNoise += .50000*perlinNoise(2.*uv);
    finalNoise += .25000*perlinNoise(4.*uv);
    finalNoise += .12500*perlinNoise(8.*uv);
    finalNoise += .06250*perlinNoise(16.*uv);
    finalNoise += .03125*perlinNoise(32.*uv);

    return finalNoise;
}

void main() {
    vec2 position = gl_FragCoord.xy / resolution.y;
    gl_FragColor = vec4( vec3( fbm(3.*position) ), 1.0 );
}

The interpolation looks fine. The main problem here is that the hash function you're using isn't very good. If I look at just one octave, and visualize the hash result by outputting hash(PT).x, I get something like this:

This is supposed to be completely random per grid square, but you can see that it has a lot of diagonal line patterns in it (it almost looks like a checkerboard), so it's not a very random hash, and those patterns will show up in the noise produced by it.

The other problem is that your hash only returns gradient vectors in [0, 1], while they should be in [−1, 1] to get gradients in all directions. That part's easy to fix by remapping.

To fix those problems, I switched the code to use this hash function (which I learned from Mikkel Gjoel, and is probably due to a paper by W.J.J. Rey):

vec2 hash(vec2 co) {
    float m = dot(co, vec2(12.9898, 78.233));
    return fract(vec2(sin(m),cos(m))* 43758.5453) * 2. - 1.;
}

Note that due to the trig functions it's going to be a bit more expensive than your version. However, it considerably improves the appearance of the resulting noise: