Inflate 3D mesh into sphere I am after a simple (not necessarily efficient) algorithm to "continuously" transform a closed surface mesh into a sphere.

It seems that Blender is able to do this quite well: https://docs.blender.org/manual/en/latest/modeling/meshes/editing/transform/to_sphere.html

However I cannot find a reference, or technical description of the algorithm they use. I tried looking at the code but did not managed to figure out what it does unfortunately.

The core calculation seems to be

  sub_v3_v3v3(vec, td->iloc, tc->center_local);

  radius = normalize_v3(vec);

  tratio = ratio * td->factor;

  mul_v3_fl(vec, radius * (1.0f - tratio) + t->val * tratio);

  add_v3_v3v3(td->loc, tc->center_local, vec);

Which in C-ish psuedo code looks like

vec = pos - center; //find vector from center to point
radius = length(vec); // get the length of the vector
vec = vec / radius;  //normalize the vector

tratio = ratio * individual_sensitivity; // adjust sensitivity per vertex

vec = lerp(vec * radius, vec * t->val, tratio); //linearly interpolate between the original and where the point would be on the sphere

pos = center + vec; // store result

So the code projects the point to the sphere and linearly interpolates the point between the original location and the sphere location.


What kind of denoising must follow locally adaptive binarization? I am experimenting with local (adaptive) grey threshold for the binarization of B&W photographs of yellowed manuscripts:

I got adaptive thresholding working in both scikit-image and cv2. In both cases (grey < threshold_niblack() in scikit-image and cv.adaptiveThreshold() in cv2 -- all Python) the result I get is successfully binarized, with the background pretty much uniform -- and completely full of noise:

What kind of step do I now need to separate the text from this background noise?

Besides the suggestions in comments, I would suggest a high pass filtering to get rid of the low frequency noise, before running your local thresholding. Here an example with GIMP, that you can probably easily reproduce with opencv as well.


Function to convert HSV angle to RYB angle I'm trying to create a RYB color wheel with a smooth-looking gradient across the RYB spectrum. The angles of the HSV wheel are incorrect (yellow at 60deg instead of 120deg etc).

If I just do a few linear transformations the results is still not appropriate; perceptually red occupies too many angles and yellow not enough. Even if I do a bit of a hack and try a non-linear polynomial gradient it's difficult to get the right "spread":

Is there an "official" function that maps HSV hue angle to RYB hue angle?

Update: Reading this gives some ideas, but the part about excluding cyan confused me more because of Adobe's RYB color wheel: Is this true RYB or is it CMY with hue angles shifted in the blue range?

I just discovered that Adobe color source includes HSV-RYB hue mapping functions (replicated in Ben Knight's Kuler-d3). Apparently Adobe uses uses piecewise linear gradients rather than the polynomial that I was trying to use (and technically it is a CMY wheel not a RBY one, I believe). Here are the relevant stops:

RYBstop HSVstop
60      35
122     60
165     120
218     180
275     240
330     300

Here is the graph of these showing the perceptual mapping:

So there'll be artifacts at the junctions where the mapping function is not smooth (which can be seen in the original Adobe image). And here is an imagemagick script for creating the color wheel.

#!/bin/bash
#assemble gradient pieces (pre-caculated stops), then join them all
convert -size 600x300 gradient:#000000-#181818 -rotate -90 grad1.png
convert -size 600x310 gradient:#181818-#2A2A2A -rotate -90 grad2.png
convert -size 600x215 gradient:#2A2A2A-#555555 -rotate -90 grad3.png
convert -size 600x265 gradient:#555555-#7F7F7F -rotate -90 grad4.png
convert -size 600x285 gradient:#7F7F7F-#AAAAAA -rotate -90 grad5.png
convert -size 600x275 gradient:#AAAAAA-#D4D4D4 -rotate -90 grad6.png
convert -size 600x150 gradient:#D4D4D4-#FFFFFF -rotate -90 grad7.png
convert grad?.png +append grad.png
#create hue
convert grad.png -alpha set -virtual-pixel Transparent -rotate 180 -distort Arc '360 -90 300' +repage -gravity center -crop 600x600+0+0 +repage h.png
#create saturation
convert -size 600x600 -alpha on radial-gradient:white-none s.png
#create value
convert -size 600x600 -alpha on radial-gradient:none-white v.png
#combine h,s,v in hsv colorspace
convert h.png s.png v.png -combine -set colorspace HSV -colorspace sRGB ryb.png


Full screen quad texture coordinates mapping I'm rendering a full screen quad using gl_VertexID as GL_TRIANGLE_STRIP

const vec2 kVertices[4] = vec2[]
(
 vec2(-1.0,  1.0),
 vec2(-1.0, -1.0),
 vec2( 1.0,  1.0),
 vec2( 1.0, -1.0)
 );

void main() {
    gl_Position = vec4(kVertices[gl_VertexID], 1.0, 1.0);
    vs_out.texCoords = gl_Position.xy / 2.0 + 0.5;
    vs_out.instanceID = gl_InstanceID;
}

Geometry shader just passes coordinates through into the output called vTexCoords.

Then, in fragment shader, I output red and green into 2x2 texture using following code:

if (vTexCoords.x >= 0.95) {
    oFragData0 = vec4(0.0, 1.0, 0.0, 1.0);
} else {
    oFragData0 = vec4(1.0, 0.0, 0.0, 1.0);
}

I always assumed texture coordinates (0; 0), (0; 1), (1; 0), (1; 1) would map to the vertices, but the output is completely red, which means X value never reaches 1.0.
Then I'm trying to use 0.75 (vTexCoords.x >= 0.75), which is a texel center, and half of the texture becomes green, as expected.

What am I misunderstanding here?

I may have misunderstood, but it sounds like the frame buffer you're rendering to is only 2×2 pixels? In that case, the fragment shader will be evaluated at each pixel center, which will be at 0.25 and 0.75 in UV space. So with a threshold of 0.95 you will indeed not get any pixels green.

If you use a larger framebuffer, say 100×100, then the last 5 pixels of each row should come out green.


How to calculate spatial frequency of an image just in one direction? FFT spectra is not what is needed; what I need is a histogram showing which frequencies are there in image in horizontal direction, and which one is dominant
Such spatial frequency is usually known for a sinusoidal grating; I need the same property for grayscale raster image (in cycles per pixel)

You can take a 1D Fourier transform of a row of pixels from the image; it will give you the horizontal frequencies present in that row.
You could sample one row out of the image, or else average all the rows together to get an overall picture of the horizontal frequencies.


Perlin Noise, surface deformation NOT along the normal vector I've a question about surface deformation using Perlin noise or Simplex noise.
You can see everywhere on google images how these results look like, if you calculate a 2D / 3D noise and move the surface in normal direction like the image in the following discussion on stackoverflow:
https://stackoverflow.com/questions/53688083/using-analytical-perlin-noise-derivatives-on-a-3d-surface
Using one noise calculation for a vertex seems to be fast, but caves can not appear. It is like generating a height map for a surface.
But, has someone tried to use 3D Perlin/Simplex noise for each dimension? So that 3 times a 3D Noise is calculated and the vertex is moved in the dimension direction?
I think that of cause caves can be appear and it maybe looks realistic as well.
I've time pressure to implement some surface distortion and I am only interested in some resulting pictures, to decide using this technique or look for another one.
So if someone has already implemented it, please post some resulting images. Thanks a lot!

You can do this, and the results can be interesting, but they’re pretty far from looking like realistic terrain. Here’s a plane deformed with 3D simplex noise (Perlin doesn’t look significantly better):

The issue is that there’s no volume to the surface, per se, and nothing preventing it from passing through itself. Deforming it only along the normal vector mostly avoids that. For a surface that looks like terrain, with actual caves, you’ll probably need some kind of voxel system like the one described in this GPU Gems article.


Rounding corners of polygon given vertices of its corners Given a polygon (regular or irregular, convex or concave), I want to round its corner, with a given radius; let's say 'x'. I have the code to draw an arc given 2 points, but how can I find the start and end points of the arc?

All I have is a list of points representing each corner of the polygon.

Here's a simple example: I have a list of vertices of corners of the figure (1).
The figure (2) is what I need so I could draw the arcs to round the corners.

PS: I'm developing a CAD drawing generator using .netdxf API (https://github.com/haplokuon/netDxf)

Ok, Xenapior and Reynolds together have the right idea. But the explanation is a bit lacking so here is a image to explain it all and some further musings. First let us  start by drawing an image (yes i know that is what they say in school for you to do but nobody does it).

From the image we can see that there are 2 equal right triangles $V_2, A, C$ and $V_1, B, C$. In this triangle we have one unknown that we can define namely rounding radius $r$, also we know right angle is 90°. The angle between the line $V_1-V_2 = \vec a$ and line $V_2-V_3 = \vec b$ is easy to compute with  the formulation fo angle between vectors

$$\cos(\beta) = \frac{\vec a·\vec b}{ |\vec a|·|\vec b|}$$

That in turn can be simplified if the vectors already are normal. Thus three things of the triangle is known, which means all is known. So if you know the rounding radius to use the calculating the points $A$, $B$ and $C$. So finally:

a = normalize(V2-V1);
b = normalize(V2-V3);
halfang = acos(dot(a, b))/2.;
// skip center if you iuse splines
C = V2 - r / sin(halfang) * normalize((a+b)/2);
A = V2 - r/tan(halfang)*a;
B = V2 - r/tan(halfang)*b;

You can simplify this a bit with trigonometric identities. Or if you use rational B-splines you can skip the calculation of C

Note: that this is only one possible formulation


Shadow map projection issue I am using Unity but this question might not be proper to this engine.

I have projected the shadow map onto this sphere but there are grazing shadows. Is it possible to avoid it or to correct it as I am using the raw depth data of the texture.

Here is the pseudo-shader code.

uniform sampler2D _ShadowMap;
float4 shadowCoord : TEXCOORD1;

// vertex shader
o.shadowCoord = mul(unity_World2Shadow[0], mul(_Object2World, v.vertex));

// fragment shader
color.rgb = tex2D(_ShadowMap, i.shadowCoord);
return color;

One of the downsides to shadowmapping is that you need to offset, or bias, the point used for shadowmap-lookups. This is done in a variety of ways - typically by a constant (shadowmap-depth) distance or based on the slope of the casting triangle. Both of these tend to be done during the shadowmap-rendering pass.

I think what you need to solve your sphere-problem, is to offset the point you are shading a bit along its surface normal, before doing the shadowmap-lookup. This happens during shading, when sampling the shadowmap, not during shadowmap-rendering. Incidentally it is also part of the Unity standard rendering pipeline, so you can steal it from the shader include-files.

ref http://www.dissidentlogic.com/old/images/NormalOffsetShadows/GDC_Poster_NormalOffset.png
ref http://www.digitalrune.com/Blog/Post/1765/Shadow-Acne


Weird artifacts in my ray tracer I am getting weird artifacts in my simple ray tracer that I made using the book Ray Tracing in One Weekend by Peter Shirley.

Example:

(circular pattern on the ground and black tint on the spheres.)

I suspect these artifacts are due to hit point of the primary rays being under or above the surface, which causes secondary rays to intersect with the object itself.

I tried to fix this by moving the hit point of primary rays along the surface normal at that point. Here's the result :

The artifacts are gone but that broke the dielectric material (sphere on the left). It is acting as a reflective sphere rather than a transparent one.

Code for the dielectric material :

virtual bool scatter(const Ray &inRay, const HitInfo &hitInfo, Vector3 &attenuation, Ray &scatteredRay) const {
    attenuation = Vector3(1.0f);
    Vector3 normal;
    Vector3 refracted;
    float refractiveIndexRatio; //n1/n2 = sin(a2)/sin(a)
    float cosine = 0.0f;

    if (hitInfo.hitNormal * inRay.direction() > 0) {
        normal = -hitInfo.hitNormal;
        refractiveIndexRatio = refractiveIndex;
        cosine = hitInfo.hitNormal * inRay.direction();
    }
    else {
        normal = hitInfo.hitNormal;
        refractiveIndexRatio = 1.0f/refractiveIndex;
        cosine = -hitInfo.hitNormal * inRay.direction();
    }

    float reflectProb = 0.0f;

    if (Vector3::refract(inRay.direction(), normal, refractiveIndexRatio, refracted)) {
        reflectProb = schlickApproximation(cosine, refractiveIndex);
    }
    else {
        reflectProb = 1.0f;
    }

    if (randomfloatGenerator() < reflectProb) {
        Vector3 reflected(Vector3::reflect(inRay.direction(), normal));
        scatteredRay = Ray(hitInfo.hitPoint + normal * CORRECTION_FACTOR, reflected);
    }
    else {
        scatteredRay = Ray(hitInfo.hitPoint + normal * CORRECTION_FACTOR, refracted);
    }

    return true;
}

I changed hitInfo.hitPoint to hitInfo.hitPoint + normal * CORRECTION_FACTOR, in the if - else case at the bottom of the function, to correct those artifacts. (CORRECTION_FACTOR = 0.0001f)

How can I correct this "self intersection" problem without breaking my materials ?

When spawning the refraction/transmission ray, don't adjust the ray origin by adding the normal: adjust it by subtracting the normal. This moves the ray origin inside the object, so it won't immediately self-intersect. Your current problem is caused by all your refraction rays immediately hitting the surface.

Don't forget that for accurate refractions, the ray needs to be able to intersect with the inside or back face of the object as well. If you're intersecting back-facing surfaces, you need to flip the normal before shading or adding/subtracting it to the origin.


How can a diffused(frosted) glass be modelled in path tracing How can a diffused(frosted) glass be modelled in path tracing?

Currently I am able to define a window poly line as a clear glass material where it transmits or reflects the ray. I am not clear though how it should work if the sheet of glass is frosted(diffuse transmission?).

Assuming that you are familiar with the concept of BSDFs, the usual way of modelling rough dielectric surfaces (i.e. glass, water, plastics) is to use microfacet-based models like Microfacet Models for Refraction through Rough Surfaces.

To make it work efficiently in a path tracer you will need a good sampling strategy, like Importance Sampling Microfacet-Based BSDFs using the Distribution of Visible Normals.

Also keep in mind, that the mentioned BSDF model tends to get darker as you increase roughness because it neglects the light which gets inter-reflected among micro-facets. This can be compensated, but it's not a trivial thing to do.


How much precision (half, float, double, etc) is enough for a Color class? While doing some reading on how real-time applications handle color under OpneGL, I noticed that some examples implemented Color as a collection of 4 floats, while others used 4 doubles. I have even seen some vertex-compression examples in the game development field that advocated saving colors as 4 short.

All this got me eager for learning more about this: what is the limit precision that OpenGl (or hardwares) handle for colors? More importantly, though, what are the limits in precision above which color differences become impossible to notice?

I have the impression that learning more carefully about that would help me better think and decide on how to implement a Color class for different applications and scenarios (e.g. making trade-off choices between memory, speed and color-variety).

Thanks for your ideas on this.

Colors shown on your display or saved to standard image file formats use 8 bits per component. So to store these colors it suffices to use four bytes (unsigned char). These colors are usually using sRGB color space, which includes a non-linear "gamma" transformation that redistributes precision to make it somewhat more perceptually uniform, so it's actually pretty close to the limit of how finely humans can perceive color differences already. (However, the sRGB gamut is only a subset of all the colors that are physically perceivable. Wider gamuts need more bits.)

However, if you're generating or processing colors in graphics software (and not just loading/storing them), there are various reasons you might want to use more precision.

Performing operations on colors, such as adjusting brightness and contrast, alpha blending, gamma correction, etc all tend to lose precision. You might want to store colors as 16-bit or more internally to give yourself more precision headroom. Otherwise, repeated rounding off to 8 bits after each operation can cause a progressive loss of quality.
Similarly, when working with linear colors (not the sRGB gamma-encoded colors) for lighting math, extra precision is needed to ensure you have sufficient precision when you eventually convert back to sRGB.
It may be faster and more convenient to work with floating-point colors instead of integer ones. This is especially true on GPUs, where integer math instructions have only a fraction of the throughput of float instructions. But even on CPUs it's certainly easier and probably faster to convert colors to float, do a bunch of operations, and then convert back to integer, rather than trying to do everything with integer fixed-point math.
If you do HDR rendering at all then you'll need more than 8 bits of precision to handle the larger color gamut and intensity range. The new HDR displays just starting to come out accept images with 10 or 12 bits per component.

Using double for colors is massive overkill, but using float for internal representation of colors is common for all the reasons above. When working with GPUs, half (16-bit float) is also common, as they support that format in hardware.


Properties of the image reconstruction filter in rendering Let $f$ denote the measurement contribution function for paths of length $k\in\mathbb N$, i.e. $$f(x)=g(x_0\leftrightarrow x_1)W_{\text e}(x_1\to x_0)t_k(x_0,\ldots,x_k)L_{\text e}(x_k\to x_{k-1}),$$ where $$t_k(x)=\prod_{i=2}^kg(x_{i-1}\leftrightarrow x_i)f_{\text s}(x_i\to x_{i-1}\to x_{i-2}),$$ and $h_j$ denote the image reconstruction filter of the $j$th pixel so that the measurement $I_j$ of the $j$th pixel color is given by $$I_j=\int h_jf\:{\rm d}\mu.$$

What can we assume know about $h_j$ in general? One thing is that it will only depend on first two vertices of a path.

There are two properties which I really would like to being able to assume: (a) $c:=\sup_j\left\|h_j\right\|<\infty$ (b) $\int|g_j|^2\:{\rm d}\mu\le\alpha\int|f_0|^2\:{\rm d}\mu$ for some $\alpha\ge0$, where $g_j:=1_{\{\:h_j\:>\:0\:\}}\left(h_jf-\frac1{\mu\left(\left\{h_j>0\right\}\right)}I_j\right)$ and $f_0:=1_{\{\:f\:\ne\:0\:\}}\left(f-\frac1{\mu\left(\left\{f\ne0\right\}\right)}\int f\:{\rm d}\mu\right)$.

However, I know almost nothing about the mathematical properties of $h_j$ and so it's hard for me to verify if these properties hold. I guess that generally $h_j\ge0$, but I'm even unsure about that. Maybe even $h_j\in[0,1]$? It would be great if someone could answer if (a) and/or (b) hold or link me to a reference.

I am replying to the last edit of the question.

You can show that $\|h_j * f \|^2_2 \leq \|h_j\|^2_1\|f\|^2_2$ from Young's theorem. Provided that  $\|h_j\|_1 \leq 1$ you have: $\|h_j * f \|^2_2 \leq \|f\|^2_2$. Additionally for $g_j$ I believe you want to clamp the indicator function to $\{h_j > 0 \wedge f > 0\}$. Then $\int_{\{h_j>0 \wedge f>0\}} a\leq \int_{\{f>0\}}a$ (since the integrands are always non-negative: $a\geq 0$). This basically proves that the inequality holds for the left terms of the quadratic expansions, the middle terms are both negative, so those may be ignored. The only thing left to prove is: $(\frac{1}{\mu(\{h_j>0 \wedge f>0\})}\int h_jf)^2 \leq (\frac{1}{\mu(\{f>0\})}\int f)^2$, which is the case if $h_j \leq 1$ everywhere. Basically, as long as your filter $h_j$ preserves the minimum and maximum of the image, you're good (this is not an unreasonable requirement for a filter, note however, that there are filers that do not preserve min-max).

Edit: My bad, the middle terms may not be ignored (at least the one on the right of the inequality). Which means you are left to prove that:

$$\frac{(\int h_jf)^2}{\mu(\{h_j>0 \wedge f>0\})} \geq \frac{(\int f)^2}{\mu(\{f>0\})}$$, which will be a problem.

I guess one has to use tighter bounds to relate the mixed term with the squared terms.

Edit 2:

Let's consider the expression as a whole then:

$$\int \left(h_jf - \frac{\int h_jf}{\mu(\{h_j > 0 \wedge f > 0\})} \right)^2 \leq \alpha \int \left(f - \frac{\int f}{\mu(\{f > 0\})} \right)^2$$
$$ 0 \leq \alpha\int (f)^2 - \int (h_jf)^2 +\alpha\frac{(\int f)^2}{\mu^2(\{f > 0\})} - \frac{(\int h_jf)^2}{\mu^2(\{h_j > 0 \wedge f > 0\})} + \frac{2(\int h_jf)^2}{\mu(\{h_j > 0 \wedge f > 0\})} - \alpha\frac{2(\int f)^2}{\mu(\{f > 0\})}$$
$$ 0 \leq \alpha\left(\int (f)^2 - \frac{(1-2\mu(\{f > 0\}))(\int f)^2}{\mu^2(\{f > 0\})}\right) - \left( \int (h_jf)^2 - \frac{(1-2\mu(\{h_j > 0 \wedge f > 0\}))(\int h_jf)^2}{\mu^2(\{h_j > 0 \wedge f > 0\})}\right) $$

$$0 \leq \left(\int(\sqrt{\alpha}f)^2-\int (h_jf)^2\right) - \left(\frac{\alpha(1-2\mu(\{f > 0\}))(\int f)^2}{\mu^2(\{f > 0\})}  - \frac{(1-2\mu(\{h_j > 0 \wedge f > 0\}))(\int h_jf)^2}{\mu^2(\{h_j > 0 \wedge f > 0\})}\right) $$

$$0 \leq \left(\int(\sqrt{\alpha}f)^2-\int (h_jf)^2\right) - \left(a \left(\int \sqrt{\alpha}f\right)^2 - b\left(\int h_jf\right)^2\right)$$

Note also that $\left(\int u\right)^2 \leq \int u^2$. Then with an appropriate $\alpha$ and $h_j$ bounded it should hold. You may want to study the $a,b$ tems also.


How would I convert a picture of a slanted surface into a straight one? Suppose I took a picture of a whiteboard at an angle. If I could detect where the corners of the whiteboard are, what algorithms and linear projections could I use to convert this slanted view into a "head on" view?

As @user1118321 mentioned, as well as @NathanReed in this answer to another question, you can do a perspective transform to transform any convex quadrilateral to any other.

OpenCV has a good library for doing perspective transformations if you don't want to roll your own.


What would be the correct way to calculate saturation in this case? I'm trying to make a HSV representation of the xyY color space. To calculate hue from an $(x, y)$ color, I use the angle between that color and red (wavelength 745) on the xy chromacity diagram, with white $(\frac{1}{3}, \frac{1}{3})$ as the center.

The saturation is the ratio between the distance between white and $(x, y)$, and white and a fully-saturated version of $(x, y)$ (which is the intersection between the line between $(\frac{1}{3}, \frac{1}{3})$ and $(x, y)$ and the edge of the chromacity diagram).

xy chromacity diagram:

The problem that I'm having is that when I plot my color space (at value=1) and compare it to the HSV representation of RGB, the saturation (distance from center) doesn't seem to match how "colorful" the color actually is:

My color space (saturation seems wrong):

HSV color space of RGB:

How should I calculate saturation instead?

There is unfortunately no good answer to this question. Simply it wont work. There is no good way to define colorful, it this context. Cie is trying to capture the physical measurement. It however does not succeed very well in relating the colors to each other.

Colors on the very outer arc represent spectral distributions of close to Dirac delta function. So one could construct a model that says that a color is very colorful when it is a Dirac delta.

There is a unforeseen consequence of this definition though. Namely the magenta colors do not exist as Dirac Deltas. As these colors do not exist in the spectrum. So they consist of mixture of 2 wavelengths only. This would mean they are less colorful than most other colors.

Other problems

Unfortunately, xyY is not perceptually uniform. So a straight line on the xyY does not represent interpolations between 2 color mixtures. Therefore making a polar transformation means you will have different color bases on same coordinates. Also precieved color do not really move over to your model. To do this properly you would need to do a extremely sophisticated transformation.

There are many problems with converting color to a polar coordinates in that that is exactly contrary how vision works. White is also a bit problematic in this context. The distance to full saturated signal is different for each of the 3 different cones in the eye. Hell, even what is while depends on the surrounding colors and ambient color conditions. So aim afraid your trying to force a worldview that does not exist.

Finally

What would this be useful for?


Fastest shader compilation strategy on windows dx9 and dx11? To compile shaders, I currently spin up a thread per core where each thread pulls a shader from a list and calls D3DX11CompileFromFile for dx11 and D3DXCompileShaderFromFile for dx9.

I've heard it mentioned that spinning up multiple processes instead of multiple threads can be faster, since there is some sort of per process driver contention.

Has anyone tried this, or does anyone have any tips on how to speed up shader compilation?

I have a modern machine w/ SSD drives.

From what I can find out (not having tried it myself), using a thread per core to do D3D compiles is reported to be safe and gives a good perf speedup, and there doesn't seem to be any contention between threads in the compiler itself.

However, Fabian Giesen notes that you might see lower perf when running under a debugger, due to contention in the Windows debug heap allocator, which can (and probably should) be disabled.

BTW, the functions you mentioned are both wrappers around D3DCompile, and don't involve the driver per se at all; they only do the GPU-independent "HLSL source to bytecode" step, and not the GPU-specific "bytecode to machine code" step. So they're entirely Microsoft code, not GPU IHV code.


Blending for Floating Point Texture I'm confused how does the GL_SRC_ALPHA_SATURATE function effect a 32 bit floating point texture attachment in FBO while blending colors. In the reference the equation is:

Here what is the value of kA for a 32-bit(GL_RGBA32F) texture?

According to wiki, when blending a floating point texture, the color value does not get normalized. If so then does min(As, ka - Ad) get divided by the kA?

And lastly, what happens if Ad is negative floating value?

Section "17.3.8 Blending" of the OpenGL spec states:

  If the color buffer is fixed-point, the components of the source and destination values and blend factors are each clamped to [0, 1] or [-1, 1] respectively for an unsigned normalized or signed normalized color buffer prior to evaluating the blend equation. If the color buffer is floating-point, no clamping occurs. The resulting four values are sent to the next operation.

So we know that no clamping occurs there. The next section is "17.3.8.1 Blend Equation" and states:

  Signed or unsigned normalized fixed-point destination (framebuffer) components are represented as described in section 2.3.4. Constant color components, floating-point destination components, and source (fragment) components are taken to be floating-point values. If source components are represented internally by the GL as fixed-point values, they are also interpreted according to section 2.3.4.

Finally, section "17.3.8.2 Blend Functions" refers to table 17.2 which explains that for GL_SRC_ALPHA_SATURATE:

RGB Blend Factors = (f, f, f)
Alpha Blend Factor = 1

where

$$ f = min(A_{s0}, 1 - A_d) $$

So it seems that the reference documentation is misleading for this particular case. However the specification is clear: there is no clamping or conversion for floating-point color buffers.

What if Ad is negative floating value?

My reading of the spec is that nothing special happens if Ad is negative. The min() operation will still produce results, but I don't see how they would be useful.

I've never used GL_SRC_ALPHA_SATURATE myself, but the way I understand it, with GL_FUNC_ADD, it is meant to accumulate color values in the color buffer until the accumulated alpha value equals 1. When the alpha value of the color buffer is 1, no further color contribution from new pixels is allowed. This assumes that you use alpha values between 0 and 1 and use an initial alpha value of 0. Otherwise everything falls apart.

In fact, I think majority of the blending factor mechanisms were designed with alpha values between 0 and 1 in mind and make little sense outside of that range.


setting up integer texture and binding as image pt. 2 Made a new question as suggested, follow up to that one.

I was experimenting with texture declaration to find solution and thats how I done that before (unsigned int):

glGenTextures(1, &m_texture);
glBindTexture(GL_TEXTURE_3D, m_texture);
glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA32UI, width, height, depth, 0, GL_RGBA_INTEGER,
GL_UNSIGNED_INT, NULL);
glGenerateMipap(GL_TEXTURE_3D);

glGenSamplers(1, &m_sampler);
glSamplerParameteri(m_sampler, GL_TEXTURE_MAG_FILTER, GL_NEAREST );
glSamplerParameteri(m_sampler, GL_TEXTURE_MIN_FILTER, GL_NEAREST );
glSamplerParameteri(m_sampler, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glSamplerParameteri(m_sampler, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glSamplerParameteri(m_sampler, GL_TEXTURE_WRAP_R, GL_CLAMP_TO_EDGE);

And for that using imagestore for int/unsigned int didnt work despite correct binding for writing as uimage , rgba32ui, uvec4 format and for reading as usampler, uvec4 format. Of course also binding sampler object for reading.

But I added two lines:

glGenTextures(1, &m_texture);
glBindTexture(GL_TEXTURE_3D, m_texture);
glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA32UI, width, height, depth, 0, GL_RGBA_INTEGER,
GL_UNSIGNED_INT, NULL);

glTexParameteri( GL_TEXTURE_3D, GL_TEXTURE_MIN_FILTER, GL_NEAREST ); //
glTexParameteri( GL_TEXTURE_3D, GL_TEXTURE_MAG_FILTER, GL_NEAREST ); //

glGenerateMipmap(GL_TEXTURE_3D);

glGenSamplers(1, &m_sampler);
glSamplerParameteri(m_sampler, GL_TEXTURE_MAG_FILTER, GL_NEAREST );
glSamplerParameteri(m_sampler, GL_TEXTURE_MIN_FILTER, GL_NEAREST );
glSamplerParameteri(m_sampler, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glSamplerParameteri(m_sampler, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glSamplerParameteri(m_sampler, GL_TEXTURE_WRAP_R, GL_CLAMP_TO_EDGE);

And now it seems to work. Still need to bind sampler thought. But I fail to see any logic in this. I use sampler objects to set texture filtering not glTexParameteri functions.

Float textures work with sampler objects as in first block of code. Anyone can explain?

I use sampler objects to set texture filtering not glTexParameteri functions.

No, you do not. Image Load/Store doesn't use samplers, so you cannot be using sampler objects to set the texture's filtering, when you're using the texture with Image Load/Store.

Sure, Image Load/Store doesn't use the built-in sampler data stored in a texture object either. However, those parameters do have an effect. They can determine if a texture is complete.

And whether you're using those parameters or not, if a texture is not complete, you can't access the texture from the GPU. Among the texture completeness rules is that the texture's sampling parameters must be valid, with respect to the texture's image format.

The default filtering parameters for textures are some form of LINEAR filtering; OpenGL does not change these defaults based on the image format. Therefore, a fresh texture with an integer image format will be incomplete unless you explicitly set the mag/min filters to GL_NEAREST (or NEAREST_MIPMAP_NEAREST for min).

You can get away with what you have when using the texture with sampler objects because their parameters override the completeness check. But as previously mentioned, Image Load/Store doesn't care about the bound sampler objects. Therefore, the completeness check must look at the texture's sampling parameters.

This is regardless of the fact that Image Load/Store itself doesn't actually use those parameters. So it's pointless, but you've gotta do it.


How to do error handling with OpenGL? Whenever I make semantic or syntax errors in OpenGL, either I get a black screen or the program crashes. I looked up on internet on how to do error handling in OpenGL and in the documentations I found glGetError(). To my understanding, glGetError() will return one or more error code(s) if I call it after calling a "normal" OpenGL function, provided I have made some error. Also I have to call it in while loop to get all the errors.

I have multiple problems with this. One I can't find the line number on which the error has occurred, second it doesn't tell me properly why the error occurred (not a big problem though) and lastly I have to include a while loop after each OpenGL function call.

Here's an example,

unsigned int buffer = 0;
glGenBuffers(1, &buffer);
glBindBuffer(GL_ARRAY_BUFFER, buffer);
glBufferData(GL_ARRAY_BUFFER, 8 * sizeof(float), vertices, GL_STATIC_DRAW);

Now if I write it with glGetError(), it would look something like

unsigned int err = 0;
glGenBuffers(1, &buffer);
while( !(err = glGetError()) ){
    std::cout << err;
}
glBindBuffer(GL_ARRAY_BUFFER, buffer);
while( !(err = glGetError()) ){
    std::cout << err;
}
glBufferData(GL_ARRAY_BUFFER, 8 * sizeof(float), vertices, GL_STATIC_DRAW);
while( !(err = glGetError()) ){
    std::cout << err;
}

Surely this isn't the way to go, right ? The source code looks so ugly.

How do I do proper error handling with OpenGL ?

My graphics card supports OpenGL 4.4 and I am using C++ .

Yes, there is a better way!  OpenGL 4.3 and later support the glDebugMessageCallback API, which allows you to specify a function in your app that GL will call to issue a warning or error. In this function you can do whatever you like, such as setting a breakpoint in the debugger, or printing the error to a log file. This way you only need to setup the callback during initialization—much nicer than putting glGetError calls everywhere.

To turn on this functionality your GL context also needs to be a "debug context", which is specified by setting a flag when you create the context. The details will depend on which OS / windowing system / framework you're using. A debug context might be slower (on the CPU) than a regular context, but that's the price you pay for more detailed debugging info, kinda like turning optimizations off in the compiler.

See also this wiki page for more information. There's a bunch of other debugging-related functionality. For instance you can give human-readable names to textures, buffers, framebuffers, etc using glObjectLabel, which will then (I assume) be used in error messages. There are also APIs for turning off certain messages or categories of messages, in case you get spammed by warnings that you don't care about or some such.


Tone Mapping in a physically based rendering ray tracer I'm implementing a pbr ray tracer. Let's say that I'm at the end of the process:
for each pixel pi, I accumulated the radiance value in vector pi = (radR, radG, radB). At this point, these values are unbounded, so I use the Reinhard tone mapper to have values between 0 and 1. Then I can scale it in the device range, for instance [0, 255] for usual computer images.

Here is my problem, the Reinhard tone mapping is not aware of the range of radiance in the whole scene. I mean to compute the tone mapping I do: pi = pi / (1 + pi).
So if pi = (100, 100, 100), the tone mapped pi is (100/101, 100/101, 100/101), but if pi = (1000, 1000, 1000), the tone mapped pi is (1000/10001, 1000/1001, 1000/10001). So the final values will be really close after tone mapping but one should be a lot greater than the other, right?

Am I doing something wrong, or missing something?

That's just how the Reinhard operator works. If the scene has very high dynamic range important detail may be lost near the high luminance region as you found since both will map near 0.99. Reinhard is a form of global operator. There are other types of algorithms using local operators which tonemap the pixel based on the intensity of the underlying neighborhood like a 3x3 or 5x5 region.

I'd recommend reading HDR Imaging Acquisition, Display and Image Based Lighting by Erik Reinhard. It's a very good book and explains much of the concepts.


Sine-based Tiled Procedural Bump I am trying to implement this formula to generate bump but I am facing some issue. The result doesn't look the same it's much darker.

Here is my result (without same parameters) but it is much darker and I don't get why.

And here is the associated code.

// randx, randy and frequencies are array with some random values for each sin wave.
for (int x = 0; x < _width; ++x)
{
    for (int y = 0; y < _height; ++y)
    {
        float color = 0.0f;
        for (int i = 0; i < _iterations; ++i)
        {
            val += Mathf.Sin(Mathf.Sqrt(Mathf.Pow(x - randx[i], 2.0f) + Mathf.Pow(y - randy[i], 2.0f)) * 1.0f / (2.08f + 5.0f * frequencies[i]));
        }
        color /= (float)_iterations;
    }
}

Any idea why I am getting this result ?
Thanks a lot !

EDIT: Thanks to @trichoplax it works by doing this.

float tmp = Mathf.Sin(Mathf.Sqrt(Mathf.Pow(x - randx[i], 2.0f) + Mathf.Pow(y - randy[i], 2.0f)) * 1.0f / (2.08f + 5.0f * frequencies[i]));
tmp = tmp * 0.5f + 0.5f;
val += tmp;

As you are taking the mean of a number of sine waves, your colour values will range from -1 to 1. From your example image, it looks like only the top half of this range of values (from 0 to 1) is resulting in colour, with everywhere else remaining black.

If whatever you are using to display the result can only handle positive values, then you will need to convert the result to the correct range of values. For example, to convert from the range [-1 to 1] into the range [0 to 1], you would add 1 and divide by 2.

In the context of your example code, this could be by following the line

color /= (float)_iterations;

with

color += 1;
color /= 2;


Anti-aliasing - Controlling colour density of pixel that comes within the "rectangle" surrounding my line I am currently a beginner when it comes to anti-aliasing. I have read some notes online that how anti-aliasing works is that you first draw the line using an algorithm such as Bresenham's algorithm. Then you modify it to include anti-aliasing.

Now, I understand that one method of anti-aliasing is to simply imagine there being a rectangle that encompasses the line and any pixel that touches that will be turned on or filled with an RGB value.

Now if I know that area of the segment of the pixel that is cut off by the rectangle (let's assume the pixel is a circle) how would I use that area of the segment to compute the density of the colour which is from 0-1 (where 0 is very light and 1 is the darkest)?

You have a background color and the color of your line, an antialiased line has additional lines drawn on either side of the first that are simply part way between the color and brightness of your origonal line and the background.
If the line you are drawing is horizontal or vertical the additional antialiasing lines (however many, depending upon the intended resulting line thickness) should be closer in color and brightness to the intended color - there is absolutely minimal antialiasing on such lines, that there is a tiny amount is due to offset subpixel rendering on some but not all matrixes.
This shows the difficulty:

When the lines are not horizontal or vertical it gets more complicated ...
If the additional lines drawn on each side of your line are drawn:

with color and brightness more closely matching the background's color then your line remains sharper and the background color near your line is fuzzier.
or

with color and brightness more closely matching your line's color then your line becomes fuzzier (wider) and the background color near your line remains sharper.

That means that if you intend to draw a thin line, a single pixel wide line, you would use the first option. (Make the result skinny).

If you want a two pixel wide line you would choose the second option but weigh the color of the background more towards the background's color and brightness. (Make the result not as skinny as a single line, with each of the two additional lines forming almost ~50% of the total width).

If you want a three pixel wide line you still use the second option but would instead weigh the color of the background more towards the line's color and brightness. (Make the result clearly much fatter than a single line).

For a four pixel wide line go back to the first method where you draw two lines of the intended color and brightness surrounded by two lines using the second method, making the two lines wider. (Sort of a blending of the first and second methods)

For five pixel wide lines the center is exactly your intended color, the two nearest surrounding line are very close to the color and brightness of the center line, with the outer two fairly close to the color of the center line (but less than the ones immediately next to the center line)

For a six pixel wide line there are two central lines of the intended drawing color and brightness, surrounded by two very similar lines, surrounded by another two less similar to the central two's color and brightness; but still very different from (lighter or darker) than the background color.

A seven pixel line or greater follows number 5 or 6 above.

The weighing of the additional lines more towards the intended color gives a sharper line, if you thought of it from a sideview analogy it would be top hat shaped. If you weighed additional lines more towards the background color the  (impossible) sideview analogy is that the profile of the 2D line would be rounded or triangular. If you are drawing 2D shapes it is in your interest to weigh towards the intended color. If you are rendering a 3D shape on a 2D screen then favoring the background color blends towards subsequent lines of a different Z depth.
A simplistic but effective algorithm with good speed is the Xiaolin Wu improvement to Bresenham's line drawing algorithm to implement antialiasing.
// Pascal
function plot(x, y, c) is
    plot the pixel at (x, y) with brightness c (where 0 ≤ c ≤ 1)

// integer part of x
function ipart(x) is
    return floor(x)

function round(x) is
    return ipart(x + 0.5)

// fractional part of x
function fpart(x) is
    return x - floor(x)

function rfpart(x) is
    return 1 - fpart(x)

function drawLine(x0,y0,x1,y1) is
    boolean steep := abs(y1 - y0) > abs(x1 - x0)

    if steep then
        swap(x0, y0)
        swap(x1, y1)
    end if
    if x0 > x1 then
        swap(x0, x1)
        swap(y0, y1)
    end if

    dx := x1 - x0
    dy := y1 - y0
    gradient := dy / dx
    if dx == 0.0 then
        gradient := 1.0
    end if

    // handle first endpoint
    xend := round(x0)
    yend := y0 + gradient * (xend - x0)
    xgap := rfpart(x0 + 0.5)
    xpxl1 := xend // this will be used in the main loop
    ypxl1 := ipart(yend)
    if steep then
        plot(ypxl1,   xpxl1, rfpart(yend) * xgap)
        plot(ypxl1+1, xpxl1,  fpart(yend) * xgap)
    else
        plot(xpxl1, ypxl1  , rfpart(yend) * xgap)
        plot(xpxl1, ypxl1+1,  fpart(yend) * xgap)
    end if
    intery := yend + gradient // first y-intersection for the main loop

    // handle second endpoint
    xend := round(x1)
    yend := y1 + gradient * (xend - x1)
    xgap := fpart(x1 + 0.5)
    xpxl2 := xend //this will be used in the main loop
    ypxl2 := ipart(yend)
    if steep then
        plot(ypxl2  , xpxl2, rfpart(yend) * xgap)
        plot(ypxl2+1, xpxl2,  fpart(yend) * xgap)
    else
        plot(xpxl2, ypxl2,  rfpart(yend) * xgap)
        plot(xpxl2, ypxl2+1, fpart(yend) * xgap)
    end if

    // main loop
    if steep then
        for x from xpxl1 + 1 to xpxl2 - 1 do
           begin
                plot(ipart(intery)  , x, rfpart(intery))
                plot(ipart(intery)+1, x,  fpart(intery))
                intery := intery + gradient
           end
    else
        for x from xpxl1 + 1 to xpxl2 - 1 do
           begin
                plot(x, ipart(intery),  rfpart(intery))
                plot(x, ipart(intery)+1, fpart(intery))
                intery := intery + gradient
           end
    end if
end function


Advantages of spherical harmonics over baked vertex lighting? My question is about spherical harmonic lightning, in the context of computing the coefficients with an expensive GI algorithm and passing them to the vertex shader

Another way is to just use an expensive GI algorithm to compute vertex colors and just use them instead.

What is the advantage of using the former method? I'm thinking that you can't do reflections with the latter method, and also there's rotational symmetry in SH (i don't know much about) that allows you to do some very simple dynamics with your lights and meshes. Is there any more points i'm missing?

Spherical harmonics are generally used for dynamic objects in your scene, while fully-baked lighting is used for static objects.

A typical game engine will use both. During the GI pass, all surfaces marked as static will have their global lighting fully baked - generally into a lightmap texture rather than per-vertex. At the same time, a bunch of light probes distributed throughout your scene will receive omnidirectional lighting, and this gets converted into SH. Since SH coefficients interpolate nicely, a mesh moving through your scene can pick up the nearest few light probes (generally 4), and use the weighted average to decode a good approximation of the ambient lighting at its current location.

Another use for SH is to render less-important dynamic lights in a scene with many lights in it. The most important light or two can be rendered fully dynamically per-pixel, then for each renderer, the SH coefficients of the less-important lights are calculated and added to the light probe data. This allows the lights to change in real-time, and be rendered effectively for free per-vertex in the shader, with a small CPU overhead.


Issue with declaring inputs to geometry shader? I am trying to pass vertex attributes from my vertex shader -> geometry shader and then to the fragment shader.

Pretty simple program, here is my vertex shader:

#version 440 core

uniform mat4 uModelViewMatrix;
uniform mat4 uProjMatrix;

in vec4 aPosition;

out vec3 modelPosition;

void main()
{
    modelPosition = aPosition.xyz;
    gl_Position = uProjMatrix * uModelViewMatrix * aPosition;
}

Here is my geometry shader:

#version 440 core

layout (triangles) in;
layout (triangle_strip, max_vertices = 3) out;

in vec3 modelPosition[];

out vec3 gModelPosition[];

void main()
{
    for (int i = 0; i < gl_in.length(); i++)
    {
        gl_Position = gl_in[i].gl_Position;
        EmitVertex();
    }
    gModelPosition[0] = modelPosition[0];
    gModelPosition[1] = modelPosition[1];
    gModelPosition[2] = modelPosition[2];
    EndPrimitive();
}

And here is my fragment shader:

#version 440 core

in vec3 gModelPosition;

out vec4 fragColor;

void main()
{
    fragColor = vec4(gModelPosition, 1.0);
}

I get the following error:

PROGRAM_LINKING_ERROR: error: "gModelPosition" not declared as input from previous stage

If I just comment out the last line in the fragment shader, and say fragColor = vec4(1.0); it works fine. And I clearly am declaring gModelPosition as an input from the previous stage.

Am I doing something wrong here?

The issue was that I declared out vec3 gModelPosition[] as an array in my geometry shader.

It should have only been declared as a vec3 (i.e. out vec3 gModelPosition)


Do modern GPUs contain barrel shifters? I'm aware that most modern GPUs, although designed for floating point, are more or less equivalent in integer performance these days, with a few caveats like the lack of a fused multiply add. I'm not sure how this applies to shift operations though. I'm doing Marching Cubes on GPU, initially writing out a 32-bit packed position for each surface cube then unpacking these in a later pass to the actual vertices in that cube, like this :

ivec3 unpackedPos = ivec3( packedPos >> 20 & 0x3FF,
                         packedPos >> 10 & 0x3FF,
                         packedPos & 0x3FF);

It just occurred to me to wonder if shader units have barrel shifters in them these days? Am I doing 2 shifts here or 30?

EDIT>>
I'm an idiot... Thanks for the answers guys, useful to know, but I've been going about this all wrong. I should just be using the RGB10_A2UI texture format then packing/ unpacking with a single image load/store instruction instead of messing around with bitshifts myself.

RE_EDIT>> Or not... This method apparently works on red boxes but not on green ones, so it's back to bit-shifts.

Yes ( with 50% of the FMA 32b throughput on nVIDIA Maxwell).

See
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#arithmetic-instructions


Clipping for perspective projection In the following two graphs,

the red lines are borders of the perspective-projection frustum,
the blue lines are to be rasterized,
the green lines are perspective-projection lines, which are wrong.

How do modern GPU pipelines do such clipping given only a perspective-projection matrix?

3D clipping is usually done in clipspace coordinates, that means the perspective matrix is used for sending the primitives to clipspace before doing the actual perspective projection. You can visualize the clipspace as a view frustum but with the shape of a box. So it is very fast to clip primitives against a AABB. The math is a little involved, but is all explained in this blog post:

Calculating the gluPerspective matrix and other OpenGL matrix maths | Unspecified Behaviour Blog


Texture Mapping jagged pixel edges I'm currently writing a software renderer. I managed to implement texture mapping and it works fine. Only issue is, that the edges between the pixels are sometimes jagged in x direction as you can see here:

Could this be caused by a lack of precision? If so, how can I improve it?
What I'm doing is simply stepping along the edges of a triangle and interpolating the UVs for each scanline.

EDIT:
Here's my code for edge stepping and drawing of scanlines. The Edge class has a method step which increments the interpolants for each scanline.

edge.h

Edge(const Vertex& minY, const Vertex& maxY){
    float dy = maxY.position.y - minY.position.y;

    yStart = static_cast<int>(ceil(minY.position.y));
    yEnd = static_cast<int>(ceil(maxY.position.y));

    float prestep = yStart - minY.position.y;

    xStep = (maxY.position.x - minY.position.x) / dy;
    x = minY.position.x + xStep * prestep;

    wStep = (1.0f / maxY.position.w - 1.0f / minY.position.w) / dy;
    w = 1.0f / minY.position.w + wStep * prestep;

    texCoordsStep = (maxY.textureCoordinates / maxY.position.w - minY.textureCoordinates / minY.position.w) / dy;
    texCoords = minY.textureCoordinates / minY.position.w + texCoordsStep * prestep;
}

renderTarget.cpp

void RenderTarget::DrawScanLine(const Edge& left, const Edge& right, int y){
    int xStart = static_cast<int>(ceil(left.GetCurX()));
    int xEnd = static_cast<int>(ceil(right.GetCurX()));
    float dx = static_cast<float>(xEnd - xStart);
    float prestep = xStart - left.GetCurX();

    float w = left.GetCurW();
    float wStep = (right.GetCurW() - w) / dx;

    glm::vec2 texCoordsStep = (right.GetCurTexCoords() - left.GetCurTexCoords()) / dx;
    glm::vec2 texCoords = left.GetCurTexCoords() + texCoordsStep * prestep;

    Uint32* p = pixels + xStart + y * surface->w;

    for(int x = xStart; x < xEnd; x++){
        float z = 1.0f / w;  //value for perspective correction
        int s = static_cast<int>(texCoords.s * z * texture->surface->w) & texture->surface->w - 1;
        int t = static_cast<int>(texCoords.t * z * texture->surface->h) & texture->surface->h - 1;

        *p = texture->pixels[s + t * texture->surface->w];  //copying the texture's pixel to the framebuffer
        p++;
        w += wStep;
        texCoords += texCoordsStep;
        color += colorStep;
    }
}

I strongly suspect that you are simply running out of floating point precision (at least in "U" dimension). With this sort of mathematics, the LSBs will soon become noisy resulting in the jagged appearance when you are (effectively) doing yes/no comparisons against known constants.

May I suggest instead that, rather than doing point sampling of your texture, try doing (at the very least) bilinear filtering which will hide the precision artefacts and also IMHO look better than having little squares.

UPDATE It just occurred to me that one of the possible causes of inaccuracy is that you are doing increments for each pixel X step, e.g. these two

w  += wStep;
texCoords += texCoordsStep;

These will soon cause errors to accumulate. Remove them and replace with multiplies by the X value and I expect things should improve.

UPDATE2 It will also be affected by how far away from the origin of the texture you are when you sample it. If, for example, your source texture is defined over [0..1]x[0..1] and then repeats outside of that range, you might expect sampling  at uv = (1000000.125, 100000.125) to be the same as (0.125,0.125) but, because the precision of FP decreases with increasing magnitude (i.e. the steps between floats also increases), you might end up with something quite different.


How does opengl fixed function pipeline determine specular lighting with an orthographic projection I have a scene set up with an orthographic projection with the near / far set from -5000 to 5000.  The model view is centered at 0 in the z-axis.  I am rendering a sphere centered at z=0 with lighting, and the specular component appears as to be calculated as if the eye were infinity far back.  See image below, in which the light only has a red specular component, no diffuse or other parameters.  However, if i render this with a custom shader that implements blinn lighting, the sphere appears black, because the "viewDir" calculation is negative, since the eye is centered inside the sphere.  How does fixed function get around this?  Does it some how know about the ortho projection and get around it?  Sources I've seen online say lighting is not supposed to take the projection into account.

This isn't necessarily due to the orthographic projection per se, but rather the classic two-sided lighting problem together with a little optimization that the old OpenGL did. When the eye space origin is inside the object, you are basically looking at the object from the inside and it's only the peculiarities of orthographic projection and the near/far planes you chose that makes the outsides appear.

It depends on how you actually implement your shader. I assume though, when you compute your viewDir, you just use the vector from the eye space vertex to the origin (or rather the negative normalized eye space vertex position). You are basically trying to light the backfaces of the sphere and if you haven't taken account for that, the angle between the view vector and the normal will thus be larger than 90°, usually resulting in a negative or 0-clamped specular term (or whatever other unexpected results).

A common way to accomodate for that would be to negate the normal vector whenever it is facing away from the camera, or maybe the light, or when dealing with backfacing fragments,... But this depends on your specific use-case.

Now, why doesn't this happen in old deprecated OpenGL when using orthographic projection?

For this we have to look at the specificities of how it did its lighting computations. There was a function glLightModel which controlled some of the global properties of lighting computations. Specifically, the paramter GL_LIGHT_MODEL_LOCAL_VIEWER. This controls an optimization where the specular computation does not use the actual vector from the vertex to the camera, but always just uses the z-axis instead. i.e. as you said it assumes an infinitely far away viewer along the z-axis. This is the reason why even in perspective projection the specular highlight on an object is always at the same position, even if you move it around in the view. If you change this setting from the default of GL_FALSE to GL_TRUE, it should perform an actual view vector computation and you should get the same results as with your own shader.

There was also another option that controls a property related to your general issue, too. GL_LIGHT_MODEL_TWO_SIDE enables two-sided lighting, which means that back-facing polygons will be lit like their front faces (however, possibly using different material settings) by using the inverted normal. If you activate this, this still won't change anything in your orthographic scene, since you are seeing the front faces of the sphere here. But once you change to perspective projection, the lighting should be black, but this time because you're seeing the backfaces, which you can enable lighting for with this option.


Better Shadow Mapping Techniques Currently I have basic shadow mapping setup in my engine. I am currently trying to implement something more realistic like PCSS (percentage closer soft shadows), to get a more physically accurate result just like the image below. (See how the shadow becomes softer & softer towards the end?)

My 2 questions are:

1) Is it worth implementing a shadow mapping technique like this? Will people actually notice the difference?

2) If it is worth it, what are some resources where I can learn more about it? What are the major techniques like PCSS?

For light sources with larger solid angle and where the shadow caster is relatively closer to the light than the receiver, you get notable soft shadowing effect. So if you render larger light sources closer to the shadow receiver it's important to handle soft shadows properly for realistic lighting. Even with the Sun which has quite small solid angle, you can still see fairly large penumbras from tall buildings.

PCSS is fairly simple algorithm for implementing contact hardening shadows, but it has some notable issues. PCSS fails when you have soft and hard shadows intersecting (blocker search fails), so it's not a good algorithm to use when large penumbras are needed. See in the below image where shadows from the box & triangle intersect. This artifact is particularly disturbing when animated.

There's a family of "back-projection" shadow techniques though, which is able to handle larger penumbras. However, there's a little research done how to implement this efficiently for real-time applications that I'm aware of.

In back-projection the principle idea is to project shadow map texels to the light source for every pixel to calculate light occlusion, which when implemented naively is obviously extremely slow and furthermore introduces shadow gaps & overlaps. However, there are some techniques to try to avoid the artifacts, such as adjusting the microquad extents as shown below.

Another improvement to handle the gaps and overlaps is to use bitmask soft shadowing algorithm, where occlusion bitmask determines the occluded areas of the light source.

The most recent paper I have seen on the topic is "Real Time Area Lighting. Now and Next" by Sam Martin from 2012, where the cost was several milliseconds for a single light, but it's definitely worth the read if you are interested of the topic, to get general idea about the state-of-the-art real-time shadow techniques.


A pseudocode to parallelize a path tracer? Where can I find a pseudocode to parallelize a path tracer using multiple cpu threads? How should I change the normal path tracer to be capable of parallelization?

An easy way to parallelize a path tracer is to have different threads (or machines!) work on different parts of the image.

For instance, you could break the image into a 20x20 grid and have each thread grab a tile, render it, and try to grab another tile.

A way I like to divy out this work is to use an atomic integer. Each thread atomically increments the integer and uses the value as an index into the work to do. They repeat this process until the atomic integer goes out of bounds.


Energy Conservation for Blinn-Phong BRDF I am trying to sample from the Blinn-Phong BRDF.

For testing, I use a spherical light source (Lambertian emission) sitting on a specular plane.  Here's a reference image using energy-conserving Phong with exponent 30000:

Notice that no energy is lost.  I would now like to achieve the same thing for Blinn-Phong.

Here's my attempt (same exponent, but on Blinn-Phong BRDF):

As you can see, a substantial portion of energy is lost.  The normalization term I am using comes from here and is $\frac{n+1}{2 \pi}$.  The problem is this is a normalization term, not a conservation term.

I believe this is the expected result.  It is common in graphics to make BRDFs reflect less than the input amount of energy, as opposed to exactly the right amount, as this is usually less difficult.1, 2

My question:

Is there a version that conserves all energy exactly?
Iff not, can someone at least confirm this is expected?

N.B. I am fairly confident this is not a ray generation/PDF issue.  These images were importance sampled using a method given in the PBRT book pg. 695 - pg. 699.  Moreover, a non-importance sampled version looks very similar (though I had to use a lower exponent to get it to converge fast enough, and it still took 10,000 s/p).

1This can be argued from a shadowing/masking argument on microfacets.  Unfortunately, it ignores multiple scattering.  The correct answer is somewhere in-between.

2Indeed, I had to derive the Phong conservation term myself.

First off, this is probably not the best way of testing energy conservation, because not all incident inclinations are visible in the image. Most of the environment is also black, so a less "peaky" BRDF such as Lambert would come out very dark with no way of telling whether it's energy conserving or not.

What is more commonly used is a so-called "furnace test", where you place a single sphere into a white environment map. When you render this sphere using the BRDF of interest, it should blend into the environment perfectly if the BRDF is energy conserving (make sure no color channels are clipping). Because it is a sphere, you will be able to see the BRDF at all incident inclinations. Additionally, the sphere is convex, so lacking any other objects in the scene, no reflected ray will be occluded, and every pixel of the sphere in the image will be an integral over the BRDF at a different inclination.

If you did this test, you would also notice that your Phong BRDF is most likely not energy conserving. The commonly used normalization term of $\frac{n+1}{2 \pi}$ (or $\frac{n+2}{2 \pi}$, depending on who you ask) only holds at normal incidence. At other inclinations, and particularly at grazing angle, up to half of the reflected rays will land below the surface and are usually terminated (because transmitting through a reflective surface does not make sense).
The correct normalization term that accounts for the part of the Phong lobe that is clipped below the hemisphere is very difficult to compute. Jim Arvo derived a closed-form solution for integer exponents using double-axis moments, summarised in the Global Illumination Compendium, paragraph 31a) and 31b) (page 17/18). These formulas are not very practical for use in a renderer though and I don't suggest implementing them.

It should be clear that if a simple model such as Phong is already this difficult to normalize, then Blinn-Phong is even more difficult. First of all, the $\frac{n+1}{2 \pi}$ normalization term you mention is only correct if you use Blinn-Phong as a microfacet distribution inside a full microfacet BRDF (e.g. Torrance-Sparrow or Walter). If you directly use the Blinn-Phong BRDF, then a more correct normalization term is $\frac{(n+2)(n+4)}{(8 \pi 2^{-n/2} + n)}$, derived in this short article by Fabian Giesen (second page) and also listed on the page you mention. However, this normalization term is again only correct at normal incidence and will not be exact at other inclinations. From what I can tell, no exact normalization term exists.


Derivative maps vs. Tangent Space Normal maps I discovered that some engines use derivative maps instead of tangent space normal maps.

After some reading, it seems to be a really awesome way to replace tangent space normals but are there some disadvantage using them ? Why still continue using tangent space normals ?

Is it possible to compare both with advantage and disadvantage ?

After some researches and some answers from professionals here is my conclusion.

Pros

Don’t require tangents or binormals. Less interpolators.
Only need two channels. less texture memory.
Don’t suffer from tangent seams.
Can be blended using alpha blending, without renormalization.
Less mesh memory: We don’t need to store a tangent vector.
Fast implementation.

Cons

More ALU
Less flexible. A normal map can represent any derivative map, but the reverse is not true. As an example, sharp edges can be difficult to represent.

So, that's a lot of pros compared to cons. But the major problem is that it's not an industry standard.
There are almost no content authoring tools nor artist know-how.

Here's is a quote from Bart Wronski that illustrates well the current status of derivative maps :

  Sadly in life / technology not always best solution wins / not even
  gets deserved attention... It's more about standards and inertia.

If you are interested in knowing more about derivative maps here are some interesting articles.

Mikkelsen and 3D Graphics: Derivative Maps | Morten Mikkelsen Blog
Derivative Maps | CodeItNow
Derivative Maps vs Normal Maps | CodeItNow
Triplanar Texturing with Derivative Maps | The Robot Seven
Derivative Normal Maps - What are they? | polycount.com
Derivative Maps | Knald Technologies Documentation

If I forgot something or if your don't agree feel free to tell in the comments, I would be glad to improve this answer.


How to get the transform amount from a screen vector and a direction vector? I am working on the move tool in a 3D modeling software. I need to know how much to move when the user drags an axis of the gizmo (for example along the x axis).

I have the 2D vector describing how much the mouse moved on the screen MouseDelta, and the vector corresponding to the selected axis AxisVector.

I would like to know the best way to compute the amount of extrusion I should apply.

Currently I try to project the AxisVector in screen space, project MouseDelta on it and get the ratio between the size of the AxisVectorOnScreen and the computed projection. Then I have to clamp the result and multiply it by a factor. Here is a pseudo code:

Vector3 AxisVectorOnScreen = GetViewProjMatrix().TransformVector(AxisVector);
Vector2 AxisVectorOnScreen2D(AxisVectorOnScreen .X * 0.5f * Screen.Width(), AxisVectorOnScreen .Y * 0.5f * Screen.Height());
Vector2 MouseDelta = EndMousePosition - BeginMousePosition;
float Size = AxisVectorOnScreen2D.Length();
float ProjectionLength = MouseDelta | AxisVectorOnScreen2D.GetSafeNormal();
float Ratio = Size / ProjectionLength;
float Amount = 10.f * FMath::Clamp(Ratio, -2.5f, 2.5f);

I think I could also project my mouse vector on a plane perpendicular to the camera and do the same maths on this plane. What is the best solution?

Note: I have the same problem to get the amount of extrusion I should apply with my extrude tool (along the normal of a face).

I would not recommend to correlate the extrusion direction with its screen projection because of the asymptotic behaviour in case the axis points toward the screen or the resulting extrusion crosses the focal point of the camera.

One consistent way of doing it would be computing the extrusion height independently of the axis direction:

Find a virtual construction plane of your operation which is parallel to the screen. For example, in the real world, this plane would be a plane of your computer monitor screen reachable by hand as you sit at your computer.

Usually, you have some kind of central point which serves as a center of screen camera rotation as you navigate your viewport. You can draw your construction plane through this point.

Another way is drawing your construction plane through the intersection of the mouse cursor and your object as you start your extrusion operation.

Either way, the construction plane must be parallel to the screen and in front of your camera view for this method to work.

If you are using an orthogonal projection (camera without a Field of View) then you can draw the construction plane through any point.

Compute the length of the projection of mouse motion on the construction plane your figured out in point (1). This is your desired extrusion height.
Provide a means of moving the construction plane back and forth along the view axis, if the current viewport has FOV. You don't need to display and the move the construction plane if your viewport uses orthogonal projection.

Using this method your extrusion operation height would be visually consistent with what user expects with mouse motion, and the user would not experience any asymptotic and singularity issues as with the axis projection method.


Why do red, green, and blue make up all the colors? Why do red, green, and blue combinations can make up all the visible colors?

Let's reminds ourselves what light is.

Radio waves, micro waves, X rays and gamma rays are all electromagnetic radiation and they only differ by their frequency. It just so happens that the human eye is able to detect electromagnetic radiation between ~400nm and ~800nm, which we perceive as light. The 400nm end is perceived as violet and the 800nm end is perceived as red, with the colors of the rainbow in between.

A ray of light can be a mix of any of those frequencies, and when light interacts with matter, some frequencies are absorbed while other might not: this is what we perceive as the colors of objects around us. Unlike the ear though, which is able to distinguish between a lot of sound frequencies (we can identify individual notes, voices and instruments when listening to a song), the eye is not able to distinguish every single frequency. It can generally only detect four ranges of frequencies (there are exceptions like daltonism or mutations).

This happens in the retina, where there are several kinds of photo-receptors. A first kind, called "rods", detects most frequencies of the visible light, without being able to tell them apart. They are responsible for our perception of brightness.

A second kind of photo-receptors, called "cones", exists in three specializations. They detect a narrower range of frequencies, and some of them are more sensitive to the frequencies around red, some to the frequencies around green, and the last ones to the frequencies around blue.

Because they detect a range of frequencies, they cannot tell the difference between two frequencies within that range, and they cannot tell the difference between a monochromatic light and a mix of frequencies within that range either. The visual system only has the inputs from those three detectors and reconstruct a perception of color with them.

For this reason, the eye cannot tell the difference between a white light made of all the frequencies of the visible light, and the simple mix of only red green and blue lights. Thus, with only three colors, we can reconstruct most colors we can see.

By the way, rods are a lot more sensitive than cones, and that's why we don't perceive colors in the night.


How to render and store a color gradient in a ramp texture for 16bit dataset? I'm working on a scientific visualization project and need to create a transfer function widget. The volume data is either in range 0-255 or it can be 16 bit i.e. 0-65535. Now the main problem is in order to color certain sections, the user needs to be able to define certain control points with varying colors and the colors need to be smoothly transitioned between those control points. Then during rendering, we find the color for every isovalue through a texture lookup in a ramp texture and color the volume data.
I've decided to do cubic spline interpolation between these control points. My question is, how do I actually render and store this gradient in an Nx1 ramp texture. Firstly, regarding the rendering part. I'm using OpenGL but also Dear-ImGui for GUI. So I don't have low level access to rendering like vertex/fragment shader for rendering this gradient. Just low level functions like rendering lines/rectangles etc.
For 8 bit volumes, I could render 256 lines of thickness gradient_width_in_pixels/256. I think this would give me the look of a rectangular gradient? However suppose I have a 16 bit volume. The max value now is 65k. I could go on and convert it to 0-255 but let's say I don't because there would be a loss of precision as in the 0-255 scale 1 value represents approximately 256 values of 0-65535 scale. What if the user wanted to color within that range? What's the proper way to handle this scenario now?
Secondly, regarding storing the gradient information. For 8bit datasets, again it'd seem pretty simple to just create an 256x1 texture. However the problem again rises for 16 bit datasets. 65k pixels don't sound good or are they okay? I know 65k x 1 texture is just a 256x256 texture but dunno if increasing only 1 dimension will cause problems.
EDIT: I thought I should clarify regarding rendering the gradient part and ask the question differently. What I wanna know is given a color control point at pixel coord (x1,y) and another point at (x2, y), how many lines of any thickness should I render to get a color gradient with smooth transition between those two colors?

It may be easier to compute the gradient on the CPU, even with 16bit precision the CPU can grind out a texture quickly. Then just upload that to the GPU as an image. Also, it doesn't sound like users will be able to pick a new color set every frame so its sounds a lot more natural to have a work flow of: Pick colors -> Generate Image -> upload to the GPU -> use new palette to render.
Are the picked colors RGB choices? If yes then it does make sense to spread the color palette out as wide as possible (maybe even more then 16bits) 65536 pixels is just a drop in the bucket. And a nice wide color range could lead to less banding in the final image.
At some  level the code will have to narrow the range down to whatever the current precision is. If that is the table, and the table has 65536 positions, then the table might contain a lot of repeating colors, but it is easy to compute the table with a new range.


How to Calculate Surface Normals for Generated Geometry I have a class that generates a 3D shape based on inputs from the calling code. The inputs are things like length, depth, arc, etc. My code generates the geometry perfectly, however I am running into trouble when calculating the surface normals. When lit, my shape has very bizarre coloring/texture from the incorrect surface normals that are being calculated. From all my research I believe my math is correct, it seems that something is wrong with my technique or method.

At a high level how does one go about programmatically calculating the surface normals for a generated shape? I am using Swift/SceneKit on iOS for my code but a generic answer is fine.

I have two arrays that represent my shape. One is an array of 3D points that represents the vertices that make up the shape. The other array is a list of indexes of the first array that map the vertices into triangles. I need to take that data and generate a 3rd array that is a set of surface normals that aid in the lighting of the shape. (see SCNGeometrySourceSemanticNormal in SceneKit`)

The list of vertices and indexes is always different depending on the inputs to the class so I cannot pre-calculate or hard code the surface normals.

I see mainly three ways of computing normals for a generated shape.
Analytic normals
In some cases you have enough information about the surface to generate the normals. For example, the normal of any point on a sphere is trivial to compute. Put simply, when you know the derivative of the function, you also know the normal.
If your case is narrow enough to allow you to use analytic normals, they will probably give the best result in terms of precision. The technique doesn't scale too well though: if you also need to handle cases where you cannot use analytic normals, it may be easier to keep the technique that handles the general case and drop the analytic one altogether.
Vertex normals
The cross product of two vectors gives a vector perpendicular to the plane they belong to. So getting the normal of a triangle is straightforward:
vec3 computeNormal(vec3 a, vec3 b, vec3 c)
{
    return normalize(crossProduct(b - a, c - a));
}

Moreover, in the above example, the length of the cross product is proportional to the area inside abc. So the smoothed normal at a vertex shared by several triangles can be computed by summing up the cross products and normalizing as a last step, thus weighting each triangle by its area.
vec3 computeNormal(vertex a)
{
    vec3 sum = vec3(0, 0, 0);
    list<vertex> adjacentVertices = getAdjacentVertices(a);
    for (int i = 1; i < adjacentVertices; ++i)
    {
        vec3 b = adjacentVertices[i - 1];
        vec3 c = adjacentVertices[i];
        sum += crossProduct(b - a, c - a);
    }
    if (norm(sum) == 0)
    {
        // Degenerate case
        return sum;
    }
    return normalize(sum);
}

If you are working with quads, there is a nice trick you can use: for a quad abcd, use crossProduct(c - a, d - b) and it will handle nicely cases where the quad is in fact a triangle.
Iñigo quilez wrote a few short articles on the topic: clever normalization of a mesh, and normal and area of n sided polygons.
Normals from partial derivatives
Normals can be computed in the fragment shader from the partial derivatives. The math behind is the same, except this time it is done in screen space. This article by Angelo Pesce describes the technique: Normals without normals.


Cause of shadow acne I know how shadow mapping works but I am not getting the cause of shadow acne!
Can anyone tell me cause of shadow acne in a simple way and how is it related to depth map resolution?

Image 1: A bad case of shadow acne. (Synthetic and a bit exaggerated)

Shadow acne is caused by the discrete nature of the shadow map. A shadow map is composed of samples, a surface is continuous. Thus, there can be a spot on the surface where the discrete surface is further than the sample. The problem does persist even if you multi sample, but you can sample smarter in ways that can nearly eliminate this at significant cost.

Image 2: A side cutaway of a shadow function and its discrete samples.

The canonical way to solve this is to offset the shadow map slightly so the object no longer self shadows itself. This offset is called a bias. One can use more smart offsets than just a fixed value but a fixed value works quite well and has minimal overhead.

Image 3: Shadow function biased (offset) forward.


What does "st" mean in the context of OpenGL? I've seen the term st come up a few times when looking at other OpenGL fragment shaders. However, I don't know what it stands for or what it's used for. Here's an example:

uniform vec2 u_resolution;

vec2 st = gl_FragCoord.xy/u_resolution;

I can understand that this is converting the pixel coordinates into normalized coordinates of 0.0 - 1.0 instead of 640 x 480 (for example).

I've also seen things like position.st.

What does it mean?

The common names for the components of texture coordinates are U and V. However, because the next letter used (for 3D textures) is W, that causes a conflict with the names for the components of a position: X, Y, Z, and W.

To avoid such conflicts, OpenGL's convention is that the components of texture coordinates are named S, T, and R. Thus, you have function calls like this:

glTexParameter(..., GL_TEXTURE_WRAP_S, GL_REPEAT);

Where GL_TEXTURE_WRAP_S refers to wrapping in the S component of the texture coordinate.

Unfortunately, OpenGL picked S, T, and R long before GLSL and swizzle masks came around. R, of course, conflicts with R, G, B, and A. To avoid such conflicts, in GLSL, the texture coordinate swizzle mask uses S, T, P, and Q.

In GLSL, you can swizzle with XYZW, STPQ, or RGBA. They all mean exactly the same thing. So position.st is exactly the same as position.xy. However, you're not allowed to combine swizzle masks from different sets. So position.xt is not allowed.


How is Gaussian Blur Implemented? I've read that blur is done in real time graphics by doing it on one axis and then the other.

I've done a bit of convolution in 1D in the past but I am not super comfortable with it, nor know what to convolve in this case exactly.

Can anyone explain in plain terms how a 2D Gaussian Blur of an image is done?

I've also heard that the radius of the Blur can impact the performance. Is that due to having to do a larger convolution?

In convolution, two mathematical functions are combined to produce a third function. In image processing functions are usually called kernels. A kernel is nothing more than a (square) array of pixels (a small image so to speak). Usually, the values in the kernel add up to one. This is to make sure no energy is added or removed from the image after the operation.

Specifically, a Gaussian kernel (used for Gaussian blur) is a square array of pixels where the pixel values correspond to the values of a Gaussian curve (in 2D).

Each pixel in the image gets multiplied by the Gaussian kernel. This is done by placing the center pixel of the kernel on the image pixel and multiplying the values in the original image with the pixels in the kernel that overlap. The values resulting from these multiplications are added up and that result is used for the value at the destination pixel. Looking at the image, you would multiply the value at (0,0) in the input array by the value at (i) in the kernel array, the value at (1,0) in the input array by the value at (h) in the kernel array, and so on. and then add all these values to get the value for (1,1) at the output image.

To answer your second question first, the larger the kernel, the more expensive the operation. So, the larger the radius of the blur, the longer the operation will take.

To answer your first question, as explained above, convolution can be done by multiplying each input pixel with the entire kernel. However, if the kernel is symmetrical (which a Gaussian kernel is) you can also multiply each axis (x and y) independently, which will decrease the total number of multiplications. In proper mathematical terms, if a matrix is separable it can be decomposed into (M×1) and (1×N) matrices. For the Gaussian kernel above this means you can also use the following kernels:

$$\frac1{256}\cdot\begin{bmatrix}
1&4&6&4&1\\
4&16&24&16&4\\
6&24&36&24&6\\
4&16&24&16&4\\
1&4&6&4&1
\end{bmatrix}
=
\frac1{256}\cdot\begin{bmatrix}
1\\4\\6\\4\\1
\end{bmatrix}\cdot\begin{bmatrix}
1&4&6&4&1
\end{bmatrix}
$$

You would now multiply each pixel in the input image with both kernels and add the resulting values to get the value for the output pixel.

For more information on how to see if a kernel is separable, follow this link.

Edit: the two kernels shown above use slightly different values. This is because the (sigma) parameter used for the Gaussian curve to create these kernels were slightly different in both cases. For an explanation on which parameters influence the shape of the Gaussian curve and thus the values in the kernel follow this link

Edit: in the second image above it says the kernel that is used is flipped. This of course only makes any difference if the kernel you use is not symmetric. The reason why you need to flip the kernel has to do with the mathematical properties of the convolution operation (see link for a more in depth explanation on convolution). Simply put: if you would not flip the kernel, the result of the convolution operation will be flipped. By flipping the kernel, you get the correct result.


OpenGL: Rendering 2D shadows I've been working on a 2d shadow system for a school project, using this method to generate visibility polygons. However, I'm having issues figuring out how to draw those visibility polygons in a way that reveals everything drawn within them, rather than drawing over them like a normal sprite would.

From what I've been able to figure out, it seems like what I want to do is to draw the visibility polygons to some texture first, and then use that texture somehow when drawing the rest of the objects on the screen to get the shadow/light effect.

Unfortunately all the tutorials I've been able to find aren't very helpful. So, my questions are:

1) How do I tell OpenGL to render to a texture instead of the screen?

2) How do I use that texture to create the effect I want?

You don't need to render to texture to achieve an effect like this. Use stencil buffer.

The visibility polygon can be drawn using GL_TRIANGLE_FAN, if you start from the middle:

At the beginning of each frame:

Clear the stencil buffer (and enable stencil test)
Set the stencil function to always pass, and to replace the current value in the buffer to a greater than zero number (for example one)
Draw the visible part as a GL_TRIANGLE_FAN. If you have multiple characters/lights, draw the visibility polygon for each.

Now the stencil buffer contains 1 at visible locations and 0 where the character can't see.

Once you have a stencil buffer like this, you can control which objects you want to draw using glStencilFunc. Do not write to the stencil buffer, just use the stencil test to discard fragments which are visible/invisible.

If you want to draw a background which is visible everywhere, use GL_ALWAYS. For objects which are only displayed in the visible area use GL_EQUAL and 1. Similarly you can draw something (for example a dark quad) just in the invisible parts of the scene with GL_EQUAL and 0.

If it is not clear, how to use stencil buffers, check out this tutorial


How to display an in-memory framebuffer I'm planning to write a small software rendering engine (before anyone asks, it's kind of a learning/scientific experiment for me).

Before writing any code, I'm already stuck at the very first step: how to display my framebuffer on the screen? What I'm planning to do is to render into an in-memory buffer e.g. 320x200 24-bit RGB format. Then of course I need to display the results on the screen; full-screen or in a window.

I know that similar questions already exists here:

How can I send custom bitmap to frame buffer?
How are digital painting canvas rendered?

The difference is that I'd like to update the image frequently, with a decent framerate like 50FPS.

It seems one possibility would be OpenGL framebuffer objects and/or buffer streaming. I'm not sure I should go that way or is it an overkill for this use case?
I've also considered displaying a simple rectangle with a texture I would update each frame, but I'm not sure about the performance (I guess transferring a few megabytes of data each frame shouldn't be a problem though).
Several years earlier I've experimented with GDI+ and WPF, but the performance was nowhere near what I wanted to achieve. Unfortunately I don't have the source code any more, but I'd admit that my approach was maybe completely wrong back then.

I'm not expecting source code or a ready solution, but rather some directions: what functionality to use, I'd be happy to read articles or a tutorial about the topic.

I was considering OpenGL first, but I'm fine with DirectX too. Other suggestions are welcome. I'm planning to use C/C++ or C#, maybe Python for the implementation.

I'm sure there must be an easy solution, but I admit I'm a complete noob when it comes to graphics programming. I guess it's just about finding an effective way to copy data from RAM to GPU memory. I was wondering how retro emulators (e.g. Vice) or fantasy consoles (e.g. Pico-8) are solving this problem?

Have you considered using SDL 2?
My use-case is basically identical to yours, and I've found this approach to work really well. They have a really nice C (and C++) API for drawing framebuffers to a window. It's also platform-agnostic, so you can share your implementation fairly easily between macOS, Linux and Windows.
If you want to see an example, you can go check out this module here on GitHub. I do warn you though, while this implementation works, and I can easily run it at 60-120FPS, it likely does use more CPU than a fully optimised implementation.
Specifically observe the initDisplay() and drawWindow() functions.


Will the gaussian kernels in this fragment shader be computed for every fragment? I am running this fragment shader on every pixel on screen. I am trying to make it as efficient as possible. BTW I am running open gl es 2.0.

The code segment below is only a sample from the code, there are about 56 different calls to Gaussian() spread across different functions. I am wondering wether it would be valuable to replace the calls to Gaussian() with there appropriate resulting float value.

I know a lot of times stuff like this is pre-calculated on compilation of the shader, or calculated only once because the gpu realizes it is the same calculation for all fragments.

So would it be worthwhile for me to manually calculate each of these and replace them with their values?

uniform float scalar;
varying float scalart;
float deviationScale = 1.2;
float finalScalar = 1.1;
float Gaussian(float x, float deviation)
{
    return (1.0 / sqrt(2.0 * 3.141592 * deviation)) * exp(-((x * x) / (2.0 * deviation)));
}
vec3 blurS5()
{
    vec3 blr = vec3(0.0);
    blr += texture2D(s_texture, (v_texcoord + vec2(2.0 * scalart, 0.0))).xyz *  finalScalar * Gaussian(2.0, 2.0 * deviationScale) ;
    blr += texture2D(s_texture, (v_texcoord + vec2(1.0 * scalart, 0.0))).xyz *  finalScalar * Gaussian(1.0, 2.0 * deviationScale) ;

    blr += texture2D(s_texture, (v_texcoord + vec2(0.0 * scalart, 0.0))).xyz *  finalScalar * Gaussian(0.0, 2.0 * deviationScale) ;

    blr += texture2D(s_texture, (v_texcoord + vec2(-1.0 * scalart, 0.0))).xyz *  finalScalar * Gaussian(-1.0, 2.0 * deviationScale) ;
    blr += texture2D(s_texture, (v_texcoord + vec2(-2.0 * scalart, 0.0))).xyz *  finalScalar * Gaussian(-2.0, 2.0 * deviationScale) ;
    return blr;

}

void main()
{
        if (scalar == 2.)
        {
            gl_FragColor = vec4(blurS5(), 1.0);
        }
        else if (scalar == 3.)
        {
            gl_FragColor = vec4(blurS9(), 1.0);
        }
        //There are waaayyy more of these
}

It depends. On a desktop, if the Gaussian function is always called with the same arguments, the shader compiler will probably optimize it so it's evaluated only once. On a mobile platform though, the compiler has less time to do optimizations and might miss it. Either way, you can confirm by looking at the generated assembly.

Note that Aras Pranckevičius of Unity wrote a GLSL optimizer to address the problem of sub-optimal compilers on mobile: https://github.com/aras-p/glsl-optimizer

That being said, regarding your sample code, I would definitely at least expose (1.0 / sqrt(2.0 * 3.141592 * deviation)) as a uniform.


Worst Case Scenario for Cohen-Sutherland Line Clipping Algorithm I'm currently enrolled in an entry level Computer Graphics class, and as I'm studying for my final, I realize I have a question regarding the Cohen-Sutherland line clipping algorithm. I understand the basics of the algorithm, such as how to compute the 4-bit outcodes associated with each region and the test conditions for the endpoints of a line segment, but where I'm struggling is how to determine what the worst case scenario is for the algorithm.

I've had the following question on both my midterm and a homework assignment: "Draw two line segments (one with a positive slope and one with a negative slope) that reflect the worst case scenarios for the corresponding checking order."

The following image file shows what the checking order was for each question, along with my original answers to the questions followed by the correct answers. If anyone could point me in the right direction, it would be greatly appreciated. The only "answer" from my professor I received when asking for an explanation was "the worst case has to do with the checking order", and that answers absolutely nothing for me.

Maybe it helps to draw out the steps of the algorithm. Here is an example for TBLR

The red is the worst case: clip top, clip bottom, clip left, clip right. Four steps!

The green is the best case: clip top, clip bottom, done. Two steps.

The blue is a middle case: clip top, clip bottom, clip left, done. Three steps.


Ray Tracing Shadows: The Shadow Line Artifact By using smooth shading (i.e. interpolation of normals), an object can have a smooth look despite low polygon count. A nasty artifact that can happen when casting shadow rays is the Shadow Line Artifact. Since shadow rays originate from a certain position and don't usually take the normal into account, a complete face on a smooth object will either be shadowed or not, revealing the hard edges again:

The shadow is, so to speak, 'correct' and the weird appearance results from the normal map interpolation 'cheat'. Nevertheless, it is visually distracting. The linked article above, written in 2009, suggests removing shadows or self-shadows on objects that show this kind of artifact, which (especially for non-convex objects) is not a satisfying solution.

This is a very common problem inherent to ray tracing algorithms. I wonder if there is any better solution to it than subdividing the mesh into flat-shaded high-poly or getting rid of the shadows altogether.

The shadow terminator problem (as it's also known) is still something of an open problem in ray-tracing. As you say, disabling self-shadowing is not a very good workaround, because non-convex meshes often need to self-shadow. There's a possible refinement to this method. Instead of throwing away all shadow ray intersections with the same object, you can only throw away the intersection if it's with the same object and it's through the reverse face of the triangle (i.e. discard the intersection if $d\dot\ N > 0$, where $N$ is the geometric normal of the intersected polygon and $d$ is the shadow ray direction).

One alternative that I've used successfully is to offset the origin of the shadow ray a short distance along the surface normal. This way, the shadow ray originates not at the geometric surface of the mesh (which is only an approximation) but at the hypothetical surface of the real object.

In theory, you should choose the distance along the normal according to the point's distance from that polygon's vertices, so that the origin is on the approximated surface, but in practice, a small fixed offset works for almost all situations.

You should also be sure to use the normal rather than the geometric surface to decide whether the light is "behind" the surface. That is, if $N\dot\ L < 0$, the shading point is in shadow (without needing to cast a ray), where $N$ is the interpolated and shaded normal.


Three quaternions interpolation with barycentric coordinates I have three quaternions A, B, C.

I also have a 3D vector P which describes the barycentric coordinate to interpolate.

I want to interpolate the three quaternions with respect to the barycentric coordinate P

This Image shows how to do it would A, B and C be usual vectors.

but how to calculate the quaternion interpolation?
Can I use the "slerp" command somehow?
How would the glm code look like?

Defining the problem
The weighted arithmetic mean $\mu$, of a set of vectors, minimizes squared L2 norm of the error ($v_i$) to all elements of the set.
$$
\begin{equation}
\begin{split}
v_i &= \mu - x_i \\
\mu &= \underset{\mu \in \mathbb{R^n}}{\text{argmin}} \; \frac{1}{2}\sum\limits_{v_i} w_i ||v_i||^2_2 \\
\end{split}
\end{equation}
$$
This empowers us to define special means, where we use different notions of distance.
Specializing for rotations
A reasonable notion of distance for SO(3) is the angle separating two rotations. That's $||\log(\mu \ q^{-1})||$ Your geometry library probably implements this $log$, and expressions for it can be found here. It is not the log one is accustomed to on the real numbers.
Intuition: Recall that every sequence of rotations corresponds to a single rotation about a single axis, so the difference between a pair of rotations can always be summarized by a single angle.
$$
\begin{equation}
\begin{split}
v_i &= log(\mu \ x_i^{-1}) \\
\mu &= \underset{\mu \in \mathbb{SO3}}{\text{argmin}} \sum\limits_{v_i} w_i || v_i||^2_2 \\
\end{split}
\end{equation}
$$
There's no analytical form for this minimum for more than two rotations. We can, however, treat this as a least squares problem and compute the result by Gauss-Newton.
To apply Gauss-Newton in general, for some error, we need to be able to find $\mathbf{J_i} = \frac{d}{d\mu}v_i(\mu)$, which requires differentiating $v_i$. Then we can take a step as
$$
\begin{equation}
\begin{split}
\Delta \mu &= \sum\limits_{i}(w_i \mathbf{J_i}^T \mathbf{J_i})^{-1} \sum\limits_{i} w_i \mathbf{J_i}^T v_i
\end{split}
\end{equation}
$$
Our error function cannot readily be differentiated, since its argument is a rotation $\mu$ and not a vector. We invent a parameter $\delta \in \mathbb{R^3}$ which we will $Exp$ into a rotation. This, we can differentiate
$$
\begin{equation}
\begin{split}
v_i &= log(Exp(\delta)\; \mu \ x_i^{-1}) \\
\mathbf{J_i} &= \frac{d}{d\delta} v_i = \mathbf{I} \\
\end{split}
\end{equation}
$$
So the update to $\mu$ is
$$
\begin{equation}
\begin{split}
\delta &= \sum\limits_{i}(w_i \mathbf{J_i}^T \mathbf{J_i})^{-1} \sum\limits_{i} w_i \mathbf{J_i}^T v_i\\
\delta &= \sum\limits_{i} w_i \sum\limits_{i} w_i v_i \\
\delta &= \sum\limits_{i} w_i v_i
\end{split}
\end{equation}
$$
Conclusion
$$
\begin{equation}
\begin{split}
\delta = \sum\limits_{i} w_i v_i &= \sum\limits_{i}w_i log(\mu_{k}^{\mathstrut} x_{i}^{-1}) \\
\mu_{k + 1} &= Exp(\delta) \mu_{k}\\
\end{split}
\end{equation}
$$
This will converge quadratically to a mean.
Caveat emptor: for rotations, there may be many "means". By analogy: What is the average point, on Earth's surface, of the north and south poles? Every point on the equator has equal right to be called the average. This is the case for all equispaced rotations.