Visual Artifact in Wireframe Grid/Plane I have a simplistic shader program that adds reference coordinate planes to a scene.

These planes are drawn as intersecting lines.

When viewed at a distance, a visual artifact is created; there is a ripple of lighter and darker areas going left to right across the image.

Given how simple the shader program is, I think this must be due to the interpolation that occurs between the vertex and fragment shaders.

(a) Does this type of artifact have a name?
(b) What causes it?

I looked into whether this was being caused by anti-aliasing. However, disabling the anti-aliasing shows that the pattern is created beforehand. See here:

in short:

this is a Moire pattern (see lfgtm's answer)
it is caused by trying to map a perspective projection of parallel lines onto a fixed grid of pixels. (see zoran's answer)

When looking at ways to address this problem, I thought that drawing the lines as surfaces with a gradient transparency across the width of the line would help by allowing the line to straddle pixels and giving it a more accurate width.

Here is how the grid looks when drawn as simple lines vs surfaces with gradient transparency.

This improved the definition between lines at a distance (i.e. they don't overlap each other and create a solid plane). However, this had little appreciable effect on the light & dark ripples.

It's a interference pattern, called Moire pattern. You get them in signal analysis and various other fields. I guess it could be considered an optical illusion.

EDIT:

To elaborate, this occurs when a periodic waveform goes out of phase with the observer. In your case the periodic waveform are the lines when observed as a scanline (from left to right of the image or vice versa), and the observer is the image with a set number of horizontal pixels. In some parts, the pixel rasterization falls on pixel boundaries and then there is a watershed moment... should it be rasterized in pixel x, or x + 1. This then manifests as the wave pattern (due to the 'watershed' moment being slightly out of phase with the previous or next scanline i.e one scanline, pixel falls on x, the next scanline it falls on x+1). Also the display device can also have an effect. The moire in your image is much more pronounced on my computer monitor, than it is with my phone. (however this could be due to my phone’s browser scaling or applying some other filtering by default, which could help to reduce it).

Anti aliasing certainly helps, however moire can never be completely eradicated, only mitigated. Also note this can occur with rotated grids in orthographic, however perspective projection will certainly amplify this effect.


Please, help me to understand why my raytracing camera rotates in wrong directions? I'm trying to build a raytracer and I use this article on how to build camera system. The problem is that when, after calculating ray direction in camera space, I multiply it by camera-to-world transformation matrix and my camera seems to rotate in opposite directions and works correctly if I inverse transformation matrix before multiplication.
Here is the code (I use glm library and right-handed coordinate system).

Some data

glm::vec3 origin_ = glm::vec3(0.f);// camera origin
cont glm::vec3 kDirection = glm::vec3(0.f, 0.f, -1.f);
cont glm::vec3 kUp = glm::vec3(0.f, 1.f, 0.f);
float aspect_ratio_ = (float)raster_height_ / raster_width_;

// bug !!! rotates in opposite direction  (camera is actually tilted down)
glm::mat4 camera_to_world_ = glm::lookAtRH(origin_, glm::vec3(0.f, 0.2f, -1.f), kUp);
// works !!! (camera is tilted up)
glm::mat4 camera_to_world_ = glm::inverse(glm::lookAtRH(origin_, glm::vec3(0.f, 0.2f, -1.f), kUp));

And function that generates camera rays

// Calculate ray as if camera is located at 0,0,0 and pointing into negative z direction
// Then transform ray direction to desired plase
// x,y - pixel coordinates of raster image
// calculate as if raster image (screen) is 1.0 unit away from origin (eye)
Ray Camera::GenRay(const uint32_t x, const uint32_t y) {
  glm::vec3 ray_direction = kDirection;
  // from raster space to NDC space
  glm::vec2 pixel_ndc((x + 0.5f) / raster_width_, (y + 0.5f) / raster_height_);
  // from NDC space to camera space
  float scale = tan(fov_ / 2.0f);
  ray_direction.x = (2.0f * pixel_ndc.x - 1.0f) * scale; // *aspect_ratio_;
  ray_direction.y = (1.0f - 2.0f * pixel_ndc.y) * scale * aspect_ratio_;
  // apply camera-to-world rotation matrix to direction
  ray_direction = camera_to_world_ * glm::vec4(ray_direction, 0.0f);

  return  Ray(origin_, ray_direction, Ray::Type::kPrimary);
}

I really can't understand the root of a problem so any help us appreciated.

The problem is that instead of a camera-to-world matrix, a world-to-camera matrix is being made by glm::lookAtRH.

This is because GLM is a math library made for OpenGL. In OpenGL, you do not move the 'camera' as there is no camera in OpenGL. You move the whole world the other way around, so that it lines up in the frustum. The lookAtRH method returns a matrix that moves the whole world in a way that it seems like you have the camera there.

Inversing the matrix is a fine way to get the proper camera-to-world matrix.

Scrathapixel does have an article regarding making your own lookAt function. One thing to note about this article is that they use a row-major notation. It might be that you need to switch it up and use a column-major notation (just swap the x index with the y index).

Placing a Camera: the LookAt Function | Scratchapixel


Distributed parallel rendering in Gaming or VR rendering VR rendering needs a lot of GPU power. VR SLI can help that. But, is it possible for us to use distributed parallel rendering technology to improve the performance dramatically? There is an open source software "equalizer" which could do that.

For example, put 4 or 8 GPUs in one PC server and disable vender's SLI driver. Use equalizer to do parallel rendering. Or, run equalizer on 16 PC servers with GPU equipped.

So that we can get very high performance easily? Right?

Equalizer supports sort-first and sort-last. Since sort-last needs a lot of network bandwidth, we'd better use sort-first.

You are essentially correct in assuming that it is technically feasible to distribute the rendering computation workload. It is, essentially, a computation workload like any other; possibly even better-suited for parallelization due to its very nature: many similarly structured units running the same code path (i.e. vertices, pixels).

What you are forgetting, however, is latency. Having all the render power in the world is useless if the image does not arrive to the HMD fast enough not to cause discomfort to the user. Here is an article that explains the importance of motion-to-photon latency:

http://www.chioka.in/what-is-motion-to-photon-latency/


How to rotate an object around any axis if i have just local x and y? I have just two local axis to rotate a 3D object: X (horizontal) and Y (vertical). How to make from this 2 local axises for example a global Y rotation or a Local Z rotation? What math magic do the trick?

If you know how to make a rotation matrix from X,Y,Z basis vectors (just make them the rows or columns, depending on whether you are using a row or column major matrix), you can do that, using the cross product of X and Y to get the Z vector.

The order of X and Y in the cross product depend on whether you are using a left or right handed coordinate system, but if you are unsure about this, you can check if doing Z=cross(X,Y) looks right and if not, switch to Z=cross(Y,X).

So, it'll be something like this:

// May be Y,X instead, depending on your handedness
Z = Cross(X,Y);

// May need to be transposed based on you using row or column major matrices
RotationMatrix = MatrixFromRows(X,Y,Z);


Efficiently rendering sprites In order to naïvely render sprites, you can send single rectangles with their position, rotation, size, and texture. This is very easy, but given that reducing state changes, draw calls, and data uploads are good ways to improve rendering performance, it's clearly not an efficient solution.
Some techniques like passing each sprite as a single primitive and assembling a rectangle in the geometry shader are very appealing but aren't available if targeting a platform like WebGL or even just older hardware.

On top of issues like batching and texture aliases, how do you efficiently submit geometry to render sprites?

I'm interested in answers that are rendering API agnostic, but using a specific API like OpenGL or Direct X is fine for illustration purposes.

A good way to improve sprite rendering efficiency is geometry instancing. This lets you define a mesh using one vertex/index buffer pair, then render many instances of that mesh in a single draw call using a second vertex buffer whose elements define the instances. This is more widely supported than geometry shaders; in particular, it's is supported in WebGL via the ANGLE_instanced_arrays extension.

To apply this to sprites, one vertex buffer could define the shape of a sprite (i.e. a quad made of two triangles), then another vertex buffer could contain per-sprite information such as position, rotation, size, and so on.

If there are 4 vertices per sprite and N sprites, the vertex shader will be called a total of 4N times, onces for each vertex in each sprite. You'd use the vertex shader to put the information from the two buffers together, transforming each vertex appropriately to the sprite's properties. This allows the GPU to do the work of generating all the vertices for the sprites, and saves on data uploaded from the CPU.

Coincidentally, I published a small OpenGL instancing demo recently.

Using instancing this way does require that all the sprites are drawn with the same state, i.e. the same shader, textures, blending mode and so on. If different states are needed for some sprites, a separate batch of instances will be needed for each state combination.


Why cubic curves provide the minimum curvature interpolants? As described by Shirley in his computer graphics book,

  Cubic curves provide the minimum-curvature interpolants to a set of points. That is, if you have a set of n + 3 points and define the “smoothest” curve that passes through them (that is the curve that has the minimum curvature over its length), this curve can be represented as a piecewise cubic with n segments.

Could you please give some hints or references about its proof?

Splines confuse me which is one reason I asked somebody else to write that chapter.

But I like this explanation:
https://www.johndcook.com/blog/2009/02/06/the-smoothest-curve-through-a-set-of-points/


Combine material coefficients and textures in Phong Shading I'm trying to implement a simple Phong shader that supports non-physically-based materials and textures. Even though the material has a texture for each light component I still want the respective material coefficient to have some effect. My doubt is how to handle both of them. Should I mix, multiply or sum them?
Right now I've multiplied them:

ambient  = material.ambient_color  * light.ambient  * texture_ambient;
diffuse  = material.diffuse_color  * light.diffuse  * diffuse_strength  * texture_diffuse;
specular = material.specular_color * light.specular * specular_strength * texture_specular;

It seems kinda dark, is this the correct way to combine material coefficients and textures?

When applying various types of attenuation you are just multiplying by some number smaller than 1.  This is akin to every atom of your material having some probability of absorbing the light that hits it.

Something to note is that these calculations should be performed ( in the context of gamma correction and colour spaces ) in a linear space.

If you don't perform the calculations in linear space you might end up with overly bright, overly dark, overly saturated or under saturated colours.


Does film filter introduce bias in path tracing? Does using box or gaussian filter introduce bias to the image when reconstructing the pixel?
Bias does not seem to be talked in the Chapter 7.8 of PBRT

This is an answer to your comment rather than the title of your question. It can be considered as a partial answer.
The bias in path tracers are introduced by the estimators.
For example, think of the following two equations:
$$ I = \int_{\Omega} f(\omega) d\omega $$
$$ F = \frac{1}{N} \sum_{i}^{N} \frac{f(\omega)}{p(\omega)}$$ where $p: \Omega \to R$.
It should be evident that as the number of samples N increases, the function $F$, ie the estimator, outputs values that are closer to $I$.
The unbiased estimator satisfies the following $E[F_N] \approx I$ or $\lim_{N\to\infty}F_N=I$, that is the expected value of the estimator matches the desired outcome in $I$ as the number of samples goes to infinity.
The biased estimator does not offer such a guarantee.
You can read more from D. Van Antwerpen's master thesis, I am giving the full citation in case the link goes away:
Van Antwerpen, Dietger. “Unbiased Physically Based Rendering on the GPU.” Master of Science in Computer Science, Delft University of Technology, 2010.


How to generate chaotic halftone pattern? I'm trying to generate a random halftone like below image:

this is a simple halftone that I tried, but I need a random halftone

float PI = 3.14159265359;

    void mainImage(out vec4 fragColor, in vec2 fragCoord) {
        vec2 p = (fragCoord.xy / iResolution.xy - 0.5)/0.5;

            vec3 n = vec3(p, sqrt(dot(p, p)));
            vec2 s = vec2(acos(n.z), atan(n.y, n.x))/PI;
            vec2 uv = fract(s * vec2(30.0, 30.0));
            float r = ceil(s.x * 30.0) / 17.0;
            fragColor =  vec4(1.)*step(length(uv - 0.5),r);
    }

https://www.shadertoy.com/view/MljBRw

I also tried using for-loop and sin,cos, but I need a low computing method because I want generate it in the game engine.

I could make it by Worley noise

  Worley noise is a noise function introduced by Steven Worley in 1996. In computer graphics it is used to create procedural textures,that is textures that are created automatically in arbitrary precision and don't have to be drawn by hand. Worley noise comes close to simulating textures of stone, water, or cell noise.

https://thebookofshaders.com/12/

#ifdef GL_ES
precision mediump float;
#endif

uniform vec2 u_resolution;
uniform float u_time;

// Permutation polynomial: (34x^2 + x) mod 289
vec4 permute(vec4 x) {
  return mod((34.0 * x + 1.0) * x, 289.0);
}
vec3 permute(vec3 x) {
  return mod((34.0 * x + 1.0) * x, 289.0);
}

// Cellular noise, returning F1 and F2 in a vec2.
// Speeded up by using 2x2x2 search window instead of 3x3x3,
// at the expense of some pattern artifacts.
// F2 is often wrong and has sharp discontinuities.
// If you need a good F2, use the slower 3x3x3 version.
vec2 cellular2x2x2(vec3 P) {
    #define K 0.142857142857 // 1/7
    #define Ko 0.428571428571 // 1/2-K/2
    #define K2 0.020408163265306 // 1/(7*7)
    #define Kz 0.166666666667 // 1/6
    #define Kzo 0.416666666667 // 1/2-1/6*2
    #define jitter 0.8 // smaller jitter gives less errors in F2
    vec3 Pi = mod(floor(P), 289.0);
    vec3 Pf = fract(P);
    vec4 Pfx = Pf.x + vec4(0.0, -1.0, 0.0, -1.0);
    vec4 Pfy = Pf.y + vec4(0.0, 0.0, -1.0, -1.0);
    vec4 p = permute(Pi.x + vec4(0.0, 1.0, 0.0, 1.0));
    p = permute(p + Pi.y + vec4(0.0, 0.0, 1.0, 1.0));
    vec4 p1 = permute(p + Pi.z); // z+0
    vec4 p2 = permute(p + Pi.z + vec4(1.0)); // z+1
    vec4 ox1 = fract(p1*K) - Ko;
    vec4 oy1 = mod(floor(p1*K), 7.0)*K - Ko;
    vec4 oz1 = floor(p1*K2)*Kz - Kzo; // p1 < 289 guaranteed
    vec4 ox2 = fract(p2*K) - Ko;
    vec4 oy2 = mod(floor(p2*K), 7.0)*K - Ko;
    vec4 oz2 = floor(p2*K2)*Kz - Kzo;
    vec4 dx1 = Pfx + jitter*ox1;
    vec4 dy1 = Pfy + jitter*oy1;
    vec4 dz1 = Pf.z + jitter*oz1;
    vec4 dx2 = Pfx + jitter*ox2;
    vec4 dy2 = Pfy + jitter*oy2;
    vec4 dz2 = Pf.z - 1.0 + jitter*oz2;
    vec4 d1 = dx1 * dx1 + dy1 * dy1 + dz1 * dz1; // z+0
    vec4 d2 = dx2 * dx2 + dy2 * dy2 + dz2 * dz2; // z+1

    // Sort out the two smallest distances (F1, F2)
#if 0
    // Cheat and sort out only F1
    d1 = min(d1, d2);
    d1.xy = min(d1.xy, d1.wz);
    d1.x = min(d1.x, d1.y);
    return sqrt(d1.xx);
#else
    // Do it right and sort out both F1 and F2
    vec4 d = min(d1,d2); // F1 is now in d
    d2 = max(d1,d2); // Make sure we keep all candidates for F2
    d.xy = (d.x < d.y) ? d.xy : d.yx; // Swap smallest to d.x
    d.xz = (d.x < d.z) ? d.xz : d.zx;
    d.xw = (d.x < d.w) ? d.xw : d.wx; // F1 is now in d.x
    d.yzw = min(d.yzw, d2.yzw); // F2 now not in d2.yzw
    d.y = min(d.y, d.z); // nor in d.z
    d.y = min(d.y, d.w); // nor in d.w
    d.y = min(d.y, d2.x); // F2 is now in d.y
    return sqrt(d.xy); // F1 and F2
#endif
}

void mainImage( out vec4 fragColor, in vec2 fragCoord ){

    vec2 uv = (fragCoord.xy - iResolution.xy*.5)/iResolution.y;

    float w = 1. - dot(uv, uv)*3.;

    uv *= 48.;
    vec2 ip = floor(uv);
    vec2 v = cellular2x2x2(vec3(uv, iTime/2.));

    float c = v.x; //v.y - v.x

    c -= .35*w;

    c = smoothstep(0., max(.1/c, 0.), c);

    vec3 col = mix(vec3(0), vec3(1), c);

    fragColor = vec4(sqrt(max(col, 0.)), 1);
}

you can see this In shadertoy https://www.shadertoy.com/view/XlSBDz


Loading a material via assimp I'm writing a raytracer and I'm using the assimp library to load objects. I have an object's material specified in a .mtl file:

newmtl rightSphere
Ka 0.01 0.01 0.01
Kd 0.01 0.01 0.01
Ks 0.30 0.30 0.30
Tf 0.10 0.10 0.10
Ns 200
Ni 2.5
illum 7

and I'm trying to load this material. I've been mostly successful so far, for example I know that to extract the diffuse component (Kd), I can use

aiMesh *mesh = scene->mMeshes[i];
aiMaterial *mat = scene->mMaterials[mesh->mMaterialIndex];
aiColor3D color;
mat->Get(AI_MATKEY_COLOR_DIFFUSE, color);

using the identifiers specified in here. However, I've been unable to extract the Tf component. I understand that this component defines the coefficient of refracted light per color channel, and I need this value along with the index of refraction (Ni) to get it to work.

My question is: how can I get the Tf value?

AI_MATKEY_COLOR_TRANSPARENT is used to define the transparent colour and AI_MATKEY_REFRACTI is the refractive index.


Algorithm for proper drawing of cosine function I am working with the cosine function with a very large input, like cos(a* t), where t varies between  1 and ~ 200; and a can be as large as 10^8-
I come across, as expected, the artifact error. If I sample t with a stepsize of 0.01 with a 100, I get a plot like this :

Notice, that for large value of a, the sampling of t needs to be fine, otherwise, you will not get regular cosine shape from -1 to 1. Sometimes, wave peaks, and other times wave troughs will be missed, due to low granularity, and you will end up with a different shape of the wave than the theoritical value.
Now, increase a to be 100000, and as expected, the effect is much more pronounced.

My question : Is there a formula / algorithm, to avoid this effect? Is there a formula, that tells you the correct step size for a large value of a ?
I tried to google the following keywords : large input cosine function computer graphics, but that did not help.
Thank you.

There is in fact a way to avoid this effect and it is outlined by the NYQUIST sampling theorem - which states that the sampling rate should be at least twice as high as the maximum frequency of the function of interest.
In this case where cos(a * t) is the function of interest, the frequency is a/(2*pi) - therefore your sampling rate should be greater than 2 * a/(2*pi) (in this case you may even want the sampling rate to be even higher so that the cosine function has a nice curve to it).
To get the step size from this sampling rate just use t = 1/(sampling rate)
Hope this helps :)


using default depth buffer in off-screen framebuffer I'm trying to implement Weighted, Blended Order-Independent Transparency

There are three passes:

3D opaque surfaces to a primary framebuffer
3D transparency accumulation to an off-screen framebuffer
2D compositing transparency over the primary framebuffer

I need to test transparent objects against default depth buffer. I created a framebuff for transparency pass and also created a depth buffer for maybe I can copy default depth to this buffer if this is the only way. But I don't know how to do that because I created depth buffer with GL_DEPTH_COMPONENT24 but I don't know default depth format.

Is it possible to bind default framebuffer's depth buffer to another framebuffer? If not, how can I copy default depth to another depth buff, or what is best way to do this?

Is it possible to bind default framebuffer's depth buffer to another framebuffer?

First, you "attach" images to framebuffers. You "bind" objects to the context; you "attach" objects to other objects.

Second, no, you cannot attach images of the default framebuffer to anything.

  If not, how can I copy default depth to another depth buff, or what is best way to do this?

You render to your own depth buffer entirely. Just copy/render the image data out to the default framebuffer when its time to display it.


Sampling the light vs. sampling the BRDF: difference in intensity (possible duplicate)

I'm trying to implement multiple importance sampling, as described in the PBRT book (no bouncing yet, I want to make MIS work first).

What my problem is, that while the BRDF sampling looks okay, the light sampling is much dimmer (yet according to the pictures in the book they should look the same in intensity).

I uploaded the (updated) compilable code (VS 2015). The relevant parts can be found in multipleimportance.frag (SampleLightsExplicit) and pbr_common.head (ConeSampleSphere).

(note that I don't apply the MIS weights yet)

for (int k = 0; k < numLights; ++k) {
    SceneObject light = objects.data[k];

    center = vec3(light.toworld[3].xyz);
    radius = light.toworld[0][0];

    // importance sample light (xyz = direction, w = PDF)
    sample1 = ConeSampleSphere(p, center, radius, pixel, seed + float(k));

    // cast shadow ray
    vis = Visibility(p, sample1.xyz, k);
    ndotl = vis * max(0.0, dot(sample1.xyz, n));

    // evaluate BRDF
    fd = vec3(0.0); //obj.color.rgb * ONE_OVER_PI;
    fs = CookTorrance_General(sample1.xyz, v, n, vec3(0.04), roughness);

    sum += (light.color.rgb * (fd + fs) * ndotl) / sample1.w;
}

My question is why light sampling looks so dim? Did I misunderstand something?

UPDATE

Looking through the code I noticed some inconsistencies which probably originated from a typo in one of the older SIGGRAPH course notes (can't tell which one tho).

The PDF of the GGX distribution is:

PDF = D(h) * dot(n, h)

While some implementations (including UE4) incorrectly write it as

PDF = (D(h) * dot(n, h)) / (4 * dot(v, h))

which essentially means that they include the Cook-Torrance denominator into this variable (ndotl cancels out). Dividing with the correct PDF cancels out D in the Cook-Torrance model, which I forgot to account for, while the 1 / (4 * dot(v, h)) term goes into the Vis_Schlick function.

So regarding intensity the second picture is the "correct" one. The dimness is still a problem tho. Changing the Fresnel F0 to 0.11 (as it supposedly is in the book) and applying the MIS weights, the picture looks a lot more promising, except for the topmost plate which should almost explicitly reflect the light sources:

So this is almost okay now. I'm gonna look deeper into the MIS part to see why it makes those highlights disappear.

Shoot me... After the fixes I posted above, I started debugging Mitsuba to see whether it does anything differently on a large scale. Of course it didn't; my code is completely correct, the dimness is due to this macro def:

#define EPSILON   1e-5

This is used in divisions to avoid dividing with zero. Unfortunately, it accumulates to a large value during the calculations. Changing it to 1e-9 fixes everything:

For future generations I leave the source code up (just remember to change this value).

UPDATE:

I found another mistake in my calculations, but it isn't apparent in this scene. Microsurface-based BRDFs are usually sampled according to the microfacet normal distribution function (D(h, a), in this case GGX).

Don't forget however, that you also have a shadowing-masking function (G(l, v, a)) which has to be evaluated according to the chosen microfacet distribution. This means that I can't just mix-and-match f.e. GGX sampling and the Smith-Schlick evaluation; I have to use the Smith-GGX solution:

float G_Smith_GGX(float ndotl, float ndotv, float roughness)
{
    float a = roughness * roughness;
    float a2 = a * a;

    float lambda_l = sqrt(a2 + (1.0 - a2) * ndotl * ndotl);
    float lambda_v = sqrt(a2 + (1.0 - a2) * ndotv * ndotv);

    return (2.0 * ndotl * ndotv) / (ndotv * lambda_l + ndotl * lambda_v);
}

Which explains the UE4 approach, as:

vec3 F = F_Schlick(F0, ldoth);
float G = G_Smith_GGX(ndotl, ndotv, roughness);

// PDF = (D(h) * dot(n, h)) / (4 * dot(v, h))
return (F * G * vdoth) / (ndotv * ndoth + EPSILON);

See this article for the derivation.


3D object to slices like in medical scan I am novice lacking professional terminology.

I want to create from 3D object (let's say from polygon mesh) a slice through to PNG file in order to create something like reverse medical MRI scan from which volumetric renders are created.

How it's called? It will enable me to find methods of achieving it. Bonus points for pointing to a library doing that.

Some terminology: Intersection of a polyhedron with a plane.

Wolfram Demo.


How can i make my ray tracing algorithm fast in terms of intersection testing? I am working on a ray tracer. But i want to put more than 10 objects in the scene. How can i make the intersection fast so that it can give me reasonable FPS.
Its a cpu-based ray tracer.

You have several techniques.

Use acceleration structures. The idea is that you store your geometry (triangles) into generally some kind of volumes. The general idea behind acceleration structure is to discard quickly large parts of the scene, which we know (using these structure) don't have any object that your ray will intersect. An introduction to the topic can be found here.
You can also take advantage of multi-threading and parallelism (threads and SSE instructions or similar - this is harder than 1. and the benefits are not always as high but you optimise for your hardware).
You can ray-trace in the GPU but ray-tracing is a memory-bound problem so you will have memory problems quicker with the GPU than with CPU.

These are just some ideas.


Transformations and change of basis in CG Sometimes in CG literature a transformation is equated to a coordinate system change. This is fine as long as the transformation includes only a rotation and a translation. When scaling is involved, however, this seems like an incorrect interpretation. The problem is specifically with how lengths and angles are measured after the transformation. Angles and length are invariant to coordinate system change: $\langle a, b \rangle = \sum_{i,j}g_{ij}a^ib^j$. In graphics, this is often substituted with the expression valid only for orthonormal bases: $\langle a, b \rangle = \sum_{i}a_ib^i$, regardless of the supposed basis that $a,b$ are in (the index of $a$ is lowered as if the basis was orthonormal without involving the metric tensor). So while this makes sense as a transformation, I do not see how this is reconciled with the idea of a change of basis. Am I missing something, or is the interpretation wrong? (think of DX/GL's standard transformations between spaces in the rendering pipeline)

You're correct, we don't usually worry about pulling the inner product through a change of basis in graphics. As a practical matter, calculations that depend on angles and distances, such as lighting and shading, are always done in world space or in a space orthonormally related to it (e.g. camera space).
The place where non-orthonormal transformations show up is usually in local-to-world transforms of meshes—most frequently with skeletal animation, where things get deformed, or where bones are set up with non-orthonormal coordinates for rigging/animation reasons. However in those instances we are always bringing the vertices, normals, etc into world space or camera space to process. We neither push back the world inner product into the local spaces, nor do we want to pull a locally defined inner product into world space, as neither of those is really useful for the typical kinds of things we need to do.
(And for the normals, we do know how to use the inverse transpose matrix to transform those properly, thank goodness!)
The other big non-orthonormal transformation in graphics is the projection matrix, but again, we don't really ever want to use the inner product in post-projective space, so the issue doesn't really arise. We just keep all our inner product-ing in world space.


How can I offset/shrink a triangular polygon in GLSL? I need to offset all (blue) triangles, each independently of the others, using the vertex-shader. In order to manipulate the triangle as a whole, I've created custom (vec3) attributes for each vertex (red) representing the leftward (purple) and rightward (green) neighboring vertices. From this, I need to derive the orange point, equidistant (in screen space) from both adjoining edges. With three such orange points derived from each triangle, the processed (orange) triangle is passed on to the fragment shader.

Ideally, the triangle will be culled (as in backfacing/ not rendered) if the offsets negate any available space within the triangle, such as in the second triangle in the second picture.

I'm using THREE.BufferGeometry() as my data structure.

Here is a screen shot of the effect I'm aiming for:

Given triangle ▲ABC, we bisect angle ∠BAC with line AD, derived with Angle Bisector Theorem:

BA / BD = CA / CD

Point E represents our objective refined position on the desired resulting inset triangle. As it lies upon angle bisector AD, it is equidistant from sides BA & CA, forming identical right triangles ▲AFE & ▲AGE. We can now use Sine for Right Triangles to find the length of AE:

AE = EG / Sin(∠EAG)

That's all the math we need, so let's cook up some GLSL!

We start off with all the typical attributes: position, normal, and transformation matrices, but since the vertex shader only works on a single vertex, we need to add the neighboring vertices as additional attributes. This way, each vertex will find its own "point E", creating the resulting inset triangle. (Note: I don't call them "B" & "C" here, because they are not yet in screen space.)

    attribute vec3 left; //vertex to the left of this vertex
    attribute vec3 right; //vertex to the right of this vertex

Speaking of screen space, I'm also including the aspect ratio of the display, (and making it a uniform, in case the window is resized.)

After preparing varying normals for the fragment shader, and transforming the face into clipping-space, we can get down to the business of applying the above math:

        attribute vec3 left; //vertex to the left of this vertex
        attribute vec3 right; //vertex to the right of this vertex
        uniform float aspect;
        varying vec3 vNormal;
        varying vec2 vUv;

        void main() {
            vNormal = normal;
            vUv = uv;

            mat4 xform= projectionMatrix * modelViewMatrix;
            vec4 A = xform * vec4( position, 1.0 );
            vec4 B = xform * vec4( left, 1.0 );
            vec4 C = xform * vec4( right, 1.0 );

            vec3 CB = C.xyz - B.xyz;
            vec2 BA = B.xy - A.xy;
            vec2 CA = C.xy - A.xy;
            float lengthBA = length(BA);
            float lengthCA = length(CA);
            float ratio = lengthBA / ( lengthBA + lengthCA );
            vec3 D = B.xyz + ratio * CB.xyz;
            vec3 AD = D - A.xyz;
            vec3 bisect = normalize(AD);

            float theta = acos( dot(BA, CA) / (lengthBA * lengthCA) ) / 2.0;
            float AE = 1.0/(sin(theta)*aspect);
            newPos.z += AE/length(AD) * (D.z - A.z);
            newPos.x += bisect.x*AE;
            newPos.y += bisect.y*AE;

            gl_Position = newPos;
        }

This code gives us the results below.

Note, there are a few edge-cases having to do with almost backface-culled triangles being flipped by this process, and I began to address this in code, yet decided to simply avoid these cases for now. Perhaps I'll revisit it when I get this project done.


OpenGL blending with the screen I don't understand how OpenGL performs the blending of a texture on the screen. By default, the clear color is set to (0, 0, 0, 0). If we use the default blending equation GL_SRC_ALPHA and GL_ONE_MINUS_SRC_ALPHA, it means that if our texture is all (1, 0, 0, 0.3), the resulting RGB will be (1, 0, 0)×0.3 + (0, 0, 0)×0.7 = (0.3, 0, 0) BUT the alpha component will be 0.3×0.3 + 0×(1 - 0.3) = 0.3, so we then have a color of (0.3, 0, 0, 0.09) which is not correct, what we want to see on the screen is rather (0.3, 0, 0, 1).
So how does OpenGL manages the final blending with the framebuffer? Does it drop the alpha component (because the screen can only display RGB colors)?

BUT the alpha component will be 0.3×0.3 + 0×(1 - 0.3) = 0.3, so we then have a color of (0.3, 0, 0, 0.09) which is not correct, what we want to see on the screen is rather (0.3, 0, 0, 1).

Your blending function did not use the destination alpha. So "correct" or "not correct", its value is entirely irrelevant.
However, if you want to make the alpha more accurate in some way, you can provide a different set of blending factors, and even a different blend equation, for the alpha component.


OpenGL ES Frag Shader - Branching Working on optimizing a fragment shader which is slowing down my OpenGL ES 2.0 shader based game. This is used with a shadow buffer to determine if the fragment/pixel is inside a shadow volume.

The problem/slowdown is the fact I need a branch if or function to determine a 1.0 (no shadow) or 0.0 (shadowed) based on comparison of the frag's shadow Z and shadow buffer Z.

    // SLOWEST
    if ( shadowValue > shadowTexZ ) return 1.0; else return 0.0;

    // ALSO SLOW
    return clamp( 1.0 + (shadowValue - shadowTexZ) * 1000.0, 0.0, 1.0 );

    // SLIGHT LESS SLOW
    return max(sign(shadowValue - shadowTexZ), 0.0);

    // ALSO SLOW
    return sign( shadowValue - shadowTexZ ) * 0.5 + 0.5;

If I disable the shadow test all together, GPU time in Xcode is 12ms and solid 60fps.  With the test on it jumps to 26ms and drops the framerate unacceptably to 40fps.

Thanks!

I think the most efficient way might be either

return float(shadowValue > shadowTexZ);

or

return step(shadowTexZ, shadowValue);


OpenGL Compute Shader generating triangle indices: How to get correct element count for glDrawElementsIndirect? I have a sequence of compute shaders that generates an indexed mesh.The last one of those writes the generated indices like this:

void addTriangle (uint i0, uint i1, uint i2) {
    uint ic = atomicCounterIncrement(indirectIndexCount);
    meshIndices[ic*3+0] = i0;
    meshIndices[ic*3+1] = i1;
    meshIndices[ic*3+2] = i2;
}

After the mesh has been generated, it is drawn with glDrawElementsIndirect. The indirectIndexCount in the code above is an atomic_uint counter at position 0 inside the GL_DRAW_INDIRECT_BUFFER (see the struct called DrawElementsIndirectCommand). This counter is now obviously too small by a factor of three, since it was incremented only once for each triangle. Currently, I multiply it by 3 afterwards just before issuing the draw call.

(At the moment this is done by mapping the buffer and multiplying on the CPU, which is of course nonsense, but shows that the whole thing basically works. Everything is drawn correctly. I could do it with an invocation of a single 1x1x1x1x1x1 compute shader, but that seems only slightly less silly.)

How do I get rid of this extra multiplication step?

Since this seems to be an obvious problem whenever meshes with variable index count are generated, I guess there must be some easy solution that I'm overlooking?

If you have access to glsl 4.3+ (or glsl ES 3.1) you can use atomicAdd

The next option is to use a barrier() after all vertices are generated in the compute and then multiplying the value in the counter:

main(){

    // generate vertices

    barrier();
    if(gl_localInvocationID == vec3(0)){
        indirectCount = atomicCounter(indirectIndexCount)*3;
    }
}

This emulates running another 1x1x1 compute shader to multiply the index.

Otherwise you can use a second atomic to hold the vertexcount:

void addTriangle (uint i0, uint i1, uint i2) {
    uint ic = atomicCounterIncrement(indirectIndex);
    meshIndices[ic*3+0] = i0;
    meshIndices[ic*3+1] = i1;
    meshIndices[ic*3+2] = i2;
    atomicCounterIncrement(indirectIndexCount);
    atomicCounterIncrement(indirectIndexCount);
    atomicCounterIncrement(indirectIndexCount);
}

Both indirectIndex and indirectIndexCount are initialized to 0 and indirectIndexCount is what is passed through to  glDrawElementsIndirect.


Texture prefiltering for Linearly Transformed Cosines I'm implementing Real-Time Polygonal-Light Shading with Linearly Transformed Cosines from Siggraph 2016.

I'm stuck on the texture prefiltering (section 5.3).

As I understand they use Gaussian Blur on the texture, but clamp and normalize kernel to fragments that intersect the original texture.

But how do they create the margin? I have tried proper Gaussian with 2D kernel and 2 pass Gaussian but neither seems similar to what is in the paper.

Also how should weight of Gaussian be chosen?

Or maybe I'm totally wrong and it is something more than simple Gaussian?

I have ported their WebGL demo in DX shader model 4, but I have not implemented such texture preprocessing. Instead, I use some abstract pattern (like a comuflage) that is twice wider and higher. So, in a reflected area only internal part is rendered if roughness is low, and with a greater roughness, mip-map with lower resolution is taken. That spreads pattern in zone of reflection realistically.
It could be some starting point for your implementation.

It is possible to make some "texture expansion" like in article, but some special code to do this is needed. This is "special", it modifies Gaussian.

But in many cases, reflected pattern could be very simple, and such expansion can be eailly created by artist and mip chain generated. Again -good for start point.


Barycentric coordinates for 5-sided polytope of two triangles and three trapezoids I have a triangle that is extended in space and each edge is refracted differently, creating a 5 sided polytope. The initial and final triangles are the view radii segments of a frustum. We need to get the u and v along the triangle and the w depth in between the triangles of a point inside the polytope.

I think I can do this using trilinear interpolation and treating two vertices on each side as merged and using my own volume calculations. I just need to interpolate some values given by the three points and a z. http://bmia.bmt.tue.nl/People/BRomeny/Courses/8C080/Interpolation.pdf


memory consumption for a true color frame buffer What is the memory consumption for a true color frame buffer (with double buffering) on an XGA display? ("XGA" = 1024 x 768 pixels) How can I calculate this?

True color is 8 bits per color channel, so 3 bytes per pixel.  1024x768 is 785,408 pixels.  At 3 bytes per pixel that is 2,356,224 bytes.  Since it's double buffered we double that number to 4,712,448 bytes which is about 4.5MB.


Inverse matrix order of operation If I have a rotation, $A$, and a translation, $B$, which I multiply like so

$C = AB$

Does the inverse of $C$ not only inverse the magnitudes of $A$ and $B$ but also become the reverse of operations?

e.g. $C^{-1} = BA$?

Does the inverse of a matrix also inverse the order of original operations (or am I conflating inverse with reverse)?

Yes. If you're compounding operations to make a matrix, then the inverse matrix will be the compound of the inverse operations, in the reverse order. So if $C = AB$ then $C^{-1} = B^{-1}A^{-1}$

Think of it geometrically. Taking a 2D example, if you have an object at the origin, and you want to move it +2 units in X then rotate around the origin by +45 degrees. To undo this transformation, you need to first rotate by -45 degrees then translate by -2 units in X.


Are these edges a result of display color accuracy, computational accuracy, or something else? I was messing with glowing lighting when I noticed an odd artifact in my first try.

For some reason I can see three defined edges inside of the glowing pattern. I downloaded the image and marked where I see the edges.

I am just curious why they appear to be there as I can see no mathematical reason why they should be there. Is it just the way I am displaying it, or is there some sort of computational precision issue?

For reference here is the ShaderToy code that created this:

void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
    vec2 uv = fragCoord.xy / iResolution.xy;
    float disty = abs(uv.y - 0.5);
    float distx = abs(uv.x - 0.5);
    distx *= 100.0;
    disty *= 100.0;
    fragColor = vec4(0.0, 1.0 / pow(abs((1.0 - distx) + (1.0 - disty)), 0.5), 0.0, 1.0);
}

Although I am not sure it is physically accurate to have those spikes on the edges of the diamond I sure do like the effect that creates.

Updated answer, now that pow is getting valid arguments:

What you are seeing is called "banding" and it comes from the fact that the color channels are quantized into 8 bits - in other words, it comes from the fact that there are only 256 different shades of green, even though the math generating the colors is capable of much larger resolution.

The banding on my system isn't as bad as what you are describing but it's still present.  The reason it isn't as bad on my system is because you and I have different displays.

Some computer displays don't actually even have a full 256 different shades of green.  For instance, instead of having 8 bit color, they may only have 6 bit color, and they either dither it, temporally dither it, or neither and just let it look worse.

One way to get around this problem is by dithering.  There are various algorithms for dithering, with different computation and quality trade offs.

A good modern dithering algorithm is called "interleaved gradient noise" which is fairly close to blue noise - which is the ideal - and is pretty cheap to calculate.

Check out this link for more detailed information on different types of dithering, and how they compare:

https://bartwronski.com/2016/10/30/dithering-part-three-real-world-2d-quantization-dithering/

First answer, from when pow was getting a negative value parameter:

The issue is that you are passing a negative value to pow for the first argument, which has undefined behavior.

What shows up on my screen is completely different than what shows up on yours.

You shouldn't rely on the behavior of anything in the red pixel areas in this image:

More info:

https://www.opengl.org/sdk/docs/man4/html/pow.xhtml


16bit half-float linear HDR images as (diffuse/albedo) textures? So I been thinking about this for a while and tried to google for an answer but without any success.

If all your textures are 8bit LDR images, like JPEGs, couldn't that potentially cause conflicts with exposure control/tone mapping when rendering. That is if you adjust the rendering exposure of your image that should expose detail in the textures that aren't really there, since they been clamped out by the low dynamic range. So wouldn't it make sense to also have the textures as HDR images, saved as .exr, in linear colour space with 16bit half-float to get a good colour representation (32bit "full" float might be overkill?). To have more detailed and correct colour values might also, I figure, have an effect on GI and how colour bleed is calculated?

Or is it simply not necessary since the end result of the rendering we want is probably going to be similar to the exposure level of the texture when it was photographed any way? And since camera mostly shoot in 12-14bit you would have to take multiple exposures of the texture and do all that extra work to piece them all together to one HDRI.

Edit: To clarify, I'm mostly interested in this from a photo realistic rendering point of view, with ray trace renderers (like mental ray, V-Ray, Arnold, etc.) with full light simulations and global illumination, rather then for real-time game engines.

In film production, we almost never use 8-bit textures for color/albedo, because of banding, etc. (JPEG is especially problematic since by spec, it's sRGB rather than linear values.) We either use 'half' (16 bit float) or 16-bit unsigned integer values for color/albedo textures.


Physically based shading - ambient/indirect lighting I implemented a physically based path tracer after studying PBRT by M. Pharr and G. Humphreys. Now I'm trying to apply physically based rendering to real time graphics using OpenGL ES (in an iPhone application).

I want to start using Oren-Nayar and Cook-Torrance as diffuse and specular BRDF but I have a problem: how do I model indirect lighting?

In a path tracer (like the one contained in pbrt) the indirect/ambient light is given "automatically" from the path tracing algorithm, as it follows the path of light rays taking into account direct and indirect lighting.

How do I model the indirect lighting in a physically based render written in OpenGL ES, so using real time computer graphics?

Real-time graphics deploys a variety of approximations to deal with the computational expense of simulating indirect lighting, trading off between runtime performance and lighting fidelity. This is an area of active research, with new techniques appearing every year.

Ambient lighting

At the very simplest end of the range, you can use ambient lighting: a global, omnidirectional light source that applies to every object in the scene, without regard to actual light sources or local visibility. This is not at all accurate, but is extremely cheap, easy for an artist to tweak, and can look okay depending on the scene and the desired visual style.

Common extensions to basic ambient lighting include:

Make the ambient color vary directionally, e.g. using spherical harmonics (SH) or a small cubemap, and looking up the color in a shader based on each vertex's or pixel's normal vector. This allows some visual differentiation between surfaces of different orientations, even where no direct light reaches them.
Apply ambient occlusion (AO) techniques including pre-computed vertex AO, AO texture maps, AO fields, and screen-space AO (SSAO). These all work by attempting to detect areas such as holes and crevices where indirect light is less likely to bounce into, and darkening the ambient light there.
Add an environment cubemap to provide ambient specular reflection. A cubemap with a decent resolution (128² or 256² per face) can be quite convincing for specular on curved, shiny surfaces.

Baked indirect lighting

The next "level", so to speak, of techniques involve baking (pre-computing offline) some representation of the indirect lighting in a scene. The advantage of baking is you can get pretty high-quality results for little real-time computational expense, since all the hard parts are done in the bake. The trade-offs are that the time needed for the bake process harms level designers' iteration rate; more memory and disk space are required to store the precomputed data; the ability to change the lighting in real-time is very limited; and the bake process can only use information from static level geometry, so indirect lighting effects from dynamic objects such as characters will be missed. Still, baked lighting is very widely used in AAA games today.

The bake step can use any desired rendering algorithm including path tracing, radiosity, or using the game engine itself to render out cubemaps (or hemicubes).

The results can be stored in textures (lightmaps) applied to static geometry in the level, and/or they can also be converted to SH and stored in volumetric data structures, such as irradiance volumes (volume textures where each texel stores an SH probe) or tetrahedral meshes. You can then use shaders to look up and interpolate colors from that data structure and apply them to your rendered geometry. The volumetric approach allows baked lighting to be applied to dynamic objects as well as static geometry.

The spatial resolution of the lightmaps etc. will be limited by memory and other practical constraints, so you might supplement the baked lighting with some AO techniques to add high-frequency detail that the baked lighting can't provide, and to respond to dynamic objects (such as darkening the indirect light under a moving character or vehicle).

There's also a technique called precomputed radiance transfer (PRT), which extends baking to handle more dynamic lighting conditions. In PRT, instead of baking the indirect lighting itself, you bake the transfer function from some source of light—usually the sky—to the resultant indirect lighting in the scene. The transfer function is represented as a matrix that transforms from source to destination SH coefficients at each bake sample point. This allows the lighting environment to be changed, and the indirect lighting in the scene will respond plausibly. Far Cry 3 and 4 used this technique to allow a continuous day-night cycle, with indirect lighting varying based on the sky colors at each time of day.

One other point about baking: it may be useful to have separate baked data for diffuse and specular indirect lighting. Cubemaps work much better than SH for specular (since cubemaps can have a lot more angular detail), but they also take up a lot more memory, so you can't afford to place them as densely as SH samples. Parallax correction can be used to somewhat make up for that, by heuristically warping the cubemap to make its reflections feel more grounded to the geometry around it.

Fully real-time techniques

Finally, it's possible to compute fully dynamic indirect lighting on the GPU. It can respond in real-time to arbitrary changes of lighting or geometry. However, again there is a tradeoff between runtime performance, lighting fidelity, and scene size. Some of these techniques need a beefy GPU to work at all, and may only be feasible for limited scene sizes. They also typically support only a single bounce of indirect light.

A dynamic environment cubemap, where the faces of the cubemap are re-rendered each frame using six cameras clustered around a chosen point, can provide decently good ambient reflections for a single object. This is often used for the player car in racing games, for instance.
Screen-space global illumination, an extension of SSAO that gathers bounce lighting from nearby pixels on the screen in a post-processing pass.
Screen-space raytraced reflection works by ray-marching through the depth buffer in a post-pass. It can provide quite high-quality reflections as long as the reflected objects are on-screen.
Instant radiosity works by tracing rays into the scene using the CPU, and placing a point light at each ray hit point, which approximately represents the outgoing reflected light in all directions from that ray. These many lights, known as virtual point lights (VPLs), are then rendered by the GPU in the usual way.
Reflective shadow maps (RSMs) are similar to instant radiosity, but the VPLs are generated by rendering the scene from the light's point of view (like a shadow map) and placing a VPL at each pixel of this map.
Light propagation volumes consist of 3D grids of SH probes placed throughout the scene. RSMs are rendered and used to "inject" bounce light into the SH probes nearest the reflecting surfaces. Then a flood-fill-like process propagates light from each SH probe to surrounding points in the grid, and the result of this is used to apply lighting to the scene. This technique has been extended to volumetric light scattering as well.
Voxel cone tracing works by voxelizing the scene geometry (likely using varying voxel resolutions, finer near the camera and coarser far away), then injecting light from RSMs into the voxel grid. When rendering the main scene, the pixel shader performs a "cone trace"—a ray-march with gradually increasing radius—through the voxel grid to gather incoming light for either diffuse or specular shading.

Most of these techniques are not widely used in games today due to problems scaling up to realistic scene sizes, or other limitations. The exception is screen-space reflection, which is very popular (though it's usually used with cubemaps as a fallback, for regions where the screen-space part fails).

As you can see, real-time indirect lighting is a huge topic and even this (rather long!) answer can only provide a 10,000-foot overview and context for further reading. Which approach is best for you will depend greatly on the details of your particular application, what constraints you're willing to accept, and how much time you have to put into it.


Selection of GI techniques working in real time I'm considering topic of my thesis connected with computer graphics - I would compare some GI techniques which work completely in real time (no precomputation to lightmaps etc. and each frame everything is calculated anew) to narrow my topic.

I plan to compare two kind of techniques:

based on RSM sampling heuristics (Radiance Hints)
based on cone tracing of scene voxelization calculated in every frame (Voxel Cone Tracing)

Each technique is based on grid moving with camera in voxel size increments.

The key I think is to describe pros and cons of kind of technique in more general way. For example some other techniques are based on RSM sampling. I think two kinds should suffice.

Is there any other important or popular kind of technique which I should include or I am not aware of?

In this vein a bit, Killzone renders one bounce reflections, but uses temporal anti aliasing to combine the last frame with the current frame.

In practice, this means you get MANY bounce reflections, since in effect each frame rendered catches the previous number of reflections as the new reflection - adding one each frame.

Not quite GI, but i wouldn't be surprised if people pursue that direction more in the future, if they aren't already working on it.

The temporal anti aliasing used normally lets you get more samples with subsequent frames, but used in this way, it also gives you higher bounce counts with subsequent frames.

I could totally see doing path tracing type path following, choosing reflection or BSDF rays based on fresnel equations for instance, and having the temporal AA combine it over time.


Compute Speed of Spheres in 3D animation I have created two animations:

One where a "camera" moves towards the spheres creating an optic flow (1st GIF).The code can be found here.
One where spheres move randomly in a 3D coordinates space (2nd GIF). To do so I have computed two angles theta and phi. The code can be found here.

I need to compute the speed of each sphere frame by frame in both animations, but I have no idea of the mathematics. I would assume that the formula would be different in the two animations. I have found only the calculation of 3D velocity, and I was wondering whether I could calculate the speed as a single value?

I'd use the euclidean distance between two 3D positions. Velocity is distance over time. Time is a single frame in this case.

$$\sqrt{(x2-x1)^2 + (y2-y1)^2 + (z2-z1)^2}$$

If you only move the camera, it is simply that frame of reference that moves, so all spheres have a relative velocity equal to the camera's velocity.


Could the Tetradic color pattern be expanded to allow /dev/random be "pretty" for cryptographic key purposes? Is there a way to take a random value (or set of values), and cause them to create a "pretty" pattern (as a thumbprint), and that array of thumbprints might have a rendering mask on just for aesthetic purposes.
Since I don't know much about all that can be done in Photoshop and Illustrator, I hope someone can help clarify my terminology such as "Random numbers (in the range of X/by way of formula Y) are best expressed in {Hue, Saturation, CMYK, RGB, chemically, theoretically}" ... etc

It depends on how you define “pretty”, but there’s plenty of possibilities. Assembling a grid of shapes as in Identicons would be one example; using the numbers for coordinates and zoom levels on a fractal like the Julia set would be another. Our eyes aren’t good at measuring absolute values of anything—perception is heavily dependent on context and lighting, cf. the dress—so you probably shouldn’t rely on color alone as the identifying factor.


Volume Rendering (Path Tracing & Real-Time) I have a few doubts and questions regarding volume rendering, I was hoping someone here could help me understand.

I commonly hear the terms 'Ray-Marching' & 'Volume Rendering' being used together when it comes to rendering fog or any other volumetric object.

Is Ray Marching the method used to render these volumes such as Fog, Smoke & Fire (this is regarding path tracing)? Since we can use signed distance fields to easily manipulate the shape of the volume? At the approximated intersection point is a BSDF evaluated (BRDF + BTDF) or is it just a BRDF?

At the approximated intersection point for the volume what BRDF/BSDF do we use (if we do use one, I might have a wrong understanding of the topic)? Are there measured BRDFs & BTDFs that is the common standard?

Ray marching is a ray tracing method where you take multiple steps along a ray to find intersection with geometry or to perform integration of in-scattered light from participating media (fog, clouds, water, etc.) along the ray. Signed distance fields doesn't really help you with participating media rendering since it's a method of finding the ray intersection instead of helping with volume integration.

So when you perform ray marching along the ray for volume integration, for each step you need to calculate incoming light to that point and evaluate the "phase function" to calculate how much of that light scatters towards the eye. This phase function can be simple isotropic function (i.e. constant $\frac{\rho}{4\pi}$) or more complex anisotropic function such as Mie phase function shown below, depending on the modeled participating media.

Additionally you need to take the extinction coefficient of the media into account, which is spatially varying for heterogeneous media, for calculating transmittance and to properly "attenuate" the in-scattered light as it's being blocked by media between eye and the point (Beer-Lambert law). This approach assumes homogeneous media between steps and for proper physically based integration you could take a look at SIGGRAPH 2015 presentation "Physically-based & Unified Volumetric Rendering in Frostbite" by Sebastien Hillaire.

To know how much light reaches a point in space for each of those ray steps, you can use path tracing. This can get quite expensive though since light can take random paths in the media until it reaches the point. This is known as "multiple scattering", which is analogous to indirect illumination. When performing the integration over the sphere for each step along the ray, you can use Woodcock tracking to get unbiased results for heterogeneous media. Also if you use Mie phase function you should use importance sampling as the phase function has very strong forward peak.


OpenGL Compute Shader Shared Maximum I'm working on an OpenGL compute shader in which each work item results in a single number output I'll call x, as well as a struct output with more information, I'll call y. What I would like is a single output for each work group being the y of the work item with the maximum x.

Now I'm aware of the GLSL atomic operations and I believe I could easily get the maximum x, but I'm having trouble thinking of how to get the y associated with the maximum x. I could, in the compute shader, do a conditional right after I check for maximum x to check if x changed and change y accordingly, but I believe there is no guarantee that no other work item has replaced the maximum x at that point. I think this might be possible with memory barriers but I'm having trouble figuring out if it really is.

So I guess my question is, is there an established method to do something like this? Am I perhaps over complicating it? Is there a way, with memory barriers, to create critical section-like behavior? Do I need to do the maximums on the CPU or in another serial compute shader?

You can figure out which thread has the max value by modifying x: pack gl_LocalInvocationIndex in the least significant bits of x. The number of bits you need for the index depends on your group size.

If you need all 32 bits of x then I suggest using 2 compute shaders - first record max per group. Then compute x again and compare.


Normalize NURBS knot vector I have NURBS surface data. I have a list of control points, knot vectors in U and V params and the degree. The U knot vector lies in range -3.14 to 3.14 and the knot V vector lies in range -100 to 100. How can I normalize this data so that both knot U and V lies in range 0 to 1?

Thanks for your help!

The relative size of the spacing of knots is irrelevant for the NURBS curve. The only thing that matters is that they keep the relation. Note this may not be wise as parametrization may have other uses behind the scenes.

Image 1: 3 differently parametrized knots result in same curve if knot values are relatively the same.

So you can scale and offset knot points as you wish. However you can not make the relative distances between entries different or your curve will change.

Image 2: On the other hand if you change the relative spacing your in trouble. So beware of floating point errors if you need to be really accurate.


Controlling reflection and refraction with material properties in ray tracing As tutors for this year’s programming languages course during our Computer Science studies we’re preparing for the students final project which is the implementation of a ray tracer. We’re implementing one on our own.

I’m having trouble understanding how the phong reflection model and reflection/refraction (Reflections and Refractions in Ray Tracing, Bram de Greve, 2006) can be used in conjunction.

Our materials currently have these properties:

ka, kd, ks – ambient, diffuse and specular terms defined as RGB colors
shininess – Controlling specular highlights (0 ≤ shininess)
reflectivity – Controls degree of reflectivity (0 ≤ shininess ≤ 1, 0 means not reflective)
refractive index – Defines the refractive index of the material (0 ≤ refractive index)

How I understand it, ray tracing as per phong reflection model with reflection and refraction works roughly like this:

The material is (semi)reflective and (semi)transparent (is this a word). In this case reflectivity and transmittance can be calculated with Fresnel equations. However then the reflectivity as specified in the material is ignored. The final color is reflectivity * trace(reflected_ray, depth + 1) + transmittance * trace(refracted_ray, depth + 1).
The material is (semi)reflective but not transparent. The reflective component is determined by further ray tracing. The other component is calculated via the equation from the Phong reflection model. Our ray tracer currently adds both components as in (1 - reflectivity) * color + reflectivity * trace(reflected_ray, depth + 1) where color is determined by the Phong equation.
The material is not reflective and not transparent. The Phong equation is used solely.

How do I know whether a material is (semi)transparent? Do I add a boolean property which I use to determine whether I’m in case 1 or 2?

It sounds to me like you're slightly mixing up several concepts: how lighting due to point-light sources (Phong model) and lighting due to other objects in the scene (recursively raytraced reflection or transmission) combine with each other, as compared to how specular lighting combines with diffuse lighting and/or transmission (using the Fresnel equations).

A useful way to think about specular vs diffuse is that specular happens at the surface of the material while diffuse, transmissive, and refractive properties are associated with the bulk (the interior) of the material. So you can think of it as two layers composited together. Specular is on top: it reflects a portion of the incoming light, defined by the Fresnel equations, and any remaining light proceeds into the material to light the bulk.

The bulk will have some combination of scattering and absorption properties. Low scattering makes a transmissive material and high scattering makes a diffuse material (the diffuse color is due to scattering on a scale too small to see). Some materials have an intermediate degree of scattering too, but we can ignore that for simplicity and just treat it as a boolean: opaque (Lambert diffuse) or transmissive.

Meanwhile, each of these potential material components (specular, diffuse, and transmissive) has to deal with light arriving both from point light sources and from the rest of the scene. Physically, all light sources are just emissive objects that are part of the scene (area lights) but again for simplicity we often approximate by using point lights.

So, putting it all together, when lighting a surface:

For each point light

Calculate Fresnel factor based on light vector
Accumulate Phong specular weighted by Fresnel reflectance
[if opaque] Accumulate Lambert diffuse weighted by Fresnel transmittance
[if transparent] Nothing; light goes through and isn't scattered into the camera.

For reflecting the rest of the scene

Calculate Fresnel factor based on view vector
Cast a reflection ray and accumulate it weighted by Fresnel reflectance
[if opaque] Nothing. (Ideally, accumulate Lambert diffuse due to all objects visible in the normal hemisphere of the surface, i.e. indirect diffuse. However, for a basic recursive raytracer this isn't feasible; it requires path tracing, photon mapping, or suchlike.)
[if transparent] Cast a transmission ray and accumulate it weighted by Fresnel transmittance

Also, a note about your material properties: physically, not all of those properties are actually independent of each other. Ks, reflectivity, and refractive index are all the same thing. Ks is the same thing as reflectivity (literally the same value) because both control specular reflections; Ks is for point lights and reflectivity is for reflected environment rays, but it's the same physical mechanism for both. And Ks is determined in turn by the Fresnel equations based on refractive index (which can vary with wavelength).

So, strictly speaking, you should pick one of those variables and let it determine the others. For opaque materials where you don't really care about refractive index, you might let users set Ks (at normal incidence) and backsolve the Fresnel equations to get the appropriate refractive index from that. For transparent materials like glass you can let users set the refractive index and then let the Fresnel equations do their thing.

BTW, to get some physics background that will hopefully help clarify all this, I highly recommend Naty Hoffman's "Physics and Math of Shading" talk from the SIGGRAPH 2015 Physically-Based Shading Course. It gives a quick and not-too-mathematical intro to the physical basis of things like diffuse and specular lighting, BRDFs and so on.


Physically realistic received irradiance values given some BRDF? I am trying to simulate the actual response of a camera given some object that is reflecting light.  I've written a ray tracer, and have a BRDF that I need to use, and I have a camera sensitivity in terms of signal/Watts.  But I am confused by one (rather important) detail:

Each ray coming out of the camera has some solid angle associated with it (this part I've already figured out).  So each ray should then have a "Radiance" value associated with it (as radiance has units of W/(sr*m^2)).  That way each ray would just multiply its solid angle by its radiance value, and you'd get an "Irradiance" value of W/m^2.  However, I am unsure how to actually calculate this initial radiance value for each ray.  The reason I am confused is because this seems to be backwards from what the BRDF is giving me.

A BRDF gives me the radiance leaving the surface in the direction of the camera, meaning the vertex of the solid angle is at the point of intersection.  The solid angle for the ray however is defined flipped, with the vertex of the solid angle at the camera itself.

How do I bridge this gap?  My idea is that is that if I can actually calculate the radiance for each ray, then I can simply multiple the radiance of each ray by its corresponding solid angle to get an irradiance value, then apply the inverse square law, and finally add up the irradiance for each ray per pixel and divide by the area of the pixel to get the wattage it receives.

But I am very lost as to how to calculate the radiance for each ray given that the only calculations I'm familiar with (the BRDF) returns a radiance value for a solid angle in the "wrong direction".

Am I misunderstanding what is going on?  Am I approaching this incorrectly?  Any help would be really appreciated!

I didn't get this part:

  That way each ray would just multiply its solid angle by its radiance
  value, and you'd get an "Irradiance" value of W/m^2.

I'll try to explain how the whole thing works though. You have a light source, it emits light rays. Those light rays bounce around the scene and eventually end up at your camera film. So at the end of the day you have multiple rays hitting the film surface and the radiance of those gets multiplied with the sensitivity function. For efficiency reasons you usually start backwards - from the camera. But if you work with symmetric BRDFs this should not matter. So what arrives at your film is radiance, you multiply that radiance with the sensitivity function and integrate within a "film pixel" to get the result in terms of your screen pixel.

Edit: To clarify what I meant by integrated out:

If you have radiance which is a measure of W/(sr * m^2), you can integrate out sr or m^2 like so:

$$E(x) = \int_{\Omega}{L(x,\omega)\cos\theta\,d\omega}$$

From where you get irradiance at a point $x$, which gives you the energy arriving from all directions at point $x$ (W/m^2). Now you can go further and find out the energy that arrives at some surface with area A, by integrating over all points on the surface:

$$\Phi = \int_{A}{E(x)\,d\mu(x)} = \int_{A}\int_{\Omega}{L(x,\omega)\cos\theta\,d\omega\,d\mu(x)}$$


Getting from the default view volume to an image on the screen I don't completely understand the last part of the MVP matrix (projection).

In orthographic projection we simply scale the view volume into the default view volume (+-1 on all axis).

In perspective projection we scale and skew the view frustrum into the default view volume (+-1 on all axis) which causes non-uniform foreshortening giving the appearance of perspective that we're used to.

So now, regardless of which type of projection we have used, we have the default viewing volume (+-1 on all axis). So how do we get from this cube to an image on the screen?

I understand that we need this volume for the z-buffer. But how are points (anywhere in this volume) finally transformed from 3d to 2d and is it the same procedure for orthographic projection as it is for perspective projection?

The mapping from NDC space to screen space is orthographic.

The x is mapped linearly from [-1, 1] to [0,viewport width], same with y and viewport height.

This mapping happens after the projection matrix has been applied and the divide by w happened.


Are there any gltf example datasets available? I'm currently researching my options on effeciently exchanging data for a webgl application. I understand the gltf format is still being drafted but I need some example data to understand whether this format is really useful for my application.

I'm currently using JSON data and I need to figure out whether there are any benefits to using data formats which are closer to the hardware. As far as I understand gltf also uses some JSON structures.

Are there any gltf example datasets available already?

Also check out the glTF models used in Cesium:

https://github.com/AnalyticalGraphicsInc/cesium/tree/master/Apps/SampleData/models
https://github.com/AnalyticalGraphicsInc/cesium/tree/master/Specs/Data/Models

Patrick


Sharing memory between compute shader and pixel shader I am relatively new to compute shaders and gpgpu in general so bear with me if i am missing the point somewhere.

The main thing i do not yet understand / see is how to get memory from a compute shader to global shader memory usable in a pixel shader without passing it back and forth between GPU and CPU.

I am implementing an irradiance caching lighting solution with Unity3D for learning purposes. I have written a compute shader that calculates the spherical harmonic coefficients for each volume. Now i can go ahead and read those values back to gpu using buffer.GetData() on the cpu side and shader.SetGlobalVectorArray(), but as far as I understand that will copy the memory to CPU resulting in allocation overhead.

I have seen similar posts where people were suggesting just doing the computation in a pixel shader to begin with, but i don't see how that is supposed to help? Which pixel shader and how? I only want these computations to be done once for each volume and they require parallel reduction.

Potentially my search terms are inadequate but i cannot find any answer to how to get results from compute shaders to global shader memory.
Thanks in advance

well, if anyone is stumbling across this and needs to know what solved it for me: We can bind a compute buffer to global shader memory if we just know what exactly we need to do

within the Compute shader the RWstructuredBuffer is declared as (without the "\" infront of float):

uniform RWStructuredBuffer<\float3> buffer : register(u1);

(not certain whether register(u1) is really needed

within the vert/frag shader the buffer is defined as:

uniform StructuredBuffer<\float3> buffer;

In a c# script before the first kernel dispatch (we only need to assign this buffer once), I assign the buffer via

buffer = new ComputeBuffer(size, sizeof(float)*3,ComputeBufferType.Default);
Shader.SetGlobalBuffer(Shader.PropertyToID("buffer"), buffer);

It took me ages to find this and it works as far as I can tell. The article I finally stumbled over that did something similar. I do not have to read the data back to cpu with buffer.GetData(container) and put it into the material manually.
If the compute shader changes the buffer data this will be reflected in the vert/frag shader immediately.


Passing mat4 array to GLSL succeeding but producing GL_INVALID_OPERATION I'm currently passing an array of bone positions to GLSL for a skeletal animation shader; the positions are passing successfully but producing a GL_INVALID_OPERATION error.

I'm passing the matrices in the following line:

std::vector<glm::mat4x4> bpos;
...
glUniformMatrix4fv(shader.uniform("bpos"), 50, GL_FALSE, &bpos.at(0)[0][0]);

Where the shader.uniform("bpos") position has been assigned earlier as:

this->m_uniforms["bpos"] = glGetUniformLocation(this->m_program, "uniform_bpos");

I'm getting the following output:

GL CALLBACK: ** GL ERROR ** type = 0x824c, severity = 0x9146, message = GL_INVALID_OPERATION in glUniform3(uniform "uniform_bpos"@1 is matrix)

Has anyone here encountered this error or does anyone here see what I'm doing incorrectly?

EDIT:

The vertex shader:

#version 330 core

layout(location = 0) in vec3 vert_ms;
layout(location = 1) in vec3 norm_ms;
layout(location = 2) in ivec4 bone_ids;
layout(location = 3) in vec4 bone_weights;

out vec3 norm_ws;
out vec3 pos_ws;

uniform mat4 uniform_mvp;
uniform mat4 uniform_bpos[50];

void main(){
     mat4 bone = uniform_bpos[bone_ids[0]] * bone_weights[0];
     bone += uniform_bpos[bone_ids[1]] * bone_weights[1];
     bone += uniform_bpos[bone_ids[2]] * bone_weights[2];
     bone += uniform_bpos[bone_ids[3]] * bone_weights[3];

     gl_Position = uniform_mvp * bone * vec4(vert_ms, 1);
     norm_ws = (uniform_mvp * bone * vec4 (norm_ms, 0)).xyz;
     pos_ws = vec3(gl_Position);
}

the function getting the uniform locations:

Shader::Shader(GLuint shader_program)
{
  this->m_program = shader_program;

  this->m_uniforms["mvp"] = glGetUniformLocation(this->m_program, "uniform_mvp");
  this->m_uniforms["bpos"] = glGetUniformLocation(this->m_program, "uniform_bpos");
  this->m_uniforms["eye"] = glGetUniformLocation(this->m_program, "uniform_eye");
}

from this file: https://github.com/EricJIsaac/opengl-engine/blob/feature/pb-materials/src/graphics/shader.cpp

uploading the vertex data:

void SkeletonMeshBuffer::predraw()
{
  glEnableVertexAttribArray(0);
  glEnableVertexAttribArray(1);
  glEnableVertexAttribArray(2);
  glEnableVertexAttribArray(3);
  glBindBuffer(GL_ARRAY_BUFFER, this->m_vbo);

  // Vertices
  std::size_t stride = sizeof(graphics::data::SkeletonMesh_Vertex);
  glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, stride, (void*)0);

  // Normals
  char* offset = (char*)0 + 3 * sizeof(float);
  glVertexAttribPointer(1, 3, GL_FLOAT, GL_FALSE, stride, (void*)offset);

  // Bone Ids
  offset = (char*)0 + 6 * sizeof(float);
  glVertexAttribIPointer(2, 4, GL_UNSIGNED_SHORT, stride, (void*)offset);

  // Bone Weights
  offset = (char*)0 + 6 * sizeof(float) + 4 * sizeof(unsigned short);
  glVertexAttribPointer(3, 4, GL_FLOAT, GL_FALSE, stride, (void*)offset);

  glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, this->m_ele);
}

void SkeletonMeshBuffer::draw(std::size_t id)
{
  if(id < 0 || id > this->m_entries.size())
  {
    printf("Invalid skeleton mesh id %lu requested\n", id);
    return;
  }

  char* offset = (char*)0 +
    this->m_entries[id].index_offset * sizeof(unsigned short);

  glDrawElementsBaseVertex(
    GL_TRIANGLES,
    this->m_entries[id].index_count,
    GL_UNSIGNED_SHORT,
    (void*)offset,
    this->m_entries[id].vertex_offset / 3);
}

void SkeletonMeshBuffer::postdraw()
{
    glDisableVertexAttribArray(0);
}

from this file: https://github.com/EricJIsaac/opengl-engine/blob/feature/pb-materials/src/graphics/skeleton_mesh_buffer.cpp

The meshes being loaded:
https://github.com/EricJIsaac/opengl-engine/tree/master/data/geometry/skeletonMeshes

The eye position (a vec3) was being bound to the wrong shader, resulting in the glUniform3 error when OpenGL (mesa) checked the validity of the uniforms. The error was due to a typo; I named my simple-mesh and animated-mesh shaders similar things.

glUseProgram(smesh_shader.get());
      smb.predraw();
      glUniform3fv(mesh_shader.uniform("eye"), 1, &eye[0]); // <- Oops
      graphics::ogl::render(scene.getRoot(), smb, mv, smesh_shader);
      smb.postdraw();

Following @Christian Rau's and @wychmaster's suggestions, I was able to debug the issue. I also needed to build a debug version of the mesa library to make sense of the stack-trace in gdb.


Do I need to use glMemoryBarrier with atomic counters? The OpenGL SuperBible 7th Edition points out that

  the glMemoryBarrier() function supports a bit specifically for synchronizing access to atomic counters with other parts of the OpenGL pipeline

and describes its usage.

The OpenGL Wiki says:

  Atomic counter memory access is not incoherent. So it follows the same rules as for texture reads, framebuffer writes, and so forth, not the Image Load/Store rules. You do not need to use a glMemoryBarrier to synchronize counter accesses.

So now I am confused:

Do I (and if so when do I) need to use glMemoryBarrier when using an atomic counter?

The post October 22, 2019 version of the OpenGL 4.6 standard gives atomic counters the same incoherency restrictions as image load/store and SSBO operations. You have to use explicit barriers to synchronize access to them and so forth. This is a textual change from older versions of the standard, but it isn't a behavioral change from implementations. The older wording is considered a bug and thus it should be assumed that atomic counters have always worked incoherently.

Note: the following predates the October 22, 2019 update to OpenGL 4.6.

From the OpenGL 4.6 specification, section 7.12: Shader Memory Access:

  As described in the OpenGL Shading Language Specification, shaders may perform random-access reads and writes to buffer object memory by reading from, assigning to, or performing atomic memory operation on shader buffer variables, or to texture or buffer object memory by using built-in image load, store, and atomic functions operating on shader image variables. The ability to perform such random-access reads and writes in systems that may be highly pipelined results in ordering and synchronization issues discussed in the sections below.

Notice something missing from that list? It mentions "shader buffer variables", and "shader image variables". But atomic counter variables are not mentioned. And atomic counter variables are a different kind of thing from either of those.

Therefore, that entire section (which is what explains the behavior of glMemoryBarrier) does not apply to atomic counters. Well, except where it specifically says otherwise, but that's only where it defines that helper FS invocations don't have side-effects. Note that even this explicitly calls out atomic counters as being something different from regular buffer or image operations: "stores, atomics, and atomic counter updates".

To lend greater credence to this view, the ARB_atomic_counter_buffer_object extension does not mention barriers at all. It doesn't require ARB_shader_image_load_store as a companion extension either, which is what defines glMemoryBarrier. Indeed, it has no interactions whatsoever with image load/store, while image load/store does have specified interactions with atomic counters.

Given the weight of the evidence in the specifications, I would have to say that the Wiki is right. In the interest of full disclosure, I did write that Wiki article. Though I also did all this research before I wrote it.

Basically, what the API seems to be saying is that buffer updates from atomic counters are treated like buffer updates from feedback operations. The implementation is required to track when you attempt to use that buffer for a read operation, then issue any synchronization needed to make sure that you can read the value. Or if you write to the value in the buffer (like clearing it to a value), then the implementation must automatically synchronize this.

The likely reason the specification permits this for atomic counters but not for image load/store is that the range of data for atomic counters is fixed. SSBOs and image load/store through buffer textures can write to arbitrary parts of the bound range of buffer data. By contrast, atomic counters write to 4 bytes times the number of atomic counter variables used by the shader.

Note however that atomic counters are affected by the in-shader memoryBarrier function. That is, atomic counters are subject to the requirements of, as the Wiki calls it, "Internal Visibility". Granted, that's not very useful most of the time, as the primary use of atomic counters is for when multiple invocations are all updating the same one.

On the other hand:

  the glMemoryBarrier() function supports a bit specifically for synchronizing access to atomic counters with other parts of the OpenGL pipeline

The specification describes this particular barrier as:

  Memory accesses using shader atomic counter built-in functions issued after the barrier will reflect data written by shaders prior to the barrier. Additionally, atomic counter function invocations after the barrier will not execute until all memory accesses (e.g., loads, stores, texture fetches, vertex fetches) initiated prior to the barrier complete.

What's curious about this is the "all memory accesses" part. This specific wording is only used in two other bits: shader storage and shader image. Other barriers for coherent operations like query buffers and transform feedbacks tend to use the term "all shader writes".

So... I would say that there is confusion at the specification level.


Are there specific techniques for generating False Radiosity in surfaces? I thought of this after my radiosity vs ray tracing question.

A radiosity algorithm seems to require increased calculations because it's viewpoint independent and therefore, false radiosity was born (at least it's the way it was explained). It is mentioned that in order to simulate radiosity, changes were done directly in texture maps (in PS for example) or omni-type lights are placed where radiosity effects occur.

I wonder if false radiosity is still used nowadays (I'm assuming this may happen more often in games development). If so, are these techniques currently used or are there others methods to achieve the same purpose? Any examples would be a plus.

Gobal illumination, radiosity, etc is not a mutually exclusive set with using a fill light to adjust the feeling. Sometimes the artist can do wonders with a few well placed lights.

So yes they are used, each game and situation is different. Perhaps they are just going for a unrealistic mood or the artist wants to tweak the result a bit.


How to remove background text from an image file I have this image below, I want to get rid of the text 'MSAR IDs' that is in the background. I am extracting the numbers from the image with OCR, the numbers that have a white background get extracted as is, however the numbers that have the text behind them don't get extracted properly and are not correct because they all have some part of the background text behind them. Can anyone please tell me how to get rid of the big text in background, any software will do for me.
(edit)PS- I got information that MSAR IDs are intellectual property of a company so to be clear I got this as part of a freelance work, my work is to put the numbers in an excel, so I was thinking of making my work a bit easier hence put up this post

It is all about filtering, you can see there are 4 colors, i.e. C_Font, C_Paper, C_Watermark_Paper, and C_Watermark_Font, the filter is just f(C_Watermark_Paper) = C_Paper and f(C_Watermark_Font) = C_Font.
Of course you can use GIMP and easily Google references, i.e. http://howto.nicubunu.ro/gimp-remove-watermark/


Generating and Combining Spectral Colors I have two somewhat related questions to ask:

What is the most accurate way to get the colors of a spectrum, without having to go deep into physics and simulating the universe? ;) Right now, I'm using these color matching functions, and simply reading off the XYZ color of a particular wavelength and converting it to sRGB. I know sRGB can't represent most spectral colors, but let's ignore that.
If I'm trying to make a color, say white, out of the spectral colors, how should I combine them? Should I add the color in XYZ space, or in RGB space, or is neither sufficient?

My plan is to experiment with spectral rendering, so it should be accurate enough to avoid biased renders. It'll most likely be on the GPU, and I need accuracy over speed.

Your way of calculating XYZ functions is probably the most efficient way to go about calculating accurate colors from a spectrum. It is standard practice afaik, for examples the books Physically Based Rendering (3rd) and Real-Time Rendering (3rd) both use this method.

You can add the colors in RGB space, but only if you convert from sRGB to linear RGB first. Otherwise you need to take into account, that sRGB sums will not lead to correct colors. The blogpost Adventures with Gamma-Correct Rendering by Naty Hoffman is a good read regarding this, topc:

  Computing shading in sRGB space is like doing math in a world where 1+1=3.

This is the problem you have when adding colors in sRGB.

As to whether $XYZ$ can be summed, I think so. If we look at the definition of the coordinates (for $S$ is the spectral function and $x, y, z$ are the CIE functions):

$X = \frac{1}{\int_\lambda y(\lambda) d\lambda} \int_\lambda S(\lambda) x(\lambda) d\lambda$

Now two $X$ coordinates $X_1, X_2$ would have the same definition, only the spectrals are different ($S_1, S_2$).

Thus you have

$X_1 + X_2 = \frac{1}{\int_\lambda y(\lambda) d\lambda} \int_\lambda S_1(\lambda) x(\lambda) d\lambda + \frac{1}{\int_\lambda y(\lambda) d\lambda} \int_\lambda S_2(\lambda) x(\lambda) d\lambda\\
= \frac{1}{\int_\lambda y(\lambda) d\lambda} \left(\int_\lambda S_1(\lambda) x(\lambda) d\lambda + \int_\lambda S_2(\lambda) x(\lambda) d\lambda\right)\\
= \frac{1}{\int_\lambda y(\lambda) d\lambda}\int_\lambda S_1(\lambda) x(\lambda) + S_2(\lambda) x(\lambda) d\lambda \\
= \frac{1}{\int_\lambda y(\lambda) d\lambda}\int_\lambda (S_1(\lambda)  + S_2(\lambda)) x(\lambda) d\lambda\\
=X$

The same holds for other coordinates.

Therefore, your best option is probably to sum your XYZ coordinates and convert them to sRGB space in the end.


Volumetric rendering fundamental concepts and terminology Literature on rendering volumetric materials and effects tends to use a lot of mathematical physics terminology. Let's say that I have a decent handle on the concepts involved in surface rendering. What concepts do I need to understand for volumetric rendering? (Both real-time and offline rendering.)

What exactly is meant by light scattering in the context of volumetric rendering? (And why is it split into in-scattering and out-scattering?)
What is the relationship between transmission, attenuation, and absorption?
What is a phase function and how does it play into volumetric rendering? (In particular, the Henyey-Greenstein phase function.)
What is the Beer-Lambert law and how is it related to light scattering?

Basically, how do I make sense out of diagrams like this?

When I was first reading about all of this I stumbled upon this link which helped me better understand this large subject.  Also this goes into some more detail on things mentioned here.
Light scattering is a natural phenomenon which arises when light interacts with particles distributed in a media as it travels through it.  From Wikipedia:

Light scattering can be thought of as the deflection of a ray from a straight path, for example by irregularities in the propagation medium, particles, or in the interface between two media

In computer graphics there are models that have been developed to simulate the effect of light traversing volume objects from an entry point (Point A) to an exit point (Point B).  As the light travels from A to B it is changed due to interactions with particles and these interactions are often referred to as Absorption, Out Scattering and In Scattering.  Often you will see these split into two groups; Transmittance (Absorption and Out Scattering) which I like to think of as 'light lost' and In-Scattering ('light gained').
Absorption is basically incident light energy that is transformed into some other form of energy and therefore 'lost'.
Transmittance
Transmittance describes how light reflected behind a volume will be attenuated due to Absorption as it travels through a medium from A to B.  This is usually calculated with the Beer-Lambert law which relates the attenuation of light to the properties of the material through which it is travelling.
As the light travels through the medium there is a chance that the photons can be scattered away from their incident direction and therefore not make it to the eye of the observer and this is referred to as Out-Scattering.  In most models the Transmittance equation is changed slightly to introduce the concept of Out-Scattering.
In Scattering
Above we have seen how light can be lost due to photons been scattered away from the viewing direction.  At the same time light can be scattered back into the viewing direction as it is travelling from A to B and this is called In-Scattering.
Particle In-Scattering itself is a pretty complex topic but basically you can split it into Isotropic and Anisotropic scattering.  Modelling Anisotropic scattering would take a considerable amount of time so usually in computer graphics this is simplified by using a Phase Function which describes the amount of light from the incident light direction that is scattered into the viewing direction as it travels from A to B.
One commonly used non-isotropic Phase Function is called the Henyey-Greenstein phase function which can model Backward and Forward scattering.  It usually has a single parameter, g ∈ [−1,1], that determines the relative strength of forward and backward scattering.


iPhone GLU(OpenGL Utility Library) This link says iPhoneGLU says, this libraray supports below futures.

Matrix manipulation
Polygon tessellation

I would like to know whether I can use this library to draw primitives(lines,points,triangles,simple polygon).

Thank you.

I personally like glad. The have an online generator as well http://glad.dav1d.de

It will generate exactly what you need for your platform.


Emission of light from texture on WebGL I have to build a texture of a cube that models the emission of light by the object. Any tips? I really don't know where to start.

Presumably you are using a lighting system already, let's say Lambert shading for simplicity..

So we have a texture which is going to be illuminated by some light source. The following psuedo/shader code will do lambert shading as normal..

diffuseIllum = lightColour * dot(normal, lightDir);
outputColour = textureColour.rgb * diffuseIllum;

But we want certain parts of our texture to emit colours instead of being illuminated by only our light source.

One method would be to use an extra channel in the input texture, say the Alpha channel, to control which pixels are emissive

diffuseIllum = lightColour * dot(normal, lightDir);
outputColor = mix(textureColour.rgb * diffuseIllum, textureColour.rgb, texture.a);

Unfortunately this won't work so well if you want say a dark red glow because the output pixel will either be the emissive colour (dark red) or the lit texture colour, which could be much brighter. In effect our emissive areas could end up darker than the surrounding illuminated areas.

We could add the emissive colour to the illuminated texture but that limits how much control over final colour we have.

e.g.

outputColour = textureColour.rgb * diffuseIllum + textureColour.rgb * textureColour.a;

A more correct method would involve using 2 textures, one for diffuse illumination and another for purely emissive. These would be added together in the pixel shader.

e.g.

 outputColour = textureColour.rgb * diffuseIllum + emissiveTextureColour.rgb;


What should I do when a new sampled direction is below a normal oriented hemisphere? I generate random directions from the pdf of D term in GGX BRDF. On the picture, white dots are new directions and the cross is the light direction, with roughness = 1. What should I do with the directions that are generated below the hemisphere? Should and can I just discard them?

Disclaimer: I am assuming that you are implementing a classical Monte Carlo estimator.
The Problem
Discarding samples will change PDF of your sampling technique. You are cutting off part of the sampled domain where PDF is non-zero, which effectively leads to a trimmed version of the original PDF but implicitly re-normalized so the remaining part integrates to 1. If you don't adjust the directly evaluated PDF accordingly, it will lead to a biased estimator.
Practically speaking, implicit re-normalization increases the actual sampling PDF $p^{new}$, and if you use $p^{old}<p^{new}$ in your computations instead, the resulting Monte Carlo estimator will yield brighter values than it should:
$$
\frac{f(x)}{p^{old}(x)} > \frac{f(x)}{p^{new}(x)}
$$
A solution
Since adjusting the PDF isn't an easy thing to do, you will most likely need to treat the under-surface samples as valid but with zero contribution. Whether you handle this within your BRDF/BSDF or elsewhere in the renderer is your design decision.
Zero-contribution samples will, obviously, introduce some inefficiency in your renderer.
A better solution
You can improve the efficiency of your estimator by using a better sampling technique which tries to avoid creating samples under the surface. In case of the GGX normal distribution, there have been proposed some solutions by Eric Heitz | Eugene d’Eon in paper Importance Sampling Microfacet-Based BSDFs using the Distribution of Visible Normals. I believe there was one improved version of this technique (other that the one mentioned in the "Related Work" section) but I cannot recall the name of it. Maybe someone else can add it here...


Access frames of livestream A friend of mine offers a live-stream via TCP. I can view the live-stream in a browser by fetching it via web-socket. What I want to achieve is to segregate the video into frames. Furthermore, the frames have to be saved to disk as images (e.g. png). I don't want to use a screen recording tool, though.

My idea was to access the graphics card buffer via OpenGL or any other library and get every frame of the video live stream there. I just don't know if this is possible and where to start. Of course I know that every more or less modern graphics card has its own frame buffer. Is it possible to access this buffer and get every frame during the live stream is played in the browser?

I wouldn't recommend trying to write this code yourself. It's not a question of using into OpenGL - which won't get you another process' framebuffer - but of asking the OS. This is not at all easy to do, and it's work you don't need to do yourself.

Depending on what the streaming protocol is, the easiest way might be just to write or use a client for that protocol directly instead of hooking another application. Failing that, a screen capture tool such as OBS can already do what you want. It can capture another application's frames or your desktop with relative efficiency. You can set up overlays or write a plugin to manipulate the video the way you want. If you access the stream through a web browser, you might not even need to capture a window: it has the built-in feature of being able to render into your video a web browser component it runs itself (you tell it what URL to point to) without having to run a separate web browser and capture it. This feature is how most streamers get things like a Twitch chat view, score board, or timings onto their video.

OBS has a C API for writing fast, native plugins as well as Python and Lua interfaces. And it's open-source, so in the unlikely event you can't write a plugin to do what you want, you could hack it into a private fork directly.