Computational Geometry - Triangulation I'm trying to triangulate complex 3D objects, that may have holes in them. Now I tried going over the problem myself, after a few hours I didn't get anything that would work all of the time, so I've been searching around and without much luck. Does anyone know of any algorithms, or free libraries, that would suit my needs. Links to any relevant documentation would be appreciated as well.

Also as a note, I'm a self taught programmer/cs-tist.

I have a set of points and I need to provide a game engine the shape in triangles. I get these points from a shape, I then add points to add area, subtract area, or split the shape. After modifying/adding points I need to give the game engine the shape in triangles as well as other information. Holes could be anywhere in any shape.

What it means to "triangulate complex 3D objects" is not unambiguous.
Just one possible interpretation: You have a 3D polygon in space, and you
want to triangulate that. This is NP-hard:

  Barequet, Gill, Matthew Dickerson, and David Eppstein. "On triangulating three-dimensional polygons." Proceedings of the 12th Symposium on Computational Geometry. ACM, 1996. (ACM link.)


Radiance at a nearly perpendicular angle is almost infinite? As far as I understand, radiance is a measure of the "intensity" of light in a direction defined as a differential cone . It's measure is Flux per solid angle per projected area.  $L = \frac{d\Phi}{d\omega dA^\perp} = \frac{d\Phi}{d\omega dAcos(\theta)} $. Using this definition, it seems like as the $\theta$ angle increases, radiance increases, thus light is somehow more "intense" in a direction nearly perpendicular to the normal at point p. This contradicts my intuition.
Where am I interpreting this wrong?

It's only more intense if $d\phi$ remains constant. However, for light source with constant luminance the flux is a function of $cos(\theta)$, i.e. depends on the light's angle of incidence with the surface and cancels out the term.

Edit: I feel the need to clarify this a bit because you say that:

  thus light is somehow more "intense"

It's not that the "light is somehow more intense", but rather that the light has to be more intense to have same flux in grazing angles as when it's parallel to the surface. If you think of flux as particles hitting to this differential surface, the number of particles hitting the surface is smaller more perpendicular the light gets to the surface. So you have to throw more particles at the surface (i.e. increase the radiance) for the same flux until when the surface is completely perpendicular to the light, none of the particles hit the surface, thus the radiance goes to infinity.


Why is this conditional in my fragment shader so slow? I have set up some FPS-measuring code in WebGL (based on this SO answer) and have discovered some oddities with the performance of my fragment shader. The code just renders a single quad (or rather two triangles) over a 1024x1024 canvas, so all the magic happens in the fragment shader.

Consider this simple shader (GLSL; the vertex shader is just a pass-through):

// some definitions

void main() {
    float seed = uSeed;
    float x = vPos.x;
    float y = vPos.y;

    float value = 1.0;

    // Nothing to see here...

    gl_FragColor = vec4(value, value, value, 1.0);
}

So this just renders a white canvas. It averages around 30 fps on my machine.

Now let's ramp up the number crunching and compute each fragment based on a few octaves of position-dependent noise:

void main() {
    float seed = uSeed;
    float x = vPos.x;
    float y = vPos.y;

    float value = 1.0;

      float noise;
      for ( int j=0; j<10; ++j)
      {
        noise = 0.0;
        for ( int i=4; i>0; i-- )
        {
            float oct = pow(2.0,float(i));
            noise += snoise(vec2(mod(seed,13.0)+x*oct,mod(seed*seed,11.0)+y*oct))/oct*4.0;
        }
      }

      value = noise/2.0+0.5;

    gl_FragColor = vec4(value, value, value, 1.0);
}

If you want to run the above code, I've been using this implementation of snoise.

This brings down the fps to something like 7. That makes sense.

Now the weird part... let's compute only one of every 16 fragments as noise and leave the others white, by wrapping the noise computation in the following conditional:

if (int(mod(x*512.0,4.0)) == 0 && int(mod(y*512.0,4.0)) == 0)) {
    // same noise computation
}

You'd expect this to be much faster, but it's still only 7 fps.

For one more test, let's instead filter the pixels with the following conditional:

if (x > 0.5 && y > 0.5) {
    // same noise computation
}

This gives the exact same number of noise pixels as before, but now we're back up to almost 30 fps.

What is going on here? Shouldn't the two ways to filter out a 16th of the pixels give the exact same number of cycles? And why is the slower one as slow as rendering all pixels as noise?

Bonus question: What can I do about this? Is there any way to work around the horrible performance if I actually do want to speckle my canvas with only a few expensive fragments?

(Just to be sure, I have confirmed that the actual modulo computation does not affect the frame rate at all, by rendering every 16th pixel black instead of white.)

Pixels get grouped into little squares (how big depends on the hardware) and computed together in a single SIMD pipeline. (struct of arrays type of SIMD)

This pipeline (which has several different names depending on the vendor: warps, wavefronts) will execute operations for each pixel/fragment in lockstep. This means that if 1 pixel needs a computation done then all pixels will compute it and the ones that don't need the result will throw it away.

If all fragments follow the same path through a shader then the other branches won't get executed.

This means that your first method of computing every 16th pixel will be worst case branching.

If you want to still down size your image then just render to a smaller texture and then upscale it.


What are the side effects of biasing brightness in continuous spectrum raytracing? I want to model rays with a continuous range of frequencies so that I can get raytraced images with colour separation on refraction. I can model a light source with a specified frequency distribution by using the distribution to affect the probability of a random ray lying in a given frequency range, or alternatively I can choose frequencies from a uniform random distribution and make the brightness of each ray proportional to the frequency distribution at its particular frequency. I see the first as more physically accurate, but I suspect the second will give images that look "finished" with fewer rays. Is this intuitive suspicion correct? Are there any features that will be lost from the image with the second approach? Is there a way to get some of the speed increase without compromising the image?

Generally, uniformly-weighted samples with a variable distribution (importance sampling) gives lower variance in the final average than uniformly-distributed samples with variable weights.  This is a common rule of thumb in Monte Carlo raytracing.

However, another thing to consider is that you'll eventually be converting the images to RGB for display (I assume).  So a potential problem might be that if a light source has very little energy in the blue part of the spectrum, for instance, then you'll put few samples in the blue frequencies, and the blue channel of the final RGB image could end up excessively noisy compared to the other channels.

One way to resolve this might be to consider the product of the light source's spectrum with the RGB color-matching curves used to generate the output.  You could normalize the three against each other to ensure you get enough samples in all three channels, but still distribute the samples to the most important frequencies for each channel.

On balance, I suspect that simply using a uniform frequency distribution of samples will be simpler and give good results as long as the light source spectra are fairly smooth.  But if you have spectra with sharp spikes (e.g. LEDs, lasers, fluorescent lamps) then spectral importance sampling will probably be necessary.


Correct way to think about Fresnel effect I am using Schlick approximation to calculate the value of the Fresnel effect I found here

When I try to find corner cases for the formula I get the value of 1 when the angle between normal and viewing direction is 90° and 0,02 for 0° (I am using IOR 1 and 1.33). This means that when looking, for example, at a sphere its edges will reflect almost all of the light and its center will reflect just 2% of light.

But he doesn't make sense for me when I try to use it in the microfacet model for BRDF.

This means that the reflectance when looking directly from above on the surface will be just 2% which is unrealistically low. 98% of the energy would be absorbed by the surface. So just after 3 bounces, the ray would transmit less than 0,0008% of the light. This would darken an image and I would have to increase its emitted light by a great amount.

Am I thinking in a bad way about it?

My second idea what that it means how much light is directly reflected as a perfect reflection without being scattered my microfacet to other directions. The rest is used for diffuse/glossy reflection. But I cannot find any evidence for it anywhere.

Or I don't simply have to use Fresnel when I am dealing with non-transparent materials.

UPDATE:

This is my code for now.

//Position of intersection is world space
float3 hitpoint = ray.origin + ray.direction * ray.distance;

//Direction towards camera
float3 wo = -ray.direction;

//Geometry normal
float3 N = ray.worldNormal;

//Microfacet normal
float3 H = SampleMicrofacetNormal(roughness, &N, &ray);

//New direction of ray
float3 wi;

// Add light emitted from surface
ray.color += ray.mask * emission;

float F = Fresnel(&H, &wi);

float r1 = new_random(&ray.seed1);

if (r1 <= F) {
    //Specular

    //Reflected direction
    float3 wi = reflect(&H, &wo);

    float D = D_ggx(roughness, &N, &H);
    float G = G_CookTorrance(roughness, &N, &H, &wo, &wi);

    float weight = fabs(dot(wo, H)) / fabs(dot(N, wo) * dot(N, H));

    //Not using F because it is already taken into a count by
    //the probability of this IF/ELSE statement
    float reflectance = G * weight;

    ray.mask *= reflectance;

    //Check - new direction cannot point towards surface
    if (dot(wi, N) < 0) ray.mask = (float3)(0.0, 0.0, 0.0);
}
else {
    //Diffuse
    //Ray change color because light to through the material
    ray.mask *= color;

    //New random direction of diffuse ray
    wi = SampleDiffuseCosineWeighted(&N, &ray);

    //Importance sampling weight
    ray.mask *= dot(wi, N);
}

//Set new direction and position for next tracing
ray.direction = wi;
ray.origin = hitpoint + N * EPSILON;

I will answer based on my experience with Fresnel models. Due to Kirchhoff's law of thermal radiation, in an opaque material the emissivity plus reflectivity plus absorptivity equals to one for every direction in thermal equilibrium. If the BRDF model you are using is physically-based, it should preserve energy conservation. That means that for a given viewing angle, the integral of the equation for the entire hemisphere should be equal to 1. It may be possible the reflectivity value near the normal is low, but that means that most of its contribution is elsewhere. For example, for a near-normal viewing direction, you can have a really low specular reflectivity but a much higher diffuse component. That may be the case of your model.


Reddish tint with spectral renderer I'm trying to implement a spectral path tracer and comparing results from my program with renders of the same scene done with pbrt and mitsuba. The scene is just some spheres in a box with one point light. All the surfaces have simple Lambertian BRDFs. The light has "uniform" spectral intensity (i.e., every wavelength has same power as others), same holds for surface reflectances. When I try to render the scene in mitsuba I get a "grayscale" image, as expected:

However, if pbrt's used to render the same scene, result image has a reddish tint:

With my program I get similar result:

After some digging through source code, I believe difference lies in mitsuba multiplying spectral intensities by D65 spectrum.

Is it reasonable to expect the image of the described scene to be "in grayscale"? If so, do light intensity spectra indeed are required to be multiplied by D65 spectrum? Again, if so, why pbrt doesn't do it?

Is it reasonable to expect the image of the described scene to be "in grayscale"? If so, do light intensity spectra indeed are required to be multiplied by D65 spectrum?

My expectation from a physically-based spectral rendered is that it would

compute the physical spectra
use the CIE color matching functions to relate that to a quantified color space (eg CIEXYZ)
convert that into the destination color space.

D65 shall not appear anywhere till step 3, and only if we actually render into sRGB (or similar).

A flat spectra translates to XYZ = (0.33, 0.33, 0.33), which is RGB = (1.20, 0.95, 0.91) prior to gamma compression. So the "red tint" is the expected result following the above procedure.

Don't know why Mitsuba would have such a 'feature'.


What is the name of the RGB thing I want to be able to do, and what free program(s) can do it automatically? I'm gonna start this off with an... "indexed image"? If that's the proper term for it?

If this 384x144 image were actually only the top-left 192x48 pixels (I'll refer to that portion as "0,0"), I feel like the other five sections could most likely be generated by using a single automatic operation that applies the following RGB value (swaps/rotations) to all pixels in section 0,0 simultaneously:

0,1: Red -> Green -> Blue -> Red (#FFAA55 -> #55FFAA for one example)
0,2: Red -> Blue -> Green -> Red (#FFAA55 -> #AA55FF)
1,0: Green <-> Blue (#FFAA55 -> #FF55AA)
1,1: Red <-> Green (#FFAA55 -> #AAFF55)
1,2: Red <-> Blue (#FFAA55 -> #55AAFF)

What is the name of the operation I'm looking for + what image editing programs can perform that operation? I tried Googling a bunch of things along the lines of "rgb rotate", "rgb exchange", "rgb swap", "rgb trade", etc. etc. etc. and I couldn't quickly + intuitively find anything that did what I wanted it to do in a single swing. The amount of stuff completely unrelated to what I had in mind that came up honestly quickly got really frustrating. I even tried Googling "triadic rotation" and nothing related to what I had in mind showed up on the 1st page.

The reason for the sorted colors instead of an actual image is basically the same reason I have to ask this question in the first place; I can't do it automatically with the free tools that I'm currently aware of the existence of.

I had to sort those 64 colors manually 6 times over with paint.net; which thankfully has functions that made it slightly easier to do, though there's definitely no way that I could quickly apply the effect I'm talking about to any actual image of any kind using paint.net (unless it has some feature I'm not yet aware of). I'm pretty sure there should be a way to apply that effect automatically; I just don't know the name of that effect + therefore can't even search for it.

One thing that came up a few times during my failed attempts at searching was "hue rotation"; however as far as I know + could observe that not only applies to only one hue range at a time, but also pays more attention to hue angle than actual RGB values (if that makes any sense). So I really don't think hue rotation is what I'm looking for; though if it's suggested to me anyway I will try it again as long as I'm given specific instructions on how to use it to achieve the effect that I desire.

If I'm not using a tag that I should be using, or even outright posting on the wrong Stack Exchange site for the question; please let me know.

Swapping color channels is called channel swap or swizzling. (Searching for "RGB channel swap" brings up lots of relevant results.) Looking at your image, the squares in the right half appear to be flipped across the diagonal.

Most editing apps have a channel mixer tool that allows you to calculate each output color channel as some combination of the input color channels. In Photoshop it's called "Channel Mixer". I've also seen it referred to as "Color Operations" or just "Color Ops".

The flipping can usually be done with a horizontal flip followed by a vertical flip.


How to randomly draw Quaternions within a specific range of Euler angles for rotation? I am pretty new to Quaternions so please bear with me. I want to draw random Quaternion samples so that their Euler angle equivalent would range within [-30, +30] degrees on each axis. Currently, I know how to sample Quaternions from the full range ([-180, +180]) using the code below but I don't know how to modify the code so that I can get samples within the range [-30, +30]. Can anyone help me with that?

I'm not sure if this is helpful to answer this question but here's a piece of information: I eventually want to convert the sampled Quaternion to Euler angles an apply the Euler rotation in some 3D shapes. The order of rotation in the software I'm using to do this is XYZ meaning that it first rotates the 3D shape along the X axis, then Y axis and then Z axis.

import numpy as np
def sample_Quaternion():
    r = np.random.uniform(0, 1 - 0.001, 3)
    while np.linalg.norm(r) > 1:
        r = np.random.uniform(0, 1 - 0.001, 3) # Just to keep the L2 norm within [0, 1.0)
    w = [np.sqrt(1 - (r[0]*r[0] + r[1]*r[1] + r[2]*r[2]))]
    r = np.concatenate(r, w) # the output of this would represent (x, y, z, w)
    return r

The simplest way to do this would be to generate the Euler angles randomly, then use them to create the quaternion. Most existing quaternion implementations in game engines and 3D math libraries will have a FromEuler ( Vector3 ) function.

Since you seem to be rolling your own quaternions rather than using a library, here's the conversion formula.

As a small optimization, since quaternions use half-angles, you could generate random values in the range ±15 degrees (pi/12) and feed these to the sin and cos functions directly.


How is lighting done in Rasterization based pipeline? So I wanted to know how do Game Engines like Unity3D and Unreal do lighting. I'm familiar with Ray-tracing, pathtracing based renderers etc. But not much with rasterization based engines. I do know a general overview of OpenGL's pipeline and things like vertex and fragment shaders but I don't know how exactly the color is computed.

I'm guessing Rasterization based pipelines use the same procedure for shading i.e. A ray is shot from the pixel and then from the triangle that got projected on that pixel to the light source?

For indirect lighting they use things like ambient occlusion I guess. Am I correct?

If this is true then the only difference between rasterization and pathtracing would be the procedure for Visible Surface Determination and Indirect Illumination. Direct Illumination would be calculated the same way in both techniques?

If you can provide a brief overview of the current state of the art techniques used in these engines for things like reflection/shadows etc it'd be appreciated.

I'm guessing Rasterization based pipelines use the same procedure for shading i.e. A ray is shot from the pixel and then from the triangle that got projected on that pixel to the light source?

Close, but not quite. There's no actual tracing of rays but the underlying linear algebra is the same. It sounds like you already understand the first part of rasterisation, as far as getting some fragments which need to be shaded. Each fragment knows the attributes of the triangle it came from (the world position, normal, etc.). In traditional forward shading, the fragment shader runs once per light, and it also knows the attributes of the light, including the world-space direction from the shading point to the light: the direction you would trace a shadow ray.

Although no shadow ray is traced, the actual lighting calculation is exactly the same: you use the view and light directions and the surface normal in whatever BRDF you want. You just repeat the process for each light and sum the results. At the end, you have the same result that you'd get from a very simple ray-tracer, without any shadows, reflections, refractions, or indirect lighting.

How to add those missing features? Well, there are several possible techniques for each, but I'll briefly go over the simplest. A simple way of getting shadows is using shadow maps. Before you render your main scene, you do a render into an off-screen buffer, but using the light as a camera. (The projection and shape depend on the light, e.g. orthogonal projection for a directional/cylindrical light.) The trick is, you don't render all the colours: you only draw into the depth buffer. What you get is a picture of the distance from the light to the closest thing in each direction. Later on, when you're shading for that light's contribution in the fragment shader, you calculate the distance from the light to the shading point, and compare it to the value in the depth buffer. That tells you whether the shading point is the closest thing to the light along that ray, without having to run an intersection test.

Planar reflections and refractions are pretty simple too: just render the scene but with the camera transformed to produce the reflection or refraction image. For example, for a planar reflection in a lake or floor, you reflect the camera position about that plane to draw the whole scene upside-down. When you draw the real scene around it, any transparency shows that reflection.


Why are quads used in filmmaking and triangle in gaming? In film school, in the classes of 3D modeling, I was told that when we model something for films we maintain topology of 4 edged polygons. Any polygon which has more or less than 4 edge/vertex is considered bad and should be avoided. Whereas if the same model is used in games, it is triangulated.

Although I'm not majoring in 3D modeling, the question is still in my mind. Why is 3 edged polygon used in gaming? Does it render faster? Then why not is it used in film renderings?

For 3D modeling, the usual reason to prefer quads is that subdivision surface algorithms work better with them—if your mesh is getting subdivided, triangles can cause problems in the curvature of the resulting surface. For an example, take a look at these two boxes:

The left one is all quads; the right one has the same overall shape, but one corner is made of triangles instead of quads.

Now see what happens when the boxes get subdivided:

Different, yeah? Note the way the edge loops have changed from being roughly equidistant on the left box to a more complicated, pinched-and-stretched arrangement on the right. Now let’s turn off the wireframe and see how it’s getting lit.

See the weird pinching in the highlight on the right box? That’s caused by the messy subdivision. This particular one is a pretty benign case, but you can get way more messed-up-looking results with more complex meshes with higher subdivision levels (like the ones you’d usually use in film).

All of this still applies when making game assets, if you’re planning to subdivide them, but the key difference is that the subdivision happens ahead of time—while you’re still in quad-land—and then the final subdivided result gets turned into triangles because that’s what the graphics hardware speaks (because, as mentioned in the comments above, it makes the math easier).


How does the Modified Phong Lighting Model from the Phong Lighting Model? How does the Modified Phong lighting model also known as the Blinn-Phong differ from the Phong Lighting Model?

How can I distinguish the two?

The Phong lighting model computes the specular response as the dot product between the mirror reflection direction and the viewing direction, raised to a power.

For example, if $\vec{V}$ is the viewing direction, $\vec{L}$ the incoming light direction and $\vec{R}$ the perfect specular reflection direction for $\vec{L}$, then the specular response is $\text{max}(\vec{V}.\vec{R},0)^p$, where $p$ is the exponent that controls the fall-off of the specular response.

The Blinn-Phong model uses a half vector $\vec{H}$, which is computed as $\vec{H} = \frac{\vec{L}+\vec{V}}{|\vec{L}+\vec{V}|}$, which is then used to compute the specular response as $\text{max}(\vec{H}.\vec{N},0)^p$, where $\vec{N}$ is the surface normal. The idea is that when the eye vector $\vec{V}$ is aligned perfectly with the perfect mirror direction $\vec{R}$, the half vector $\vec{H}$ would be exactly aligned with the surface normal $\vec{N}$. Hence the Blinn-Phong can get away with computing an expensive mirror direction $\vec{R}$ everytime by using the half vector which can model the specular response.


How do you add text dynamically to an image and export it? Apologize for crossposting, but I posted this in Stack Overflow and thought it might be a better post here.

Recently, Spotify released their 2019 Spotify wrapped web application, which is essentially a way for both music artists and music listeners to rewind and review what they listened to/who listened to their music in the past year.

As part of their site, they dynamically created these images that people could either export or share to one of their social media networks.

Above is an example of one of those dynamically generated images (of course, the image, and the text/statistics below change).

How would I go about doing this in code?

Sorry if this is in the wrong place - just wasn't sure where I would post this. Thanks in advance!

To do this in a web app is trivial. Judging by the image, even the built-in HTML canvas should be able to do everything that you need.

Once you have obtained your image and text, to give you a vague idea of the ammount of code involved, drawing them with the canvas API looks roughly as follows:

ctx.drawImage(img, x, y, width, height);
ctx.font = '12px serif';
ctx.fillText(txt, x, y);

Saving the image is equally simple:

canvas.toDataURL("image/png")

It's actually the obtaining and sharing of images are the scary parts. Spotify surely have all the images on their cloud and similarly that is where they export them to before sharing to social media. There are so many sollutions that without more detail this is probably something you'll need to find yourself.

If you just need a tutorial on html canvas, mozzilla usually have great interactive tutorials.


What is the purpose of this cube in color quantization? So, I go to install GMIC for Gimp via the package manager and landed myself upon the demo program. I have discovered things such as Hough transform and  color quantization. So, I get the number of colors on the bottom bar, I understand what it is doing, but I do not get the cube in the middle, mainly why it's 3D or that it has other shapes inside of it (other than the dots for the colors chosen). Basically, I want to know the purpose of this cube, such as why is it a cube, or why is it 3D.

Edit:

I hope this is a better picture. Turns out the image is different too!

The cube looks like just an RGB cube, i.e. X,Y,Z map to R,G,B. Since there are 3 color channels, the 3D cube is a convenient way to visualize the space of all possible RGB colors. However, I don't know what the pie-wedge-looking shape outlined inside the cube is.


Is it good practice to use all the available texture units? When applying multiple textures to a mesh, like for bump-mapping, I usually bind the textures to the first few fixed texture units, e.g.: diffuse = unit 0, bump = unit 1, specular = unit 2, then keep reusing those to each different mesh with different textures. But I've always wondered why glActiveTexture supports so many texture units (in the previous link, it says at least 80).

So it occurred to me that one possible way of managing textures is to bind distinct textures to each available unit and leave them enabled, just updating the uniform sampler index. That should improve rendering perf by reducing the number of textures switches. If you have less textures than the max texture units, you never have to unbind a texture.

Is this standard practice on real-time OpenGL applications (I believe this also applies to D3D)? And are there any non obvious performance implications of taking this approach? Memory overhead perhaps?

Modern hardware doesn't really have the concept of texture binding points as exposed by OpenGL. Rather, the shader unit uses a descriptor (which is just some kind of fat pointer) which can potentially address any texture as long as it's resident in video memory. This is what makes things like bindless textures possible. So the large amount of "texture units" available in current implementations is simply trying to handwave this now-irrelevant part of the API.

That said, the recommended way of implementing what you're trying to do (avoiding re-binding of textures) is to use texture arrays, which allow you to dynamically index into a set of textures in the shader, as long as they all have the same format and size. This presentation contains more details about these and other techniques for reducing driver overhead when rendering with modern OpenGL: Approaching Zero Driver Overhead


how to map rectangular coordinate system onto JavaFX GraphicsContext canvas I am trying to figure out how to apply the proper sequence of translate and scale commands (or a single .transform command) so that the default pixel grid (0,0,1920,1080) for instance is setup so that the coordinate system  (minx,miny,maxx,maxy) can be used where for instance
minx=100 maxx=200 miny=-5 maxy=5

I am trying to render mathematical functions and map the results to pixels and use the mouse to select certain regions to zoom in on.. my current code is at

https://bitbucket.org/stephenc214/fastmath/src/default/src/fastmath/fx/HardyZMap.java

I would really appreciate any help... I know this is simple but I just can't seem to get it to work properly

You can map [x, y] pixel coordinates in an image of size [width, height] to your given range as follows:

x'=minx+(maxx-minx)*(x+0.5)/width;
y'=miny+(maxy-miny)*(y+0.5)/height;


light probe (for global illumination) placement techniques? I'm learning light probes used for dynamic global illumination. I have a question regarding the placement of light probes, as based on most of the pictures I have seen, they seem to be placed uniformly in a grid. Is that a reasonable way for placement? I feel that more should be placed in corners than in the middle of an area.

is there any research on light probe placement that minimize the overall data needed for rendering?

this GDC talk http://twvideo01.ubm-us.net/o1/vault/gdc2012/slides/Programming%20Track/Cupisz_Robert_Light_Probe_Interpolation.pdf

mentioned irregular placement on tetrahedrons and how to do interpolation. but it doesn't seem to say much about placement itself.

this paper http://melancholytree.com/thesis.pdf mentioned that only one probe is needed in a convex shape, but I don't seem to see people do this, is it because this is only for static global illumination without moving objects?

what's the the latest development in light probe placement?

Looking for answer for similar question, I found that work: http://www.cse.chalmers.se/~uffe/xjobb/bowald_final_master_thesis_v2.pdf

It is based on automatic grid placement which depends on scene voxelization. Inspired by this I developed similar technique:

First scene is voxelized with small voxel size (binary voxelization for memory saving)
Result is downsampled to desired grid size (size of each cube with eight probes on each corner)
Based on result - data structures are prepared
Empty areas are optionally computed - cubes size can be bigger to save memory of volumes and probes buffers but it must be always multiplication of smallest cube size to fit in the grid
Lighting for each probe is precomputed

Data structures are:

Volume map buffer - mapping world position to volume index
Volumes buffer - each is array of eight probe indices
Probes buffer - SH data of each probe

This way performance is constant, fast and independent of quality (size of cube) unlike in tetrahedra grid searching algorithm.

However I don't know if there is more recent advancement in that field. You might check http://jcgt.org if there are any new research papers.


How does the rendering equation incorporate shadowing This is how the rendering equation is written in the textbook

$$L(p,\omega) = L_e(p,\omega) + \int f(p,\omega_i,\omega) \, L(p*,-\omega_i)\cos \theta \, d\omega_i$$

which component of this equation handles the shadowing?

Light that is blocked will mean that $L(p*, -w_i)$ under the integral is 0 plus how much light the blocking object itself reflects.

In other words the shadowing is embedded in the incoming light function.


Illumination model used in the "Raytracer in a weekend" series I was going through the raytracer in a weekend book by Peter Shirley and the subsequent book raytracing the next week and I couldn't make out what was the illumination model that was being used here.

all the previous raytracing samples that I have come across use the Phong shading model in which

For a diffuse surface, a shadow ray is generated towards all the light sources and the final color is calculated based on the diffuse color and the non-occluded light intensities
For reflective materials and dielectrics, reflective and refractive rays are generated which are then recursively projected into the scene

The method used in the book (very briefly) is as follows (Please do correct me if my understanding is wrong in any way)

For diffuse objects, a ray is traced recursively in a random direction biased towards the normal at the hitpoint and the color from the resulting ray is multiplied with the current object's albedo to give the final color
For reflective and dielectrics, rays are cast using the law of reflection and the Snell's law respectively.
Lights also scatter rays and also add their emitted parts.

The major differences I see in this approach are

We do not directly sample the light sources.
Diffuse items also reflect rays (for inter-diffuse reflections)

What exactly is the illumination model used in this approach? and why does this approach seem so simple and yet produces pretty good results?

Sorry if I missed something, I am relatively new to raytracing

It’s a “naive” Monte Carlo sampler so it’s too dumb to send rays toward lights— it shotguns.   But the light is just a hemisphere (like a cloudy day) so you don’t get punished with too much noise.   Cloudy days look pretty good in real life.   But if you try to add a small light it will work but get very very noisy.   The third book in that series uses one method to address this (basically sending more rate where lights are rather than shadow rays per se)


Weird patch on reflective spheres on planes in raytracer I am writing a small raytracer in python , everything was working fine but when i implemented plane geometry then the reflective spheres are getting weird patches on lower bottom. image below:

I guess this is happening because of some error in calculating lighting with wrong normals? but i am unable to figure out exactly what.

code to find palne-ray intersection:

class Plane(Shape):
    def __init__(self, color, y=0, specular=-1, reflective=0):
        self.specular = specular
        self.reflective = reflective
        self.center = Vec3(0,y,0)
        self.normal = Vec3(0,-1,0)
        self.color = color

    def intersect_at_point(self, origin, ray):
        denom = ray.dot(self.normal)

        if abs(denom) > 0.0001:
            diff  = self.center - ray
            t = diff.dot(self.normal) / denom

            if t > 0.0001:
                return t
        return None

trace ray function.

def trace_ray(self, origin, direction, t_min, t_max, depth):
    closest_object, closest_t = self.closest_intersection(origin, direction, t_min, t_max)

    if closest_object == None:
        return Vec3(173/255, 216/255, 230/255)

    # Compute local color
    P = origin + closest_t * direction  # Compute intersection
    N = P - closest_object.center  # Compute  normal at intersection
    N = N / N.mag()
    local_color = closest_object.color * self.compute_light(P, N)

    # If we hit the recursion limit or the object is not reflective, we're done
    r = closest_object.reflective
    if depth <= 0 or r <= 0:
        return local_color

    # Compute the reflected color
    R = self.reflect_ray(-1*direction, N)
    reflected_color = self.trace_ray(P, R, 0.001, math.inf, depth - 1)

    return local_color * (1 - r) + reflected_color * r

closest intersection method.

def closest_intersection(self, O, D, t_min, t_max):
    closest_t = math.inf
    closest_object = None

    for object in self.world.objects:
        t1, t2 = object.intersect_at_point(O, D)

        if (t_min < t1 < t_max) and t1 < closest_t:
            closest_t = t1
            closest_object = object

        if (t_min < t2 < t_max) and t2 < closest_t:
            closest_t = t2
            closest_object = object

    for plane in self.world.planes:
        t = plane.intersect_at_point(O,D)

        if t and (t_min < t < t_max) and t < closest_t:
            closest_t = t
            closest_object = plane

    return closest_object, closest_t

Calculate light method.

def compute_light(self, Point, Normal):
    i = 0.0
    P = Point
    N = Normal

    for light in self.world.lights:
        if light.type == 'ambient':
            i += light.intensity
        else:
            if light.type == 'point':
                L = light.position - P
                t_max = 1

            if light.type == 'directional':
                L = light.direction
                t_max = math.inf

            # Shadow check
            shadow_sphere, shadow_t = self.closest_intersection(P, L, 0.001, t_max)
            if shadow_sphere != None:
                continue

            n_dot_l = N.dot(L)

            if n_dot_l > 0:
                # print(f'light intensity {light.intensity}')
                i+= light.intensity * n_dot_l / (N.mag() * L.mag())

    return i

I have to disagree with @jack11111, this is definitely not a clipping issue. I have two possible explanations which you can easily test and verify.
Clipping theory

The patch looks pretty constant in color, clipping would be sort of a gradient (modulo 255) of full intensity in at least one channel (R/G/B).
There is no direct light source to cause a strong specular highlight. According to the ground shading and the Lambert-shaded green balls a point light source is placed above, little bit in front of the camera. It seems to have normal intensity that should not cause any trouble. In fact I see in your code that you apply only basic Lambert n . l shading, so no specular highlights possible.
There is no indirect source to cause such a peak. The rays reflect to towards ground that is rather dark. The bright spot on the ground is located else where. That is well visible on the red sphere where the error occurs closer than the next green sphere, but the highlight on the ground is even further away.
It is a strange coincidence that the secondary color of the patch is exactly the same for both spheres.

Normals theory

Normals seem to be computed correctly elsewhere, all your shading is consistent with nice gradients.
You reflections are also correct elsewhere, that will be on problem.

It's a trap! (almost)
Looking closely at your image I believe I can explain what is happening there. The rays hitting the suspicious regions at the sphere will reflect towards ground in a direction very similar to the ground normal.
You ground does not seem to be reflective, but if it were: The rays would reflect back in almost the exact opposite direction hitting back the ball and then heading towards the camera. So they would hit the ball twice which could cause its color to gain dominance. You can test this theory by comparing the rendering results with maxDepth = 1. If the discontinuity disappears then that is the problem.
Upside down
My second explanation is much more likely to be the case. It is also based on the observation that the rays reflected from the suspicious patches hit the ground in a direction pointing almost exactly down. I noticed a bit strange setup of your plane in your code. It has its normal pointing down!?
self.normal = Vec3(0,-1,0)

I am not sure how the Lambert shading of the ground works then (should be then all just ambient in fact), but assuming Vec(0,1,0) is your up then the intersection will in fact yield None
denom = ray.dot(self.normal) #something like 1e-6 in the suspicious area
if abs(denom) > 0.0001: #evaluates to false
    #... true branch ignored
return None #false branch activated, returns no intersection

Having no intersection means that the hit point is shaded only with a constant color

if closest_object == None:
   return Vec3(173/255, 216/255, 230/255)

So to test this explanation, just change the returned color for this case to black and see if the patch darkens.
if closest_object == None:
    return Vec3(0, 0, 0)

I hope that helps. Please let me know the results.
Update
After the author confirmed my second explanation I found the reason in the code. The source of the problem is in the ray-plane intersection method.
Most important of all the epsilon in the comparison should be omitted, since that is causing the patch to be discontinuous. As mentioned earlier, the incoming ray direction is almost Vec3(0,-1,0). You have set your plane normal to exactly Vec3(0,-1,0) so the dot product yields something very close to 0.
So the quick fix would be:
denom = ray.dot(self.normal)
if abs(denom) >= 0: #omit the epsilon
    #...

Your plane normal is inconsistent with how a normal is usually defined. I believe your plane is not supposed to face down. It would be much better and correct to have the plane normal pointing up and then testing for a negative dot product.
A front-face hit occurs when the face normal and the ray are pointing in opposite directions, i.e. their angle θ is smaller than -π/2 or larger than π/2, so cos(θ) < 0. Note that denom is in fact cos(θ). If |θ| < π/2, i.e. cos(θ) > 0, then it is a back-face hit. You ignore such hits for now but once you add transparency to your raycaster you will need them as well.
The correct way would be:
def __init__(self, color, y=0, specular=-1, reflective=0):
    # ...
    self.normal = Vec3(0,1,0) #pointing UP

    def intersect_at_point(self, origin, ray):
        denom = ray.dot(self.normal) # denom == cos(θ)

        if abs(denom) <= 0: #is negative <=> |θ| > π/2; omit the epsilon
            diff  = self.center - ray
            t = diff.dot(self.normal) / denom

            if t > 0: #I suggest to omit the epsilon here as well
                return t
        return None


Rounding the edges in a mitered line segment inside of a fragment shader As some of you may know drawing line segments between sets of points is really difficult especially when you want wide lines or variable widths.

Because I am looking to draw a few hundred of these line segments each frame I am having to go with the cheapest option to achieve a connected segment. So far this has meant that I am using a miter to join the two rectangles like so (described here)

The problem with this approach is that the extra length added by the miter is sometimes quite large especially in the case of acute angles and I would like to have rounded corners where possible.

I believe it is a job for the fragment shader as I don't have enough processing resources to calculate more geometry.

Here is a diagram showing what might end up happening after the fragment function:

Purple represents the area between the two lines that ought to be kept; the rest should be removed.

Does anyone know a way to achieve this rounded rectangle in the fragment shader even though texture co-ords are pretty useless in a scenario where the shape is not a polygon.

I wish I was allowed to just draw the rounded circle over the disjointed

Remember, whatever it is it needs to be fast.

First, adressing the concerns of speed and performance, you will always have to make some trade-offs between quality and performance, as you already do. If the first version looks fine enough for your use-case and all other versions are much slower, you have to make the decision yourself. You will, however, not get around actually measuring any time difference. But since various ways to do this are not too hard to integrate into an existing wide lines renderer, just testing the various approaches in action and getting some time measurements shouldn't be too difficult, especially with OpenGL timer queries (or whatever the Direct3D equivalent).

So let's get to some possible improvements to your situation. It seems from your link and your explanations that you're using a classic geometry shader based approach for wide line rendereing, converting each line into a screen-aligned quad (or two triangles), computing the vertex positions based on the line width you want to achieve. I recently built something similar, too and there's a few ways you can improve the quality.

As you said yourself, since you're working on rather arbitrary rounded shapes it's probably best to let the fragment shader figure out the exact pixel-shape rather than trying to approximate the circular middle region in the geometry shader. Though, that idea is not completely off-the table and might give for a nice adjustable quality-speed trade-off by adjusting the number of vertices of that circle-part. But it also, together with your current approach, has the problem that it actually needs to know the line's neighbouring lines in order to compute the middle section, which makes it difficult to just put into an existing shader-agnostic rendering system that just draws all the lines as vertex pairs (not to speak of line strips/loops), since you need to add the adjacency information for each line.
If you just want to let the fragmet shader do the job using simple alpha testing, you generate a screen-aligned rectangle that's a little larger than the line, i.e. it spans half the width across the endpoints and thus covers the circular end caps, too. In the fragment shader you just check each fragment's screen-space distance to the nearest point on the actual line (which you transfer from the geometry shader to the fragment shader as a varying). And whenever that distance is larger than the line width (or half the line width, to be precise), you discard the fragment. This way you will basically render a rounded box for each line.

To be precise you render your line as three quad-sections (i.e. 6 triangles in a single strip). The middle section between the ednpoints as one rectangle and the two end-caps as rectangles spanning from the endpoint half the width in the direction of the line. You do this separation to give the rightmost 4 vertices their respective endpoint as varying value and the leftmost one their respective endpoint. This way you get smooth interpolation of the line point across the middle section and clamping in the cap sections.

This distance computation might also be used to antialias the lines, i.e. by using alpha blending (or multisampling with alpha to coverage) and letting alpha go from 1 to 0 along the outermost ~1-pixel thick border of the line.

But this approach has the disadvantage of drawing the uneccessary grey area from your above picture, twice even. This not only causes possibly slowing overdraw (and the explicit discarding inhibits early z-testing), it will also interfere with any possible alpha-based antialiasing as described above.
A solution to mitigate this could be to actually just draw the completely uncapped lines as rectangles from one endpoint to the other (i.e. like your very first picture). And then drawing the connection points (i.e. the line's vertices) as additional circular large points (i.e. using a similar technique of transforming points into screen-aligned quads with the geometry shader and testing for the distance to the actual point center in the fragment shader to get pixel-perfect circles).

This has the advantage that you can actually use stencil testing to only draw the connective sections in the regions where you haven't drawn other parts of the line set before and thus only the violet area, which will tremendously reduce overdraw (at the cost of more complicated rendering and per-fragment operations). This gives very nice results with an alpha-based antialiasing approach as described above (there are however still some minor areas where there's some tiny artefacts).

This stencil testing approach might as well be used on the earlier approach where you draw the line section together with its end-caps when you don't perform any kind of antialiasing or blending, but I think to remember that it caused more artefacts than the separate rendering if you do. I haven't spend too many thoughts on this right now, though, and it might be worth some more investigation. However the seperate solution also has the advantage of requiring the expensive fragment test and possible discard only in the actual connective regions, the line middle sections can be rendered with a straight-forward fragment shader.

Those are just a few thoughts that I happened to ponder on just recently and deemed worth sharing. As said, if they are appropriate for your use-case has to be evaluated, possibly with some timing and quality measurements.


Why there's no z-fighting for coincident planes if the camera look perpendicular at them? There's a similar question Why do identical polygons not z-fight in OpenGL?
 but I believe I have a different setup.

As can be seen in the picture the vertices for the planes are different and on the left side, there's a clearly visible artifact. On the right side, there's no z-fighting even in movement. Why?
Update: the white plane draw call submitted first. The depth compare operation is VK_COMPARE_OP_LESS_OR_EQUAL.

Even if two triangles aren't coincident...
Z fighting happens because of floating point quantization, this results from all the math the shader does to compute vertex positions such as: $glPosition = Projection*View*Model*vertex\_position$
When the vertices are all the same distance and angle from the camera then all the floating points values are effectively the same and the tiny error that crops up is so close that the Z fighting appears to go away.
On the other hand if you change the angle of the camera even a little bit then all the floating point values for each vertex are now slightly different and so is the quantization. This results in the z fighting that you are seeing.
But keep moving the camera back, and the results for the perfectly aligned camera will eventually begin to exhibit the same behavior. The camera position is also a floating point value and subject to floating point error so no matter how perfectly the camera is aligned, it will still, eventually, have enough error to cause the same issue.


Manually fetching 8 neighboring texels from 3D texture greatly decreases performance I'm implementing manual interpolation between texels in a 3D texture to be able to discard some of them when needed. Compared to hardware interpolation, this process takes a lot of time. If I simply leverage hardware interpolation using texture function, rendering takes ~5ms, but with manual fetch using texelFetch it rises to ~13ms (using texture to fetch 8 texels manually increases frame time even further - to ~35ms). The multiplication/summing part of the manual lerp does't matter much - I've tried fetching same texel manually 8 times and the performance is close to hardware interpolation. I thought fetching neighboring texels should give me good cache behavior, but maybe that only applies to 2D textures because neighboring layers of 3D tex are not so cache friendly? Don't know what to think here.. It would be nice if there was some way to use hardware interpolation but discard some texels from the process. The absence of a textureGather function taking a 3D texture also makes me think I'm missing something here.

What's probably happening is that you're doing indirect texture fetches. Are you doing something like this:

vec4 a = texture3D(sampler, coord);
vec4 b = texture3D(sampler, coord + vec4(0, 0, 1));
vec4 c = texture3D(sampler, coord + vec4(0, 1, 0));
vec4 d = texture4D(sampler, coord + vec4(1, 0, 0));
//...etc.

If so, that can slow you down. What you can do instead is to pass the texture coordinates for each sampler call into your vertex shader from your CPU code. You'd set it up as if you were doing multi-texturing, but all the texture units would point to the same texture, just sampled at different coordinates.

Taking the 2D case, instead of passing in a single texture coordinate like this:

float2 texCoords[] = {
    { 0, 0 },
    { width, 0 },
    { width, height },
    { 0, height }
};

You'd pass in 4 of them:

float2 texCoords[] = {
    // texture 0 coords
    { 0, 0 },
    { width, 0 },
    { width, height },
    { 0, height },

    // texture 1 coords
    { 1, 0 },
    { width + 1, 0 },
    { width + 1, height },
    { 1, height },

    // Texture 2 coords
    { 0, 1 },
    { width, 1 },
    { width, height + 1 },
    { 0, height + 1 },

    // Texture 3 coords
    { 1, 1 },
    { width + 1, 1 },
    { width + 1, height + 1 },
    { 1, height + 1 }
};

In your fragment shader, you'd do something like this:

vec4 pixel0 = texture2D(sampler0, coord0);
vec4 pixel1 = texture2D(sampler1, coord1);
vec4 pixel2 = texture2D(sampler2, coord2);
vec4 pixel3 = texture2D(sampler3, coord3);

Now you have the 4 pixels needed for interpolation, but there was no fragment shader math done to get the texture coords, so there are no indirect look-ups which are slower than direct look-ups.


How do I efficiently calculate the distance to the edge of a shape? I'm trying to implement something like an "inner glow".
That is, I have an image which defines the shape of an object, and then
I want to create overlay that is a gradient that goes from the edge of the shape
and into the shape itself.
In programs like Photoshop and Krita, this is called an inner glow.
If I could first produce a texture which contained values that represented distance
along the inward travel path from the edge, the rest would be easy.
I want this to happen in a shader, but any general insights about the problem would be helpful, I guess.
One idea that I was working with, was to just blur the outline, and call the result shades around the edges the "distance", but this might be an unnecessarily expensive operation on the GPU, maybe not, I don't know.

A naive approach would be to check all distances to all other pixels, which comes out to something like 256⁴ square root calculations for a 256x256 pixel image. I believe that's not optimal. Also I don't know if that's even possible in a shader.

A texture that stores distance from the edge of the shape, like you described, is called a "distance field" (you'll find lots of results if you google that phrase). Distance fields can be calculated efficiently on the GPU using the jump flooding algorithm (JFA).
This is a multi-pass algorithm that works by sampling pixels at a distance of first ½ the image size, then ¼, then ⅛, and so on down to single pixels. At the end it comes out with a texture with the shortest distance from each pixel to any of the initial "seed" pixels you put in (which would be along the edge of your shape). For more details see the paper, or also check out this blog post by Alan Wolfe which also has some shadertoys with example implementations.
Another article which may interest you is The Quest For Very Wide Outlines by Ben Golus, which addresses this exact issue, discusses a wide variety of GPU implementations (and settles on JFA), and addresses some of the implementation details such as using JFA on an antialiased edge.


What is the practical difference between 3D Graphics Engine, 3D Game Engine, 3D ........? What is the practical difference between 3D Graphics Engine (OGRE), 3D Game Engine (Quake), 3D Software Rendering Engine, 3D Graphics API (OpenGL/DirectX)?

Is OpenGL/DirectX a 3D Graphics API or a 3D Software Rendering Engine?

If I want to start learning 3D computer graphics and geometric modeling, which one do I need?

At the base, rendering APIs like OpenGL or Direct3D provide a unified interface to communicate with graphics adapters at a low level. Differences in hardware are hidden behind the abstraction which simplifies development a lot.

The low-level APIs provide basic functionality, which gives you total control but leaves you without a lot of desired features. 3D Rendering Frameworks like OGRE or Irrlicht introduce the more understandable concept of 3D objects that can be moved in space (without having to worry about transformation matrices most of the time), introduce light sources and algorithms for shadow casting (e.g. shadow mapping), loading of common mesh formats, and the list goes on. I call this a framework, because you still need to develop the application yourself using actual code, while an engine provides you with an application to develop your software in. This is just my personal distinction and both terms are used differently in a variety of contexts.

In a 3D Authoring / Rendering Software like Blender or Autodesk Maya (offline) or 3DExcite DeltaGen (real-time) you can load, display or create meshes using the built-in functionality. These applications target the creation of 3D data, images or movies, and usually you can't create interactive applications easily with it. Be aware that some 3D Authoring have modules specifically targeting this, e.g. Blender and the integrated Blender Game Engine.

A 3D Game Engine finally provides you with all the tools to make it easy for you to develop a game. You will usually be provided with a level editor, a material editor and maybe a visible scripting language (like Unreal's Blueprint or earlier Kismet) to implement dynamic behavior and interactivity. You don't have to worry about rendering algorithms, because they are already implemented for you. Your engine of choice may for example feature SSAO and cascaded shadow maps for dynamic objects and a solution to bake global illumination for static geometry. Also, a game engine includes way more than only rendering, e.g. artificial intelligence, sound, loading of assets, exporting your content into a ready-to-install package etc.

As you can see, each software package generally puts another layer of abstraction on, ideally making your life simpler. Each has its own purpose and allows you to do different things. Note that said terms are very broad and people have different opinions and sometimes even debates about where one category ends and the next one begins. My advice is to not lose sleep over it.


Is this Russian roulette code correct? I'm using www.scratchapixel.com among other resources to help me learn how to implement a renderer. I am looking at the following code from this page where a packet of photons moving through a material is being considered. For each photon packet, the weight $w$ is initialised to $1$. $dw$ is the probability of absorption.

The confusing part to me is when $dw$ is subtracted from $w$. I can see this would make sense when the packet has full-weight of $1$ because $1 - dw$ is the unabsorbed proportion of photons. E.g. if probability of absorption is $33\%$ then $w = 1 - 0.33 = 0.67$ and $67\%$ of the photons remain. I can't see how this makes sense on subsequent iterations. For example, on iteration two, $w = 0.67 - 0.33 = 0.34$ so half the photons are absorbed on this iteration, not a third.

int photons = 10000;
...
int m = 5; // there's 1 over 6 chances for the packet to be absorbed
for (int i = 0; i < nphotons; ++i) {
    float w = 1; // set the weight to 1
    Vec3f P(0, 0, 0);
    Vec3f V(0, 0, 1);
    while (1) {
        ...
        float dw = sigma_a / sigma_t;
        absorption += dw;
        w -= dw;
        if (w < 0.001) { // perform russian roulette if weight is small
            if (drand48() < 1.0 / m) {
                break; // we kill the packet
            }
            else
                w *= m; // adjust weight
        }
    }
}

I think you're right and the subtraction is a mistake. The code should rather be multiplying the fraction of photons not absorbed into the weight. Something like:

float fraction_absorbed = sigma_a / sigma_t;
absorption += w * fraction_absorbed;
w *= (1.0f - fraction_absorbed);

This makes absorption the total fraction of photons absorbed so far, and w the fraction of photons remaining.


Multiple samples per pixel and texture blurring As far as I know path tracers (or super-sampling antialiasing) typically calculate the final color of a single pixel by averaging the result of all samples taken inside that pixel. This gives a nice anti-aliasing effect on the edges, however has the side effect of slightly blurring textures, because they most certainly do not have enough samples for each pixel, so samples have to be guessed with some kind of filter (like bilinear). From a mathematical point of view, averaging those samples applies a box low-pass filter to the reconstructed texture. This causes blurring.

How do path traced renderers/SSAA usually deal with this blur? Is there no way around it?

In signal processing, it is well-understood that you cannot accurately reproduce an analog signal of higher frequency than half the frequency of your digital sampling rate. That's just how the math works. Various aspects of rendering are just forms of signal processing, so this applies here too.

So you're going to get some kind of artifact. You have a choice: aliasing or noise (aka: blur). You're always going to have some of these. Aliasing artifacts are usually very noticeable and considered quite distracting. Human vision tends to focus on motion, and aliasing artifacts almost always create motion where none existed (especially when animating). Human vision is generally much more tolerant of noise, as noise patterns don't appear to move nearly as much. Plus, our eyes have their own anti-aliasing filters that impart some noise to what we see, so we're somewhat used to it.

Broadly speaking, you deal with it by not dealing with it, because it's preferable to the alternative. Yes, you can reduce the effect of noise with better anti-aliasing mechanisms, but broadly speaking, you're still going to have noise.


Efficient rendering with many light sources To render a scene with a single light source using phong shading, one can calculate the final color of each fragment passed into the fragment shader based on the ambient/diffuse/specular components of both the material and the light source.

This can be easily extended to accommodate for multiple light sources by adding together the results of applying each individual light source onto the fragment as such:

final_color = (0, 0, 0, 1)
for each light:
    final_color += apply_light(material, light)
final_color = clamp(final_color, (0,0,0,1), (1,1,1,1))

However, with a very large number of light sources, this process is quite slow; with N lights, this approach requires calculations for phong shading to be done N times per fragment.

Is there a better approach to rendering scenes with a very large number of light sources (hundreds, thousands, etc.)?

Yes, but you need a paradigm shift.

What you are accustomed is called forward rendering. You submit your geometry and then you proceed immediately with the shading pass. In the basic forward rendering you can either loop inside the shader for each light or perform one pass per light and blend the result together (with additive blending).

But things have evolved quite a lot. Enters: Deferred Rendering

Now that are so many variants that describe them all in details will take way more than acceptable for an answer here. So here I am just going to describe the gist of Deferred shading, there are plenty of other resources that you can easily find using google, hopefully after reading this you'll have the right keywords to find what you need.

The basic idea is to defer the shading to after down the pipeline. You have two main steps:

Render your geometry and all the information needed for shading into multiple render targets. This means that typically in a basic implementation you would have a depth buffer, a buffer containing the normals of your geometry and the albedo colour. You'll soon find that you need other information of the materials (e.g. roughness, "metallic" factor etc.).

This image from wikipedia shows three buffer (color, depth and normals)

Again, the amount, type and content of the buffers used varies quite a lot among different projects. You will find the set of buffers with the name of GBuffers.

After this is the time of applying the actual lighting. During the lighting pass for each light you want to draw a light volume that depends on the type of light:

For a directional light you render a full screen quad.
For a point light you render a sphere where the radius is based on the attenuation of your point light.
For a spot-light you render a cone which dimensions again depend on the characteristics of your light.

In the pixel shader of this pass you pass your GBuffers and perform your lighting and shading using the information in them. This way you process only the pixels affected by each of the lights having a sensible speed-up if compared to the classical forward rendering.

It has also various disadvantages, most notably the handling of transparent  objects and higher consumption of bandwidth and video memory. But also it is trickier to handle various models for materials.

You have other side-advantages (as having lots of info ready for post-processing) and is also pretty easy to implement. But this is not the coolest thing around for lots of lights anymore.

Newer techniques are for example Tiled rendering ones. The main idea of those is to subdivide the scene in screen space "tiles" and assign to each tile the lights affecting it. This exists both in a deferred and forward fashion.  These techniques lead to some problems when you have various depth discontinuities in a tile, but is generally faster than the classical deferred and it solves various problems of it. For example, among the advantages, with tiled deferred you read the GBuffers once per lit fragment and pixels in the same tile coherently process the same lights.

Further evolution on this side is the Clustered shading which is conceptually similar to the tiled based approaches, having instead of screen space tiles, clusters with a 3D extent. This method handles better the depth discontinuities problem and generally performs better than the tiled methods.

IMPORTANT NOTE: I have described the basics of the deferred shading. There are multiple variations, optimizations and improvements around so I urge you to experiment with a simple version and then do some researches on other techniques such the one I mentioned above.


Why isn't the clear color visible on the screen between frames? I am told that in computer graphics, the screen refreshes in number of times(in this scenario let say that it renders 60 times per second). The screen is cleared and drawn to 60 times per second.

My question is how can I see the Image being shown 60 times per second yet I cannot see the Clear Color which is also being displayed 60 times per second too?
Or does this have to do with how our eyes see or how our brain interprets things we see?

You're confusing the screen (the physical display) with the framebuffer (the memory which gets read out by the display controller). The framebuffer might get cleared at the start of each frame period, but that doesn't change the display. The order of events is as follows:

At the start of a new frame, glClear or equivalent sets the framebuffer to the clear colour.
glDraw or equivalent calls draw more stuff on top of the clear colour.
eglSwapBuffers or equivalent queues the finished frame to be drawn on the display.
When the display is ready, this process is followed. It reads out the contents of the framebuffer and shows them on the physical screen.

The framebuffer that's being drawn into is never shown on the screen; only when it's finished can it be shown. Double- or triple-buffering gives you multiple copies of the framebuffer, so the application can be using one while the display is using the other. It's nothing to do with your eyes or brain.

It's actually a lot more complicated than this, because most GL implementations today are not immediate-mode, so none of the real work happens when you call the functions named above. In fact, clearing the buffer might not even happen at all: if every pixel is drawn to anyway, there's no need to. Back when clearing the framebuffer took significant time, games would go faster by not calling glClear. Today it's more important to call glClear because it lets the implementation know when the start of the new frame is, but it will probably do something smarter than just writing to every pixel.


Does this 3D rendering method exist? How is it called? Is there any C++ algorithm for it? I've been thinking about making my own 3D rendering engine in C++, but I don't know much about math that's required to do it. I'm not even sure if it's possible or if it even exists or how it's called.

It goes like this. Let's imagine a 3D body that has 6 vertices and 8 faces. It's like a pyramid on top of an upside-down pyramid. Each face is a triangle, but let's imagine that every line between the triangle points is a three-dimensional bezier curve. That would make each face look like a tent. If we do that, then we get a sphere. Now, imagine if instead of rendering that sphere by subdividing the tents into triangles we render the tents exactly how they are and imagine if we don't render raster graphics, but rather vector graphics in PostScript/Flash/ActionScript/GhostScript format in ASCII that can be opened by Notepad and rendered as a semi-transparent PNG image at any resolution without losing quality.

When rendering 3D triangles, matrix operations are used if I'm not mistaken. There's a matrix with some values and then a determinant is calculated. But these values are constants. What if the values depend on a formula of the bezier curve that's in 3D space? How can we stuff the formula into the matrix? And a better question; how can we stuff that into an algorithm or into OpenGL or into a language that a graphics card can process instead of the CPU? Does such a formula, matrix or an algorithm exist?

Now, I'm not thinking about rendering simple spheres, but rendering something like this: https://www.blendswap.com/blends/view/24878
I'm thinking about making a lightweight renderer in assembly for an ESP32 SoC or some other Arduino microcontroller/processor/SoC which would render all that onto an LCD or a PAL/NTSC TV via a composite AV jack, but that's another topic.

My question is whether or not such a rendering method exists, how it's named, is it free to use, is it patented and is there any permissive-licensed C++ code for it (MIT, zlib/png, Apache, BSD). Please let me know.

Yes, such rendering exists. It is called a parametric surface and there are several methods you can render such entities. In this case you are talking of bezier patches.

Turn the object into enough triangles. The triangilation is called tesselation or meshing depending on what literature you read.
There are lots of applications that do this allready. Most cad applications work this way and many graphics apps like blender can work
this way.

raytrace or raymarch the surface in a shader.

Microdice the surface

etc.

The reason this is not often done is that the on line tesselation is for most intent and purposes just a hindrance to asset developpers. You get the same with subdivision schemes and its just easier. The primary reason to drag the bezier (or nurbs, or t-splines...) is that unless you need to find better intersection of underlying primitives then it is often just a waste of resources. Although subdividing your mesh  via a subdivision surface is allready more useful


How does everyone count "Rays per Second"? Recently I'm developing a Monte-Carlo path tracer. To measure my tracer's performance, I decided to implement a simple mechanism to count how many rays can it trace in a second. Then here comes the problem, there are two ways to define a single ray:

  A complete ray, i.e., starting from camera and bouncing around the scene until it terminates.
  The ray originated from each intersection point.

Although many people use "Rays per Second" to measure ray/path tracer's performance, how they recognize "rays" is unclear to me. Perhaps anybody knows?

I've seen both, unfortunately. I'm not fond of rays per second as meaning exclusively primary rays and I'd suggest "paths per second" or better yet "samples per second" instead. "Complete ray" is not a common term: a ray is a (potentially unbounded) line segment and a sequence of rays is a path.

Rays per second in your second sense of total ray casts is not well-specified for a path tracer: do shadow rays count, for instance? It's a useful metric for evaluating an acceleration structure or intersection testing framework (i.e. Embree or OptiX) but I'd avoid it for a renderer.

Finally, be aware that samples per second still isn't a great metric of actual performance since sample quality will vary wildly depending on implementation details. It's probably the best thing you can do starting out though, as the better solutions involve fairly complex variance estimates.


Non jagged line algorithm for javascript canvas I'm trying to draw a line between two specified points, by updating values in an array which is heightxWidth with each element a set of four bytes (r,g,b,a). In my naive implementation I get the top line which reaches a point with jagged steps in the top line shown below. My question is, how do I accomplish the standard canvas line effect without jumps seen here as the lower line shown below?

The problem you are seeing, i.e. "jaggies" or "staircasing", is an example of the more general problem known as "aliasing" and, in the graphics field, the term you want to search for is "Antialiasing".

Aliasing occurs when you undersample a signal. If a signal contains frequencies at or above the Nyquist Frequency, which is 1/2 the sampling frequency, those higher frequencies will come back "in disguise" i.e. "under an alias".

The example I often give to new starters to explain the problem is:

  Imagine you've started at a new job and work solidly, 9 to 5, except for a 1hr break from 12 to 1.
  The boss, however, regularly walks around the building at 12:30 and never sees you at your desk. He therefore concludes you are lazy and so don't get a pay rise

That is an example of aliasing. The boss needs to sample at a (much) higher rate. (Randomised/stratified sampling would also help :-) )

In your line drawing example, I expect you are taking one sample per pixel, but if you do a Fourier analysis of a line, you will see it contains an infinite spread of frequencies. It's these higher frequencies (plus the reconstruction of the sampled results - which I've ignored) that can lead to the jaggy results.

The ideal solution is to low pass filter your incoming signal before sampling it, but that's 'nontrivial' with geometric data.

There are, however, multiple practical approaches to mitigating the problem, including supersampling (SSAA) and multispampling (MSAA), but in your case, perhaps something like Wu's method for antialiased line drawing will suffice.


Direct illumination, specular surfaces, and MIS Multiple Importance Sampling (MIS) is a technique used by Veach's VCM technique to balance responses of different surfaces to different types of importance sampling for direct illumination. My understanding is that specular surfaces are best lit without sampling the light at all; instead, you pull a direction from the BRDF and hope that it hits the light source. Diffuse surfaces "spread" light more broadly over their domain, so firing rays towards random points on the area covered by the light produces good results without the wastage produced by sampling over the BRDF.

I'm still unsure about a few things, though:

Veach's demonstration images show flat specular surfaces eventually converging with light sampling; how is this possible? All rays fired towards random points on the light will reach the light (assuming no obstructions), so shouldn't the specular surfaces wash out under all the assumed illumination?
"Multiple Importance Sampling" implies that tracing gather rays towards the light is an importance-sampled technique (which makes sense, it's very difficult to have invalid rays if all rays approximate the direction of light relative to the surface); does this mean I need to be dividing my direct gathers by a PDF? Given that the "direction" of the light could be anywhere on the unit sphere surrounding the lit point, should the PDF be $4\pi$?

Specular surfaces which use MIS are not perfectly specular like a mirror. They have a small amount of blur, otherwise there is indeed no point in sampling the light as all the samples will evaluate the BRDF as 0. In fact, you would only need to trace a single reflection ray.

A small amount of blur means that a given camera ray will see a small area of the lightsource (assuming it hits where the specular highlight is visible). Thus, a small fraction of the light samples will get a BRDF which is not 0 so it will eventually converge. The less blur, the more samples it will take.

You are correct that such a case will converge faster if you sample only the BRDF. Likewise, a very diffuse BRDF will usually converge faster if you sample only the light. But this is not true if your light is very large and your surface very close to it1. Where MIS shines is that it handles those cases as well as the BRDFs which are neither very diffuse nor very specular.

If you choose to ignore it, what you have to do is a simple integral. There are a few ways to go about it but if you start from MIS equations, just use 0 as the PDF of the BRDF since you're not generating those samples. And be prepared for some cases which won't converge well.

The assumption is that the light samples all have a similar contribution to the final result. Even with very good sampling of the light, this must assume that the BRDF will be similar for all light samples. At least I don't know of any direct light sampling formula which takes the cosine term of the diffuse surface into account. Pushing this to the extreme, you can build an example where most of the light is behind the surface, so most of the light samples will contribute nothing at all to the final result. BRDF samples are obviously sent only on the front side of the surface so they might converge faster, even if only a small % of them hit the light.


Combination of Triangles and Quads in a mesh Is it topological correct to combine triangles with quads to form a mesh. If not, why?

Topologically correct is a very vague.

I believe you think of Polygon-Meshes when you say meshes, which is to represent the surface of the object by a patch of polygons, in most cases triangles or quads. There are other options to model meshes, one idea would be to use signed distance functions as for example in constructive solid geometry.

So back to the question. Is it correct to mix the type of polygon? Yes. There is, mathematically and theoretically nothing stopping you from doing it.

The only real 'correctness' criteria that comes to my mind is the one of the mesh being a manifold. This is in (overly) simplficated the property that the mesh has a defined outside and inside and no strange places where you can't decide. (This happens for example if the mesh intersects with itself, but it get's freaky). And having different types of primitive does not violate the manifoldness of the mesh.

But most algorithms can actually deal with non-manifolds. There is no problem in rasterizing or raycasting a non-manifold as these algorithms process each primitive separately.

The more interesting question here is: Why DONT we mix the type of primitive?

And here the answer is very simple: Efficiency. You want to design an algorithm that handles your mesh as efficient as possible, and so you assume it's only one type of primitive. And at some point in time, because triangle-meshes are really handy, hardware and hardware-api began to adpot mainly to processing triangles. And now they are good at that and even other primitive types get split into triangles since splitting and rendering triangles is still faster than rendering the polygons directly.

So in short: You can mix polygon types, but if you want to be fast, use only triangles since they are simple and widely used.


Get screen size and then resize image accordingly I'm a programmer working in Java and I'm not sure if this is the right place to ask this kind of question, here it goes:

In my program, I need to show an image (.jpeg .png etc.) based on the screen size, this image is used to compare to real size (physical) of an actual label which is later printed on a specific machine.

The issue is, for example on my machine I have the laptop screen and a '27 monitor they are both using the same resolution 1920x1080 but as you imagine if you resize any window/program/image it looks smaller on the laptop as the physical screen size is different although it has the same resolution (I imagine is because the physical size of the pixel is smaller than the ones on my monitor)

Now what I would need to do is somehow always show the image on it's corect physical size (as it were printed), on any type of screen.

Regarding to programming I could easily get the screens size and resize the image but, this is because the OS gives the size (it's set up size) back to the program, yet even if I have the correct size the physical screen could display the image smaller for example, as the screen's pixels are smaller.

Any ideas where to start with this?
How can I know the real physical screen size?
This seems to be more a hardware issue than graphics..

The OS does not, in many or most cases, know the physical screen size! Often the only thing you can glean is the make and model, sometimes not even that. So if you do not want to have a database then the best you can do is ask your user.

There are also some interesting corner cases. What do you do if the monitor is in fact a projector? What do you do in multi monitor setups? What if the image is halfway between 2 monitors? What do you do if the operator is using pixel zoom for accessibility reasons? What do you do if the image is shown on 2 monitors at the same time? What do you do if the monitor is huge, like one used in a conference room and your image turns out to be very small in comparison (like 4 by 4 pixels)? ...


Why do adjacent triangles never overlap when rasterized? Apologies for the following noob question, it's just I can't find a satisfactory answer anywhere...

When I render a simple axis-aligned quad made up of two triangles, I don't understand why there's never any visible artefacts on the diagonal edge where those triangles meet. Some of those pixels must be in both triangles, right? So if my fragshader always emits a partially-transparent color - 50% black say - then shouldn't there be a visible dark diagonal line in the resulting quad?

Obviously it's great that this isn't a real problem that has to be coded around, I just don't understand why it isn't. What GPU magic is it that makes this always work out?

This is the same problem as discussed in What are sample gaps during scan conversion?

Briefly, rasterisation - at least with the majority of hardware systems - tests at a single 'infinitesimal' point for each pixel to determine if that pixel is inside a given triangle.

If the sample point lies exactly on an edge, then additional tie-breaking rules, often referred to as "fill rules", come in to play. Typically these will be something equivalent to, say, only consider the edge points on the top/left** edges of a triangle to be "inside".

**Other rules are available ;-)


How do fragment shader invocations subgroups map onto triangles? Should I expect that every invocation/fragment within a subgroup, when using something like subgroupBroadcast(), will belong to the same triangle?

Fragment shader subgroups are not required to be restricted to a single primitive (and such a restriction would negatively impact performance). So subgroup broadcasting can cross primitive boundaries.
Note that the most recent version of the Vulkan standard states that invocations in the same quad scopes are required to be within the same primitive scope.


Total emitted power of diffuse area light I am reading the book Physically Based Rendering (Pharr, Humphreys). In the chapter on lights, they talk about approximating the total emitted power of different kinds of lights. For example, a point light's total power is intensity * 4 * pi. Here 4pi represents a solid angle over the entire sphere.
This makes sense to me because intensity * solid angle = power (or radiant flux if you will). You can see this by the units as well. Intensity is W/sr and solid angle is sr, so W/sr * sr = W and power is measured in watts. It checks out.

However, I don't understand the corresponding calculation for the DiffuseAreaLight. From my understanding of the book they calculate the total power emitted from a diffuse area light as emitted radiance * area * pi.
Since the unit of radiance is W/(sr*m^2) multiplying area gives W/sr. This makes me think the pi factor represents solid angle - but why only 1pi? I would have guessed 2pi since each point on the area light would radiate in a complete hemisphere (corresponding to 2pi steradians).

You can find the actual code mentioned in the book here.

What am I misunderstanding? Why does total emitted power = emitted radiance * area * pi for diffuse area lights make sense?

Radiance (in terms of flux) has the following definition:

$L_o = \frac{\mathrm{d}\Phi}{\mathrm{d}\omega^\perp\mathrm{d}A}
= \frac{\mathrm{d}\Phi}{\cos{\theta} \mathrm{d}\omega \mathrm{d}A}$.

Thus in order to get the total emitted power we need to integrate over the area of the light and we need to integrate over the projected solid angle of the hemisphere (in the direction of the surface normal). The radiance for the area lights in pbrt is constant. This gives us:

$\Phi = \int_A \int_{H^2(\mathbf{n})} L_o\mathrm{d}\omega^\perp \mathrm{d}A =
\int_A \int_{H^2(\mathbf{n})} L_o \cos\theta \mathrm{d}\omega \mathrm{d}A =
\int_A L_o \int_0^{2\pi} \int_0^{\pi/2} \sin\theta \cos\theta \mathrm{d}\theta \mathrm{d}\phi \mathrm{d}A =
\int_A L_o \pi \mathrm{d}A =
A \cdot L_o \cdot \pi$


How to handle specular highlights with reflection / total internal reflection in a ray tracer? Let's say that I have a raytracer that is rendering a scene that includes a refractive object.

When a ray hits the refractive object, I know that I can use the fresnel equation (Schlick's approximation in my case) as well as the object's base reflectivity amount to figure out how much light is reflected or transmitted at that point on the surface.

At this point, I can get the reflected color using a reflection ray (possibly recursively), an environment map, and also calculate a specular highlight for the light source(s) I have.  I sum whatever reflection colors I have, multiplied by the amount of reflection from the fresnel equation.

Next, I calculate what amount of light comes in due to refraction by following the refracted ray (recursively) and multiplying that value by 1.0-reflectionAmount.

That makes sense to me so far, but where I get a little confused is for the refraction rays.

When I follow the refracted ray into the object and hit the back side of the surface the first time, i once again use the fresnel equations to calculate the reflected (internally) and refracted multipliers for the light, taking into account the possibility of total internal refraction (assuming here that the inside of the object has a higher IOR than outside of the object, an ok assumption for my case).

But, at this point, do i apply a specular highlight to the surface that i hit from the inside?

On one hand it's a surface so i feel like i should, but which ray should i use for the lighting - the refracted ray or the reflected ray, or both?

If using the refracted ray, it feels wrong to use the direction to the light source for the specular light, and feels like i should shoot the refracted ray to the surface of the object and see what the specular highlight would look like from there and use that - but do so recursively, using the fresnel equation again at each step.

Can anyone explain a decent way to handle a specular highlight in a ray tracer that supports refraction and total internal reflection?

Since this is a ray tracer, not a path tracer, instead of being limited to physically accurate solutions, I'm looking for any plausible solutions, but of course, physically inspired solutions are likely to be better!

Thanks!

But, at this point, do i apply a specular highlight to the surface that i hit from the inside?

Unless you have a light source inside your object, there's no point in doing lighting on the inside surface as you'll hit the object again before you reach the light or environment.

If you want to support a light in the object, then by all means do your lighting but make sure to trace rays to check that you actually reach the lights without exiting the object.

Otherwise, that lighting should come from your (internal) reflected ray which will eventually hit the surface again, be refracted and get light from the outside.

  which ray should i use for the lighting - the refracted ray or the reflected ray, or both?

I think the confusion here is that when you write of "calculate a specular highlight" there is an assumed BRDF which may not make much sense for refraction. At least probably not without a few tweaks. What you really need is a BTDF (bidirectional transmittance distribution function).

If you want to keep it simple, I think you could adapt a Blinn-Phong model for refraction by using the half vector of equation (16) in Microfacet Models for Refraction through Rough Surfaces.


OpenGL compatibility, naming conventions and ARB vs EXT I thought I had formed a general understanding of how OpenGL naming conventions and extensions worked, until I stumbled upon a case that confused me.

Here's my understanding so far:

No suffix -- e.g. glGenBuffers(). This function is a part of the core profile. The wiki page tells me that this was added to the core profile starting from version 1.5.

ARB -- e.g. glGenBuffersARB(). This function is part of the standardized GL_ARB_vertex_buffer_object extension. The spec of this extension clearly declares GenBuffersARB() in the "New Procedures and Functions" section. The "Dependencies" section tells me that I can potentially access this from a 1.4+ context, if the hardware supports the extension.

EXT -- These are vendor-specific extensions and functions that only some vendors may support. Vertex buffer object doesn't seem to have an EXT extension in the registry.

Here's where my understanding breaks down:

glGenFramebuffers, as the wiki shows, was added to the core in 3.0.

Now I want to access the frame buffer features at a lower core profile version than 3.0. So I want to use it as an extension. The spec registry tells me that there are two available extensions - ARB and EXT.

Question 1 -- If an ARB extension exists, why does an EXT extension exist? Wouldn't you always choose the standardized one over the vendor-specific one?

A look at the ARB spec in the "New procedures and functions" section tells me that the extension defines the GenRenderbuffers() function. No ARB suffix this time. GLEW doesn't have a function prototype for glGenRenderbuffersARB() at all. Weird.

The EXT spec does however have a GenRenderbuffersEXT() function in the new functions section, and GLEW also hasglGenRenderbuffersEXT().

Question 2 -- Why no ARB suffix if there's an EXT suffix? How does this work for ARB, given that the names of the ARB function and the core function are the same?

Question 3 -- I ultimately want Framebuffer features from a 1.4 profile. Which extension and which function-set should I use so that I get maximum hardware compatibility coverage?

Question 1 -- Usually the EXT version comes first as a collaboration between two or more vendors. ARB extensions require more discussion among voting members of Khronos and can have changes from the EXT version before getting approved. See the GL_ARB_direct_state_access extension which has many changes compared to GL_EXT_direct_state_access.

Question 2 --
The Issues section of the GL_ARB_framebuffer_object extension states why the functions do not have ARB suffixes:

  (8) Why don't the new tokens and entry points in this extension have suffixes like other ARB extensions?

   RESOLVED: Unlike most ARB extensions, this is a strict subset of
   functionality already approved in OpenGL 3.0. This extension
   exists only to support that functionality on older hardware that
   cannot implement a full OpenGL 3.0 driver. Since there are no
   possible behavior changes between the ARB extension and core
   features, source code compatibility is improved by not using
   suffixes on the extension.

Question 3 --
If you want to use framebuffer objects on a context where the GL version is less than 3.0 you need to look at the extension string:

If GL_ARB_framebuffer_object is supported, use the non ARB functions.
If only GL_EXT_framebuffer_object is supported, use the EXT functions.
If neither extension are supported, you need to fall back to OS level offscreen rendering like pbuffers.


How much does driver quality affect Vulkan and DX12 compared to OpenGL and DX11? Considering the fact that Vulkan and DX12 delegate most of the memory management / work scheduling tasks to the application itself, how much do drivers affect performance?

To put my question into context, people on the internet talk a lot about NVIDIA drivers being more finely tuned and optimized (especially for the most popular games) compared to AMD drivers. Now that DX12/Vulkan are becoming more common, is it safe to assume that AMD video cards will become more competitive even if they have less optimized drivers?

TL;DR:  There's some room for optimization in Vulkan drivers, but not nearly to the degree that there is in GL / DX11 drivers.

  how much do drivers affect performance?

It depends a great deal on how the application is written and the hardware.  Specifically it depends on whether you're using features where there are large opportunities for optimization on the driver side.

For example, if an application is written to take full advantage of renderpass subpasses and properly specifies the subpass dependencies and properly sets up transient attachments for intermediate work, then the difference between an extremely naive Vulkan driver and a well written one could easily be an order of magnitude.  But at the same time, this is basically the difference between naive client code and well written code both running on a well written driver.

  people on the internet talk a lot about NVIDIA drivers being more finely tuned and optimized (especially for the most popular games)

This isn't just a matter of driver tuning.  It's arguably a form of cheating.  It's pretty well known that in the case of certain AAA games, nVidia will actually swap out functionality from the game with baked functionality from it's own drivers, doing things like swapping out shaders that were bundled in the game with their own customized shaders, or toggling internal flags that conventional developers have no access to.

The problem is that DX11 and GL are both very old APIs with lots of legacy functionality.  Their very design in some places is at odds with modern GPU hardware, especially in the area of bundling individual commands together before actually sending them to be executed.  This is why all modern APIs like Metal, Vulkan and DX12 have the concept of a command buffer or batch.  So that the developer can actually be involved in the process of bundling lots of small bind and draw commands together in a logical way.  Older API's like OpenGL have to guess at where the command buffer boundaries should be and (one assumes) drivers tend to have a variety of strategies for doing this depending on how the client application is calling functions.

This is where the driver level optimizations come in, where nVidia or AMD can look at how a given game is calling functions and decide that strategy A is better than the strategy B (the one that driver version X is using), and so they release driver X + 1, and it forces the use of strategy A, or even a new strategy C that they came up with for this specific game.

The end result is that OpenGL (and I assume DX < 12) drivers end up being monstrously complex, not just because of the enormous amount of legacy functionality they have to support, but also because of this per-game baggage that they accumulate (although I suspect that such per-game baggage has some kind of finite lifetime.... no one is going to be benchmarking a RTX 2080 with Battlefield 1942).