Can Frameless rendering reduce latency? And, can FPGA do 3D rendering instead of GPU? In virtual reality, the motion-to-photon time is very important. Oculus says it has to be less than 20ms. Maybe 10~15ms is better.

Some people try to introduce Frameless rendering technology into VR. I think it's good to reduce the latency. But I don't know how the display quality is. According to the paper: "Adaptive Frameless Rendering" and the video (https://www.youtube.com/watch?v=ycSpSSt-yVs), it seems not very good. But, in this paper "Construction and Evaluation of an Ultra Low Latency Frameless Renderer for VR", the authors used FPGA instead of GPU to render and used Frameless Rendering to reduce the latency to 1ms.

In my mind, FPGA is very hard to replace GPU in 3D rendering area. And, Frameless Rendering needs "racing the beam", it is also very hard to program.

What do you think? Is it realistic to use FPGA and frameless rendering?

First of all, the "frameless rendering" technique is in the context of raytracing, not rasterization. It's not obvious how it could be made to work effectively with rasterization, given that the basic idea of it is to update an image by a combination of temporal reprojection plus firing rays specifically at areas where the algorithm thinks the image is undersampled.

So this technique is, prima facie, not compatible with rasterization-based graphics applications. But if you're already doing raytracing for other reasons, this technique would be interesting to look at; it certainly appears to improve quality relative to an image raytraced from scratch each frame, with the same number of rays per second.

Raytracing on a GPU is certainly possible; you don't need an FPGA for that. I've only skimmed the second paper, but my reading of it is that the main reason for the FPGA is to get a close coupling between the display scanout and the rendering activity: namely they "race the beam" and evaluate pixels just before they're about to be scanned out, thus obtaining low latency.

The GPU equivalent of this is probably to split the image in thin horizontal strips, and kick off a compute dispatch to render each strip just before it starts to be scanned out. Today, this is difficult to accomplish as it requires either millisecond-precise scheduling that desktop OSes are not currently set up for, or it requires the GPU to be able to dispatch based on an interrupt from the scanout unit—a hardware feature that doesn't currently exist AFAIK (or if it does, it isn't exposed in any graphics APIs). Or you might be able to make it work with a long-running asynchronous compute dispatch, if you can find a way to stay in sync with scanout.

So, there are obstacles, but they aren't insurmountable, and I think if there was sufficient interest in racing-the-beam-style rendering, then OS and GPU vendors could come up with a way to do it in the future. So I don't think an FPGA is required for this kind of technique to work. On the other hand, the fact that it's based on raytracing is a much bigger obstacle to using it in "real-world" apps and games.


SSAO random noise vector generation I am currently trying to implement SSAO on my graphics engine. I am following John Chapman's tutorial here. This is a pretty popular article that is used as reference in many SSAO articles.

The part that confuses me is on the generation of rotation vector.

for (int i = 0; i < noiseSize; ++i) {
   noise[i] = vec3(
      random(-1.0f, 1.0f),
      random(-1.0f, 1.0f),
      0.0f
   );
   noise[i].normalize();
}

Notice that the z component of the vector is zero. In the article John explain that he want to rotate the sample vector around the z axis. But later when he construct the TBN matrix, he use it as if it is a view space coordinate.

vec3 rvec = texture(uTexRandom, vTexcoord * uNoiseScale).xyz * 2.0 - 1.0;
vec3 tangent = normalize(rvec - normal * dot(rvec, normal));
vec3 bitangent = cross(normal, tangent);
mat3 tbn = mat3(tangent, bitangent, normal);

I don't see any reason why the z vector of the noise vector should be zero. I think that the zero component could be any component. And then what happen if the normal vector is parallel with the noise vector, wouldn't it create a wrong TBN matrix?

Because you are working on screen space, the view ray points towards the eye (being Z the camera space depth). In order to create an orthonormal base, the vectors must be perpendicular to each other. To do this, you set the coordinate z to 0 for the tangents and bitangets, which ensures these vectors will live in the plane perpendicular to the view ray.

Lastly, you need to make the tangent and bitangent perpendicular to each other, which is accomplished with this

vec3 tangent = normalize(rvec - normal * dot(rvec, normal));
vec3 bitangent = cross(normal, tangent);


Sphere intersection occlusion (for hybrid raytracing) Thinking about hybrid raytracing, hence the following question:

Suppose I have two solid spheres $s_1$ and $s_2$. We know their centres and radii, and we know that they have some overlapping volume in space.

We have a typical 3D graphics setup: assume eye is at the origin, and we are projecting the spheres onto a view plane at $z = f$ for some positive $f$. The spheres are beyond the view plane and don't intersect it.

Let $c$ be the circle in space that is points on the surface of both spheres, i.e. the visible (from some angles) 'join' of their overlapping volumes.

I want to calculate if any of $c$ is visible when projected onto our view plane. It might not be, if $s_1$ or $s_2$ get completely in the way.

Any ideas for approaching this?

Given that I didn't miss anything, you can probably cut this down to a problem in the 2D space. Viewing onto the plane defined by the center points of the spheres and your camera origin, the scene looks like this:

The spheres become circles with the center points $C_1$ and $C_2$, and the intersection circle is now only 2 points with only the closer one $P$ being interesting. The camera/eye is arbitrarily set to the point $E$.

Calculating if one point on the spheres is visible or not is easy: Simply check whether or not the angles at point $P$ between $E$ and $C_1$ respectively $E$ and $C_2$ are both greater (or equal to) 90 degree1.

If $P$ is visible, some part (e.g. at least that point) of the intersection circle is visible. Otherwise the whole intersection circle must be occluded by one of your spheres, namely the one which creates an angle of less than 90 degree.

Here is how it looks if $P$ is not visible from $E$:

You can clearly see how that point is occluded by the circle around $C_2$ and that the angle between $E$ and $C_2$ in $P$ is less than 90 degree.

1 Having an angle of exactly 90 degree means that the line between $E$ and $P$ just touches the respective circle/sphere in point $P$ as a tangent.


"Artist Friendly Metallic Fresnel" reversible? While implementing the "Artist Friendly Metallic Fresnel" paper by Ole Gulbrandsen (http://jcgt.org/published/0003/04/03/paper.pdf), I read that it is fully reversible:

  (section 2.3.2): for all r,g ∈ (0,1), the map from (n, k) to (r,g) is a bijection

However, this doesn't seem to be the case? When converting (r, g) to (n, k) and back to (r, g), I do not get a matching output to my initial parameters. Is this expected?

r: 0.7 0.55 0.075
g: 0.8 0.65 0.35
n: 2.39006 2.5479 1.44145
k: 4.98834 3.604 0.537046
r_check: 0.737183 0.601524 0.0773389
g_check: 0.827686 0.699977 0.35995

Here is the implementation to verify, note that I only implemented get_k(), the other functions were already supplied by the paper. get_k() is derived from the following formula in the paper:

$$k(r, n(r,g)) = \sqrt{ \frac {1}{1-r}(r(n(r,g)+1)^2 - (n(r,g) -1)^2)}$$

#include <iostream>
#include <vector>
#include <cmath>

float n_min(float r) {
  return (1.0-r)/(1.0+r);
}

float n_max(float r) {
  return (1.0 + std::sqrt(r))/(1.0-std::sqrt(r));
}

float get_n(float r, float g) {
  return g*n_min(r) + (1.0-g)*n_max(r);
}

float get_k(float r, float n) {
  return std::sqrt( (1.0/(1.0-r)) * (r * std::pow(n+1.0, 2)) - std::pow(n-1.0, 2) );
}

//float get_k2(float r, float n) {
//  float nr = (n+1.0)*(n+1.0)*r-(n-1.0)*(n-1.0);
//  return nr/(1.0-r);
//}

float get_r(float n, float k) {
  return (std::pow(n-1.0, 2)+std::pow(k, 2)) / (std::pow(n+1.0, 2)+std::pow(k, 2));
}

float get_g(float n, float k) {
  float r = get_r(n, k);
  return (n_max(r)-n)/(n_max(r)-n_min(r));
}

int main() {
  std::vector<float> r = {0.7, 0.55, 0.075};
  std::vector<float> g = {0.8, 0.65, 0.35};
  std::cout << "r: " << r[0] << " " << r[1] << " " << r[2] << std::endl;
  std::cout << "g: " << g[0] << " " << g[1] << " " << g[2] << std::endl;

  std::vector<float> n(3);
  std::vector<float> k(3);
  for (int i = 0; i<3; i++) {
    n[i] = get_n(r[i], g[i]);
    k[i] = get_k(r[i], n[i]);
  }
  std::cout << "n: " << n[0] << " " << n[1] << " " << n[2] << std::endl;
  std::cout << "k: " << k[0] << " " << k[1] << " " << k[2] << std::endl;

  std::vector<float> r_check(3);
  std::vector<float> g_check(3);
  for (int i = 0; i<3; i++) {
    r_check[i] = get_r(n[i], k[i]);
    g_check[i] = get_g(n[i], k[i]);
  }
  std::cout << "r_check: " << r_check[0] << " " << r_check[1] << " " << r_check[2] << std::endl;
  std::cout << "g_check: " << g_check[0] << " " << g_check[1] << " " << g_check[2] << std::endl;
}

I wasn't really expecting that, no. The formula in the paper is not the most elegant - there's quite a few parentheses in there. In this case I think it's just a matter of shuffling the parentheses around a little in the get_k implementation - both terms should be divided by (1-r):

    float get_k(float r, float n) {
        return std::sqrt( 1.0/(1.0-r) * ( r*std::pow(n+1.0, 2) - std::pow(n-1.0, 2) ));
    }


Setting up integer texture and binding as image When I set up a texture as float it works (color is written -  BTW I'm doing voxelization):

glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA32F, width, height, depth, 0, GL_RGBA,GL_FLOAT,NULL);

then setting filtering...

then binding:

glBindImageTexture(0, textureID, 0, GL_TRUE, 0, GL_READ_WRITE, GL_RGBA32F);

writing:

layout(rgba32f ,binding = 0) uniform image3D geometryGrid;
...
imageStore(geometryGrid, index, ivec4(1.0)); //ivec4 or vec4, it works

reading:

uniform sampler3D geometryTexture;
...
vec4 g = texture(geometryTexture, geomCoords);

But I want to use an integer texture (so in future I can do binary voxelization and bitwise operations - each bit = slice in z direction). I set it up like that:

glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA, width, height, depth, 0, GL_RGBA, GL_INT, NULL);

then setting filtering...

then binding:

glBindImageTexture(0, textureID, 0, GL_TRUE, 0, GL_READ_WRITE, GL_RGBA32I);

writing:

layout(rgba32i ,binding = 0) uniform iimage3D geometryGrid;
...
imageStore(geometryGrid, index, ivec4(1.0));

reading (same as float):

uniform sampler3D geometryTexture;
...
vec4 g = texture(geometryTexture, geomCoords);

Then it's black. Whats wrong with that? There is sparse documentation about it. BTW The info that iimage is for integer texture, uimage is for unsigned int is actually nowhere to be found.

glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA, width, height, depth, 0, GL_RGBA, GL_INT, NULL);

This is not an integer image format. It's an unsigned, normalized, fixed-point image format. For those, you use floating-point image/sampler variables. So for a 3D texture, you would use image3D, not iimage3D.

If you want a true signed integer format, you would do this:

glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA32I, width, height, depth, 0, GL_RGBA_INTEGER, GL_INT, NULL);

Note two changes. First, the internal format is GL_RGBA32I, which specifies a 32-bit-per-channel signed integer format. The second is the use of GL_RGBA_INTEGER for the pixel transfer format. Even though you're not actually performing a pixel transfer (since you're passing NULL), you still must use an appropriate format and type. And the _INTEGER formats must be used if you're transferring to/from a texture with an integer format.

When you do this, you must of course use the iimage3D and isampler3D types in your shaders.


Where can I find those visual effect demonstration? I'm studying Unreal Engine, this chapter is talk about visual effects: Rendering Overview

Some of them has obvious picture I can see whats the differences. But not all of them. Is there a site can let me input the keyword, like 'bloom', then it shows the description, a image contains 'no bloom' and 'has bloom' both?

The Unreal Engine documentation has good explanations for their various post-processing effects like Bloom. You can find more information about it here: https://docs.unrealengine.com/en-us/Engine/Rendering/PostProcessEffects/Bloom

In the case of UE4, bloom becomes observable once color values push beyond the 1.0 value.


Inconsistent behavior involving VkPhysicalDeviceFeatures.fillModeNonSolid I departed slightly from the Vulkan tutorial I was following and decided to draw a wireframe instead of a solid triangle.

As soon as I call vkCreateGraphicsPipelines(), I get the following message from Validation Layer even though the call finishes with VK_SUCCESS.

  vkCreateGraphicsPipelines parameter, VkPolygonMode pCreateInfos-> pRasterizationState->polygonMode cannot be VK_POLYGON_MODE_POINT or VK_POLYGON_MODE_LINE if VkPhysicalDeviceFeatures->fillModeNonSolid is false.

For good measure I check the VkPhysicalDeviceFeatures right after I call vkCreateGraphicsPipelines() and fillModeNonSolid is VK_TRUE.

The output is a wireframe triangle as expected but the message got me worried if I am missing something.

I am using Nvidia GeForce GTX 1050Ti (driver 417.22), Vulkan 1.1.82.1 on Windows 10(64-bit).

PS: I am experiencing similar problem with wideLines feature as well:VkPhysicalDeviceFeatures has wideLines as VK_TRUE but Validation Layer thinks it is set to false

It turned out that the fillModeNonSolid feature needed to be explicitly enabled during logical device creation.


What software was used to make KDA - Villain? I want to try making small clips with realistic environment with the lighting in the KDA music video...
Any ideas what kind of program to use?

The video was created by a studio called Trizz; you can see their page about the project here. The site doesn’t have any information about what software they used, but you can achieve this look (with enough work) in pretty much any modern renderer. If you use Blender, Cycles is a good one; with other 3D software, Redshift, Octane, and Arnold are popular options.
If you’re also looking to make the kind of physics-based animation—cloth, fire, snakes, etc.—that appears in the video, you’ll probably also want to check out Houdini.


Compute shader textures compared 2D float4 arrays I am wondering is there a difference in performance between using floating point textures as my 2D float4 array vs just passing in 2D float4 arrays?

2D float arrays eliminate the need to do texture look ups to get the colour value, but i am not sure if perhaps textures have some kinda performance boost on GPUs that an array would not due to what ever clever optimisations GPUs can do.

It depends on what you’re doing. If any given thread of your shader is working with data from multiple nearby pixels, like by sampling from fractional between-pixel locations or doing some calculation involving a pixel’s immediate neighbors, then you should probably use a texture: the GPU can do some caching and even load multiple values in a single operation in that case. If your threads are more independent and working on individual values, an array is fine.


With modern fillrates and deferred rendering, is occlussion culling still relevant? For example, while it's the current top-of-the-line GPU, the GTX 980 has a staggering 72.1 gigapixels/second fillrate, which with back-to-front rendering and/or Z buffer checks, seems almost ridiculously large, possibly even at 4k resolutions. As far as polygon counts go, modern GPUs can do tens to hundreds of millions of textured triangles without a hitch if you batch and/or instantiate them properly.

With forward rendering, the amount of fragments that shaders will run on can quickly become overwhelming, but with deferred rendering, the cost is usually more-or-less constant, depending on the resolution, and we long since passed a point where most shading or post-processing effects can be done in realtime in 1080p.

Either way, the limiting factors nowadays are most commonly draw call counts, and shading costs, both of which are kept relatively low by proper deferred rendering and geometry batching, so with that in mind, is culling more than just backfaces and out-of-frustrum polygons of any substantial benefit? Wouldn't the costs(CPU/GPU time, programmer time) outweight the benefits, a lot of the time?

Yes, occlusion culling is still worth it.

At minimum, a draw call that you skipped due to culling is a draw call that doesn't have to run the vertex shader. Triangle count goes up as quickly as GPUs start supporting more triangles, because why not? With unified architectures, vertex shaders use the exact same hardware that pixel shaders do, so every vertex that you skip because of culling is more compute time for the stuff that you can see. Not to mention all the other stuff you're skipping (CPU draw call processing, and throwing the tris far enough through the pipeline that the rasterizer realizes that it doesn't need to shade them).

There was a great presentation from two Ubisoft studios at SIGGRAPH 2015 about GPU-driven rendering pipelines. On the surface it's about some of the things you mentioned: batching and instancing, reducing draw call counts. But one of the major advantages they get out of a GPU-driven pipeline is incredibly fine-grained occlusion culling: better culling than you would normally see at the draw-call level. It's all in the service of asymptotically approaching the goal: processing only what you can see, which means that what you can see looks better.

(Also: consider console, mobile, VR, and desktop-without-the-latest-and-greatest-GPU-that-money-can-buy. Even if all your tris disappear into the gaping maw of your top-of-the-line GPU, you may not be the primary target.)


How is screen space ambient occlusion implemented? I do not understand the explanation from wikipedia.

  For every pixel on the screen, the pixel shader samples the depth values around the current pixel and tries to compute the amount of occlusion from each of the sampled points.

How can depth values of surrounding pixels tell you something about occlusion? Occlusion, as I understand, happens when an object A stands in front of another object B, so you cannot see the object B. But why would you now look at the depth pixels of surrounding pixels? I mean you can see those pixels, so there is no occlusion. Maybe I understood occlusion wrong.

And what I also did not understand is the term kernel in some other tutorials. What is a kernel and why would you use it for ssao?

Could someone make a detailed explanation of the algorithm, with regard to my questions?

As Alan and trichoplax mention in the comments, the effect that ambient occlusion simulates is not the occlusion of a surface from the camera but the surface’s occlusion from its surroundings.

Think of it this way: say you have even illumination coming from every direction, so that the total incoming light at any point sums up to a value of 1. If you place a flat plane in that environment and look at one side of it, that side is going to receive 50% of that illumination, or 0.5, because the other half is blocked by the plane itself. In other words, any point on the plane’s surface can only “see” the light coming from half of the environment, so it’s half as brightly illuminated. If you fold that plane towards your viewpoint (a “valley” fold), then you decrease the incoming illumination to that side of the plane further, to some value below 0.5, because, again, each point on the plane “sees” a bit less of the light coming from the surroundings.

Screen-space ambient occlusion works more or less by looking for these “folds”—areas where the depth changes sharply, as defined by comparing the depths of neighboring pixels—and darkening them to simulate the decreased illumination from the points’ environment.


Outcodes in Cohen-Sutherland Clipping Algorithm What advantages do the outcodes offer in this algorithm ? Do they help in simplifying the algorithm ? Do they decrease the number of calculations ?

The outcodes just represent four boolean flags as a bitfield. Codewise it is easier to move a bitfield around than four booleans. Functionality-wise it becomes very easy to check if any bits are set (just check if zero), and combining outcodes is as simple as a bitwise OR.

On a side-note: Since bit-operations are fast, it is not a performance issue that the individual bits need to be extracted (e.g. outcode & BOTTOM).


Is the microfacet GGX BSDF normally implemented as separate BRDF and BTDFs? I'm trying to build a toy path tracer using something similar to Disney's BSDF, where many materials can be represented as combinations of properties like color, metalness, roughness, transmission, ior.

I'd like to use the GGX microfacet BSDF, but without splitting the ray each time to get & sum the Fr and Ft terms for reflection and transmission. Is it normal/ok to use schlick's approximation to randomly choose between transmission and reflection (Ft vs Fr), and then to just use the BRDF and BTDF formulas separately depending on which path the system chooses?

For example, roughly:

On hit (into)
- Reflect or Transmit via Schlick(i, o, n)
    - Reflect:
        - Direct (N times):
            - Choose point on disc of random light
            - Calc transmission t via BRDF(i, o, n)
            - Calc pdf by area of cone / area of hemisphere
            - Add light * t / pdf
        - Indirect (1 time):
            - Choose direction o by BRDF.Sample(i, n)
            - Calc transmission t via BRDF(i, o, n)
            - Calc pdf by BRDF.PDF(i, o, n) ??
            - Add trace(o) * t / pdf
    - Transmit:
        - Choose direction o by snell(i, n) + some kind of roughness * offset
        - (how do you apply the BTDF here? Or is it important?)
On hit (out of)
    - Choose direction o by snell(i, n) + some kind of roughness * offset
    - Reduce transmission via beer(distance)

I also wonder how diffused reflections factor in. For instance, does GGX only provide specular reflection & refraction? In that case, how do you also handle diffusion?

In current common material models, Diffuse and Specular material properties are handled separately.

Check out slide 43 on http://blog.selfshadow.com/publications/s2015-shading-course/hoffman/s2015_pbs_physics_math_slides.pdf

Basically use a diffuse BRDF (lambert, oren nayar) if the material's refraction is scattering is less than 1 pixel. Use refraction and subsurface models if the scattering is larger than a pixel. I.e. in a glass refraction case the light exit is much greater than 1 pixel away from the light entrance.

You may also be interested in a diffuse parameterization for GGX so that you can use share the same roughness term. https://www.gdcvault.com/play/1024478/PBR-Diffuse-Lighting-for-GGX


Delta angle of two vectors about positive x axis How do I calculate angle C?
I can only know what angle between vector and x positive vector {1, 0}

Angle between green and red is 60 degrees.
Angle between blue and red is -170 degrees.
Shortest angle between green and blue is 130 degrees.

What formula should I use to calculate it?
It must be positive value.

Generally when wanting the smallest angle between two vectors, the dot product is used.
$$
\vec A \cdot{} \vec B = \cos{\angle \alpha}
$$
Where $\vec A$ and $\vec B$ are vectors with the length of $1.0$.

However, you do not have access to the actual vectors themselves, if I understand your question correctly. You just have access to the angles between the vectors and a common direction.

There is a different way you could calculate that angle. You take the absolute value of the difference between the two angles, given that both angles are from a common direction and either both clock-wise or both counter clock-wise.
$$
\angle \phi = |\angle \alpha - \angle \beta|
$$
Where

$\angle \phi$ is the smallest positive angle between the two vectors.
$\angle \alpha$ and $\angle \beta$ are angles in the range from $[0, 360)$, between a common direction and the vector, and either both clock-wise or both counter clock-wise.

What do I mean with how I defined those angles?
First off all, that both angles are between a common direction and the vector. In this case it means that the angles should be between the vector and the red vector where the red vector is the common direction. Which is the case for the angles you provided. It is difficult for me to properly explain why they need to share a common direction, but it should be self explanatory.

What do I mean with that they should be either both clock-wise or both counter clock-wise? When having an angle between two directions, it can either be clock-wise or counter clock-wise. The angle between red and green is 60° and if you start at the red vector (which will be the common direction) you go counter clock-wise with an angle of 60°. The other angle should then also be counter clock-wise. This is the case between red and blue with an angle of -170°. If you look at the red vector, then you go clock-wise. However, the angle is negative, so you go backwards. This means that you actually go counter clock-wise.

And lastly, the angles should be in the range from $[0, 360)$. This makes the formula work a bit better. It is not needed, but then you would have to do some stuff with $\angle \phi$ instead. Basically, if the angle is larger than 360° you just keep subtracting 360° until it is within the range. If the angle is smaller than 0° you keep adding 360° until it is within the range.

Let us try this method with your example.

We have three vectors. $\vec Red$, $\vec Blue$ and $\vec Green$. We now the angles between $\vec Red$ and $\vec Blue$ and between $\vec Red$ and $\vec Green$.
$$\angle RB = -170$$
$$\angle RG = 60$$

$\angle RG$ is within our range, but $\angle RB$ is not. So we add 360° to $\angle RB$ and we get.
$$\angle RB = 190$$

They also both have a common direction, $\vec Red$ and are both counter clock-wise.

When adding them into the formula, we get;
$$\angle \phi = |190-60| = 130$$

And there we have it. The answer.

I do not know if this is a formula with a name or something like that. This method is something I just came up with.

Good luck!


Is there a term for this sort of graphics? I'd like to try to write some C code to create similar animated graphics to these, which are from a very old Commodore 64 game called "The Sentinel".  I've been researching but I can't seem to find if there is terminology that would lead me to information/algorithms on how to implement this sort of thing.  Does anyone know what terminology relates to this sort of super low poly/resolution terrain generation?
thanks

If the terrain is indeed represented by a 3D mesh as it looks like in the images, then I'm not aware of any specific terminology (and I dont believe there might be one).
I could say it was similar to a faceted shading model, but shading looks very rudimentary and lighting computations might not exist at all.
In case you intend to replicate these looks, I'm afraid you already hinted how to proceed very low-polygon non-textured meshes and no lighting.


Is it possible to find out how many fragments made it through the stencil test? I have an OpenGL application which uses stencil tests quite extensively to render irregular shapes (a bit like a simple 2-D CSG). If I could find out how many fragments passed the stencil test and were actually rendered, this would be very helpful in simplifying some computations down the line. Specifically, it would allow me to determine the area of the rendered shape for free instead of having to approximate it with a Monte Carlo simulation later on.

I know that there is a similar concept for primitives emitted from the geometry shader, called transform feedback. I'd like to know if a similar concept exists for fragments and the stencil test.

One possible approach could be the use of Hardware Occlusion Query.

You can use the facts that, by specification, the Stencil Test is executed before the depth test, and only the fragments that pass the depth test are counted by the Occlusion Query.

A simple example (not tested) would be like:

    GLuint samples_query = 0;
    GLuint samples_passed = 0;
    glGenQueries(1, &samples_query);
    // Initialize your buffers and textures ...
    glEnable(GL_DEPTH_TEST);
    glEnable(GL_STENCIL_TEST);

    // Set up the values on the stencil buffer ...

    // Now we count the fragments that pass the stencil test
    glDepthFunc(GL_ALWAYS); // Set up the depth test to always pass
    glBeginQuery(GL_SAMPLES_PASSED, samples_query);
    // Render your meshes here
    glEndQuery(GL_SAMPLES_PASSED);
    glGetQueryObjectuiv(samples_query, GL_QUERY_RESULT, &samples_passed);
    // samples_passed holds the number of fragments that passed the stencil test (if any)

    // Release your resources ...
    glDeleteQueries(1, &samples_query);

Note that the call to obtain the number of samples will call forcibly the flush of the pipeline and wait for the query to finish.
If you need a more asynchronous approach you can query wether the occlusion query is done or not by using:

    GLuint query_done = 0;
    glGetQueryObjectuiv(samples_query, GL_QUERY_RESULT_AVAILABLE, &query_done);
    if (query_done != 0)
        // Your query result is ready
    else
        // Maybe check the next frame?


Why is there so many macros in OpenGL? I would just like to know why are there so many things defined as macros that point to memory addresses in OpenGL?

I see them all over the place. For example when I go to choose parameters for a function such as glDrawArrays(), I have to choose a mode like GL_TRIANGLES as the first parameter which is a macro like all the others.

These aren't macros pointing to memory addresses. They're just integer preprocessor constants. Back in the days when OpenGL originated, using preprocessor constants for your numeric constants was common practice in C.

Nowadays it's considered better practice to define new types (single-element structs) for your constants, but it's too late to change OpenGL.


reconstruction drawing sequence from video I have a video that record the drawing sequence with a pencil by a painter.

I want to reconstruction the sequence from the video.
And I have google some keywords like drawing sequence reconstruction, sketch reconstruction, and have searched like moving object detection but cannot found what I want to find.

I want to know how to detect and recognize the drawing between different frames. I have thought the line detect algorithm may work but fail. And pixel comparison seems to be bad idea. Because that the new drawing may overlap the old ones.

Could you give me some information that I can read or some keywords to search? Sorry about that because I am not familiar with Computer Image Processing.

This is more a Computer vision question I think.
I don't know how exactly you can achieve that but some ideas and keywords you may use to find additional resources:

Feature tracking of hand and pen can yield the movement of said things. Assuming you know that the camera is not moving I think that would give you good results. Related is maybe motion tracking. Any information you have about the hand and the camera can in some way be used to make more accurate estimates.

Removing the hand from the video could be much harder. I don't remember that part from my lecture. We copied parts of the image onto the part we want to remove and used image metrics to make sure it fits best possible... But since the hand covers a drawing we don't know much about this could be very hard to do in an convincing way.


Has anyone found an analytic solution to the product of spherical gaussians from Xu et al? In section 4.2 of this paper:
http://cseweb.ucsd.edu/~ravir/274/15/papers/a209-xu.pdf
Xu et al approximate the product of 2 anisotropic Gaussian distributions. I want to know if someone has found not an approximation but THE solution to what the resulting ASG distribution is after taking the product.
EDIT:
The goal is to either show (or prove it's not possible) that given 2 arbitrary ASG's their product is a third ASG with specified lobe $\bar z$

Consider $f(v) := S(v;z_1)\cdot S(v;z_2)$ with $z_1=z_2=<1,0,0>$. Then
$$f(v) = \max(v_1,0)\cdot\max(v_1,0) = 0, \qquad \text{ if } v_1<0, $$
$$f(v) = \max(v_1,0)\cdot\max(v_1,0) = v_1^2 \qquad \text{ if } v_1\ge 0. $$
We want to find a $\tilde z$ such that $f(v) \stackrel{!}{=} S(v;\tilde z) = \max(v\cdot\tilde z,0)$ for all unit vectors $v$. But $v=<0,\pm 1,0>$ implies $\tilde z_2 = 0$ and similarly $v=<0,0,\pm 1>$ implies $\tilde z_3=0$. Consequently, $S(v;\tilde z) = \max(v_1 \tilde z_1, 0)$ and it is not possible to choose $\tilde z_1$ such that
$$\max(v_1\tilde z_1,0) = f(v)$$
for all unit vectors $v$.


Explanation of the Marching Squares saddle points resolution I was reading the Wikipedia entry on Marching Squares, and was confused by the explanation of the saddle point disambiguation:

Ambiguous Case:

Basically, given the saddle points, the ambiguity is resolved by using the average data value for the center of the cell. I don't understand why this would help resolve the ambiguity. It seems to just select the longer pair of lines over the shorter pair. Is there any reason for this approach?

https://en.wikipedia.org/wiki/Marching_squares#Disambiguation_of_saddle_points

Imagine that a black dot represents a value of -1 whereas a white dot represents a value of +1. You are looking for the iso-line(s) where each point upon the line is 0.

For this example you could either place the iso-lines like on the left or the right:

 or

The difference between the solutions is the value in the center. In the left case we assume 1s whereas in the right solution we assume -1s. So we check the center value and based on that value decide which solution fits best. If the center value is 1, we pick the left solution. If the center value is -1, we choose the right solution.


Difference in pixels between different monitors' sizes If I have an image of x/y resolution and display it on a monitor, does this mean that between a 15-inch monitor and a 42-inch, the same amount of pixels (x/y) will be displayed, only that each pixel will be of different size?
In relation to my first question, if I connect a 25-inch monitor or a 13-inch monitor to a computer, will the GPU/CPU work harder when connected to the 25 inch one? Could there be more lags on the bigger screen, or is it simply a matter of pixels' stretching?

The answer to both your questions is the same: the physical size of the monitor makes no difference at all. If your screen is 1920x1080 pixels, and you display a 960x540 image without any scaling, the image will stake up a quarter of the screen, and it won't take any more GPU time to display on a larger monitor.

The only caveat is that most OSes have some kind of pixel scaling method now, so that icons, buttons, and text aren't unusably small on modern high-resolution displays. In Windows the option is called "Change the size of text, apps and other items". If this setting isn't set to 100%, most things on the screen are scaled up even if the program isn't scaling them. This is more likely to be set high on a small, high-resolution screen than on a large, high-resolution screen. If you try to test displaying the same image on different-sized monitors, and they are using different values for this setting, the image probably won't be the same number of pixels on the monitors, because it's being scaled up according to this setting.


BDPT - How to handle deterministic ray PDF? I'm trying to implement a simple BDPT without MIS as described in Lafortune's paper. I've used the weighting scheme defined in the paper in which they suggest that the weight be based on the surface specularity i.e. specular surfaces should add more weight to continue following the eye path rather than connect with the light path (which makes sense since the eye path is based on the current surface BRDF sampling).

I shoot rays from the light source and the camera a fixed number of times. Each bounce stores the Hit Information and the PDF of the next ray generated. The light path can easily be inverted to get the directions similar to the eye path and by connecting both the paths and using the weighting scheme we can use the simple path tracer algorithm to calculate the total radiance.

There are only 2 things that are different from what happens in a normal path tracer.

1) First is the determinstic ray step i.e. the connection of each eye path vertex with the light vertex. This ray is not sampled from a PDF but instead shot deterministically. What should I use the value for the PDF for this step? Do I need to inverse sample the determinstic ray direction from the eye vertex BRDF PDF to get the probability? Or perhaps from the light vertex?

2) The first ray shot from the light has its own PDF unlike in the normal path tracer where light is found explicitly using Direct Light Sampling. How can I incorporate this factor when I have path tracer that uses Next Event Estimation and Direct Light Sampling? I suppose I can fallback to the naive algorithm (no NEE or explicit Direct Light Sampling) since I already have pin pointed the light source from the light path. Just wanted clarification regarding this.

Thanks.

Sorry about leaving it for so long but I was so busy implementing the goddamn algorithm, I forgot about this :)

So after reading up on Veach's thesis pretty closely, I understood what was going on. First of all Veach's thesis describes about BDPT using MIS. However I implemented without MIS using the techniques described in the original Lafortune's paper.

One thing to note is, that we have to calculate the radiance by the light path inversely as opposed to the normal flow in the eye path. As lightxbulb said for point 1, there is no pdf since the step is deterministic. However the deterministic step needs to be multiplied by the Geometric factor, the BRDF evaluation of light vertex to eye vertex and the BRDF evaluation of eye vertex to light vertex.

For the second point, you don't have to worry about NEE since at each eye path vertex there is a path that connects to the light vertex directly, according to PBRT and the paper we can discard the already generated light source and use explicit light sampling there. Calculating the light path radiance separately (as done in light tracing) handles all the respective PDFs for generating the vertices there.

Will post another question regarding the result comparison soon.


How to take the derivative of a Bézier curve? I want to know how to take the derivative of a Bézier curve. I visited this website https://pages.mtu.edu/~shene/COURSES/cs3621/NOTES/spline/Bezier/bezier-der.html, but I am unable to figure out how they did it. Please help!

A simple example of taking a the derivative of a B'ezier curve can be shown using a cubic curve.
$$C_3(u) = \sum_{i=0}^3 B_{3,i}(u) P_i,$$ where $u \in [0,1]$ and $B_{n,i} = {n \choose i} u^i (1-u)^{n-i}$ is the $i$-th Bernstein polynomial of degree $n$. $P_i$ are the control points.
written out it is:
$$C_3(u) = (1-u)^3 P_0 + 3(1-u)^2 u P_1 + 3 (1-u)u^2 P_2 + u^3 P_3$$
we can differentiate this with respect to u
$$\frac{\partial}{\partial u} C_3(u) = -3(1-u)^2 P_0 - 6(1-u) u P_1 + 3(1-u)^2 P_1 - 3 u^2 P_2 + 6 (1-u)u P_2 + 3u^2 P_3. $$
Now we can rearrange the terms
$$\frac{\partial}{\partial u} C_3(u) =  3 \left((1-u)^2 (P_1-P_0) + 2(1-u) u (P_2 - P_1)  +  u^2 (P_3 - P_2) \right), $$ or
$$\frac{\partial}{\partial u} C_3(u) = 3 (\sum_{i=0}^2 B_{2,i}(u) (P_{i+1} - P_i))$$
We obtain a quadratic B'ezier curve where the control points are the vectors between the initial control points, scaled by a factor $3$. Therefore the derivative of a Bezier curve of degree $n$ is a B'ezier curve of degree $n-1$, where the control points are difference between initial control points scaled by $n$.
so:
$$\frac{\partial}{\partial u} \sum_{i=0}^n B_{n,i}(u) P_i = n (\sum_{i=0}^2 B_{n-1,i}(u) (P_{i+1} - P_i))$$


What is the use of homogenous divide? This question perhaps has been asked and answered a thousand times, and yet I haven't found any that satisfy me. The reasons are often these:

1/ You need a 4 dimensional vector to work with 4x4 matrices.

2/ You need to have z stored in some way so that you could later use it for z-buffering, etc.

3/ You need a matrix that is independent of z so that you could multiply matrices together and save computations. Instead of having to multiply each vector with each of the matrix, you could multiply 1 result matrix with each of the vector.

Here's the problems I have with each of them.

We can pretend that the 3 dimensional vector is 4 dimensional (imaginary w = 1). This is after all Computer Science, not math.
When multiply matrices with vector, we don't need to crush z. Keep it and use it. When drawing, we can pretend that z is already crushed.
You will have to do division by w after projection matrix anyway. Why not divide by z then and get something like [x/z, y/z, z] (I'm not dividing by z because it will be used for z-buffering). Both of them cost 1 computation.

So why do we need homogenous divide?

First of all we need to understand why do we need 4x4 matrices in the first place. With 3x3, we couldn't represent translation as it wasn't a linear transformation (it displaces the origin). So in order to avoid extra work, homogeneous coordinates and affine transformation was introduced.

Now instead of doing

$v' = Lv + t$

where L is a linear transform and t is the translation, we can do

$v' = Av$

Where A is the affine matrix. This makes it cleaner. So 4x4 matrices are a real necessity, we just can't work without them. In order to distinguish between vectors and points we use w = 1 for points and w = 0 for vectors. So you are suggesting to make this 4th dimension implicit and don't store it as it'll actually use space/memory.

  We can pretend that the 3 dimensional vector is 4 dimensional (imaginary w = 1). This is after all Computer Science, not math.

This works but up-to a certain extent. Using this approach we would have 4x4 matrices and 3 dimensional row/column vectors. But now when doing multiplication we will have to make a check whether it's point or vector and based on that we multiply the elements of the 4th column of the transformation matrix by 0 or 1.

This alone starts creating mess, check every time whether it's a point or a vector so why not just use the space for a single float more and get rid of this stupid check.

That aside most modern CPUs have SIMD registers 128 bits wide perfect to fit 4D vectors. If you are doing your calculations on the GPU, all the more reason to store 4D vectors as branching instructions are much more costly on the GPU.

If you start feeling the need to use 4D, great. If not then we still have the problem of composing matrix transformations. Instead of multiplying individual transformations with our vector we would like to pre-multiply all of the transformations then do a single vector-matrix multiplication. This can't work with projective transformations where we need to divide by "something"

Let's come to reason number 3.

  You will have to do division by w after projection matrix anyway. Why not divide by z then and get something like [x/z, y/z, z]. Both of them cost 1 computation.

This is the simple case for just a projective transformation. Consider the vector going through 5 or 6 transformations  and in the last comes the projective transformation. If we pre-multiply all these transformation to create a single matrix, you will notice that now when we multiply the vector with this combined transformation matrix the division factor isn't just a simple -z value. The 4th row of the matrix won't be 0 0 -1 0 as in the standard projection matrix. It might have changed due to multiplying all the transformations together. Now when you multiply that 4th row with the 4D vector, you will get your w value by which you need to divide now.


Constant screen-space-width rim shading As of lately I've been studying a bit of shaders (in Unity), and am trying to recreate the "light-based inline" that can be observed in the latest Legend of Zelda. But maybe the task is a bit above my knowledge, for now.

As from what I can analyze, it seems to behave (when moving) similar to what a "rim shading" does, except it has a sharp cutoff (simple to do), and — the part I'm struggling with — it seems to have a constant width independent from the local curvature of the mesh and distance from camera, essentially having a constant "width" in relation to the screen. (well it actually varies a bit with camera distance, but mostly it doesn't so let's assume it doesn't)

It's a bit hard to show in a still, but you can find plenty videos on the game, and the effect is visible everywhere.

I've managed to do the simpler part, the cut-off rim shading. But this gives the awkward effect that depending on camera orientation and the geometry curvature, the "inline" occupies more or less screen area. Below is an image of the current results I get on various camera angles, with green arrows showing good/acceptable areas and red arrows demonstrating deviation from the intended effect.

This is my current implementation for reference, as a Unity Surface Light Model:

half4 LightingToonPoster(SurfaceOutputStandard surf, float3 viewDir, UnityGI gi) {
    // Material properties
    half3 specular;
    half oneMinusReflectivity;
    DiffuseAndSpecularFromMetallic(surf.Albedo, surf.Metallic, /*out*/ specular, /*out*/ oneMinusReflectivity);
    float3 reflected = normalize(-reflect(gi.light.dir, surf.Normal));
    float specularity = dot(viewDir, reflected) * _Specularity;

    // Light setup
    float ndotl = dot(surf.Normal, gi.light.dir) * 0.5 + 0.5;
    float lightIncidence = gi.light.color * ndotl;
    float3 ramp = tex2D(_Ramp, lightIncidence.xx).rgb;
    float3 specularRamp = tex2D(_Ramp, specularity.xx).rgb;
    gi.light.color = ramp * _LightColor0.rgb;

    // Rim and camera-light convergence
    float rim = 1 - saturate(dot(surf.Normal, viewDir));
    float facing = -dot(viewDir, gi.light.dir) * 0.5 + 0.25;

    // sigmoid of rim for smooth falloff centered at _EdgeBias
    float outline = 1 / (1 + exp(-100 * (rim - _EdgeBias)));
    float highlight = _HighlightStrength * outline * facing * outline;

    float4 std = LightingStandard(surf, viewDir, gi);

    float3 toonDiffuse = surf.Albedo * (gi.light.color / 2 + ShadeSH9(float4(viewDir, 1)));
    float3 toonSpecular = specular * gi.indirect.specular * specularRamp;
    float4 toon = half4(toonDiffuse + toonSpecular, surf.Alpha);
    return lerp(std, toon, _UnshadeFactor) * half4(1 + highlight.xxx, 1);
}

Or check this Gist for the full shader file

The question now being, what can I use instead of rim-shading to obtain a similar effect, where the curvature is not taken into account and I get a constant "inset"?

Nice question! I hadn't seen the game myself but this question got me interested so I watched some footage on Youtube.

This one for example gives a good view of it: https://youtu.be/Ktjdg3zgRzk?t=15m46s

A few things I notice:

Hair and right leg: note how only the diffuse lit part has rim lighting.
Left wrist: note how the width becomes thinner at the angle with the hand.
On some other part you can also see the rim lighting is affected by cast shadows too.

From these observations, I think there are two effects used:

A diffuse shading with a sharp transition between lit a unlit, that happens somewhere below 0 so the light leaks behind a little.
The rim lighting that, as you mention, has an almost constant screenspace width.

From the look of it and some of the artifacts, my best guess is that it's actually some edge detection filter based on depth, which is then used in combination with the diffuse lighting so the rim lighting doesn't affect the unlit parts.

It's a pretty cool effect, thank you for point it out!


First steps towards CAD standard curve fitting I am interested in building a program to accomodate the following workflow:

A user begins with an arbitrary vector/CAD shape, which they wish to transfer onto paper without using a printer (there are both realistic and novelty reasons why this problem is interesting to me). The program will accept this file and create a list of instructions to draft the shape using ruler, square, compass, and standard curve forms. I am not currently concerned with the details of this program (file formats, types of vector curve [spline, bezier, etc], output formatting, etc) but with the core geometric description problem. I think it is fairly straightforward to accomplish this with straight lines, angles, and offsets.

However, I am not sure how to approach the problem of curve fitting. Ideally, the user would be able to have curves in the cad file fit (probably approximated) so that they may draft them using standard French curve forms, including:

Hip Curve

Vary form curve

Assume that these curves are made to standard shapes and sizes and are modeled with dimensional accuracy. Furthermore, they are ruled at specific points and a user can place them on the paper diagram at specific positions and angles (relative to the other elements in the diagram that have already been drafted).

What would be the first steps towards solving this problem? Are there existant computational geometry methods for approximating curves with (segments and combinations of) standard forms? What literature would I want to read? I apologize for the somewhat open-ended nature of this question, but it's difficult for me to determine where to begin.

If it's helpful, the CAD patterns are probably such that the curves can, in fact, be very closely approximated with just one or two of these shapes - the primary use would be for fashion pattern drafting, and these French curves are specifically designed for creating the geometry of flat patterns that fit a human body.

Since you have a limited set of tools you are not actually doing a classical fitting. What you have is a discrete problem. And since you are looking for a somewhat easily drawn fit, no more than twice segmented for example.

One way to approach this is to find all the points that match your curvature requirements. Then find the point x units away from point on both ends for all of these and see how well it matches. If the curvature match at the endpoint is too much off then this isn't a good candidate. If you don't find a good candidate split the problem into two. Once you have found a goodish candidate slide the result to see if relaxing the thing makes the error more symmetric.

Some things to note. Since your french curve is accelerating from one end to another you need to split beziers and b-splines at each span where the curve changes direction. Also once you've found good endpoints be sure to check that the halfway isn't too much off.

Now if you want to do a classical fitting like approach then you need to parametrize the sliding and scale of your curve. This is not likely useful since you in fact can not scale your rules at whim.

Or then you could just get a spline, you know one of those things that were used before computers and stuff, and write instructions on how to set up the spline.


Make Object created in a list blend in/dissolve out of a scene I have a number of (opaque) objects created using the glGenLists() glNewList() utilities. As generated, the Alpha values are all set to 1

Is there a way to make these objects when summoned blend into the scene (sort of as if they were being teleported using the Star Trek transporter). Likewise, I'd also like them to dissolve away when dismissed.

I could of course use time-varying alpha blending in a subroutine if they weren't list objects, but I'm wondering if there's a performance penalty associated with that.

What I'm really trying to do is replace larger coarser ground texture quads  with finer smaller quads as I descend in altitude from a satellite while looking down.

Thanks

It looks like there are a few steps to make this work:

Linearly interpolate the alpha of the mesh from transparent to fully opaque, or vice versa.

Note 1: It seems as if the alpha starts to fade a few seconds after the effect has started.
Note 2: The alpha of the "effect meshes" (covered in #2 and #3) should be changed independently of the original mesh. This will help produce the desired effect.

Generate (just the color buffer for) a mesh with identical vertices, but make the color blue or green. The brightness of each vertice's/triangle's color should be calculated using either random noise (i.e. using a random number generator), or Perlin Noise; or a combination of the two. For the flashing effect, it would be best to regenerate the mesh each frame (with a different noise seed, if using Perlin Noise). Next, calculate a weighted average between the colors of this "effect mesh" and the those of the original mesh, according to how much time has passed. For instance, assuming t is the percentage of the way through the effect that has passed; at t=0, make the color buffer 0*effect_mesh + 1*original_mesh; and at t=0.3, make the color buffer 0.3*effect_mesh + 0.7*original_mesh.
For the vertical line effect, think about ray-tracing all the vertices (using multiple vertical rays). If a vertex is hit by (or near) a ray, make that vertex's color lighter/darker. This effect would be best done with a weighted average also, according to the time (since the lines are stronger towards the beginning).

*** Although this "Star Trek" effect varies from scene to scene, you should order these effects; don't do them all at the same time.


GPU Architecture: What are the theoretical limits on triangle throughput in modern GPUs? For each of major vendor's latest GPU architectures, is there a clear maximum "triangles/second" bottleneck? If so, what is it architecturally and what is the performance?

Your question is not that easy to answer, as there is no standard definition of what "triangles/second" actually means.

What shading is used? What calculations are done? How are the triangles represented? This and much more has to be taken into account when looking for benchmarks on GPU-performance.

To actually compare the performance of GPUs you would have to look at a range of different measurements. GFlop/s and memory-speed as some basic ones, but also look at full benchmark scores (of which there are I am sure many).

There are websites that compile these measurements and display them in a comparison mode. A quick search brought me to this page which seems to have everything needed (I entered an example search).

This helps to compare the performance of two GPUs but if you want to know if a specific GPU is able to do the specific thing you want it to do, that may be very hard to answer and impossible without much more knowledge about what you want to do.

Calculating the maximum performance is also very hard because I believe the GPU vendors do not share everything they do on their GPUs so it can be hard to estimate the actual performance.


BRDF for point lights should not return values over 1 This is standard Cook-Torrance BRDF in formulation for a single point light. No intergrals over a hemisphere, just a single direction to a source of incoming energy.
$$
L_o = { \bigg (  (1-F) \frac{albedo  \cdot  (1-metallness)}{\pi} + \frac{F \cdot D \cdot G_2}{4 (n \cdot v)(n \cdot l_i) } \bigg )  \cdot L_i  \cdot  (n \cdot l_i ) }
$$

$L_o$ - outgoing pixel energy
$L_i$ - incoming energy
$l_i$ - direction to a light source
$n$ - pixel normal
$v$ - direction to the camera
$F$ - calculated Fresnel, ratio of specularly reflected energy, in range [0; 1]
$D$ - calculated integral of GGX NDF over some set of possible micronormal directions (depends on light source size and direction), in range [0; 1]
$G_2$ - calculated Smith geometry masking factor, in range [0; 1]

Usually $D$ in papers is just NDF sampled at one point, which is, of couse, won't give correct value, since it's a probablity density function and its values may exceed [0; 1] range.
$G_2 = G_1(v) * G_1(l)$ doesn't go higher than 1 if we use this Smith $G_1$ formulation:
$G_1(dir) = \frac{2 (n \cdot dir)}{(n \cdot dir)+ \sqrt{a^2 + (1 - a^2) (n \cdot dir)^2}}$ , so it's good as well as $F$.
$(n \cdot l_i )$ in the denominator is cancelled out for the specular part.
Question 1: what's wrong with the $(n \cdot v )$ in the specular denominator? At grazing angles why do we need to divide the specular part by some small value getting big values as result? Or should it be cancelled out due to the fact that we render on per-pixel basis and a pixel covers bigger area of surface at grazing angles and already takes whis divider into accout?
Question 2: there are some "approximation" of Smith $G_2$ like this (Earl Hammon, PBR Diffuse Lighting for GGX + Smith Microsurfaces, GDC 2017):
$$
\frac{G_2}{4 (n \cdot v) (n \cdot l)} = \frac{0.5}{lerp((n \cdot v) \cdot (n \cdot l), (n \cdot v) + (n \cdot l), \alpha) }
$$
Again, why this one gives large values when dot products are close to zero?

I think a single direction can be greater than 1.
For a BRDF the integral over the hemisphere must be less than 1 for energy conservation.
If you consider a specular brdf there is only one direction, where brdf > 0 and all other directions return 0. Therefore when integrating over the hemisphere that direction must be much greater than 1.


Why did AMD's putting an SSD on a GPU board reduce latency so much? AMD have recently been in the news with an interesting Radeon Pro board with a couple of M2 SSDs on it.

As some of the more detailed stories (here or here, for example) note, the benefits may principally accrue not from high bandwidth (the M2's are just on 4 PCIe lanes each, so the board's own 16 lane connector ought to have more) but from low latency.  This story includes the claim "this results in 10x lower memory access latency".

My question is basically: why should PCIe-connected SSDs on the GPU board have significantly less latency than the GPU accessing main system RAM or storage devices on the system PCIe bus?  What is it about the main system that "gets in the way" and means the on-board SSD's can be so much faster to access?

Answering from a somewhat layperson's perspective (I'm not a hardware expert, but I have built my own computer), I'll analogize, using the situation of getting some ingredients for cooking food.

Using "main system" RAM/storage is like you looking for an ingredient, not finding it in your cupboards, and, say, walking over to your neighbor and asking if they have any.  It turns out they do; they bring you it, you thank them and return to your house and use it.

Using "on board" RAM/storage is like you looking for an ingredient, finding it in your cupboard, and using it.

With on board RAM/storage, the GPU just has to do less work to obtain the information it needs.  The information is also simply closer, both physically and when considering the amount of hardware and software the information needs to go through.  With the on board RAM/storage, the GPU needs only look up where the information is, and then retrieve that information from RAM/storage, which then travels through maybe a couple inches of wire to the GPU's processing units.

With main system RAM/storage, there is a larger distance of wire for the signal to travel through which, although it may be only a minuscule fraction of a second, can still add up over thousands of calls to retrieve information.  Also, there are more potential bottlenecks, with all the different pieces that are interconnected.

Lastly, with on board RAM/storage, AMD has the opportunity to optimize the system, since they're controlling precisely hardware is being used.  This is just like how Apple tailors Mac OS to its Mac computers.


Math for adaptive supersampling for raytracing How do I determine how many (jittered) rays to trace for a given pixel, as a function of the statistics of a small initial set of test rays? Also, what size should the initial set be? (it's currently 20, based on eyeball tests)

Currently I'm calculating the variance of the initial set and multiplying that by a arbitrary number (4000..40,000) to give the number of additional rays to trace. This gives acceptable results but I would prefer something grounded in real statistics, mostly so I can get some kind of confidence interval for how close to the true mean my sample mean is likely to be.

Additional possibly relevant info: RGB values are 0..1, pixel color is mean of all samples, jitter is currently random but I'm looking into using a Halton sequence to ensure better distribution, I am modeling diffuse inter-reflections, I'm rendering on a CPU using two threads per core, each rendering thread gets a row of pixels all to itself

I found a paper from 1997 that addresses this subject: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.6799


Confusion about data types in Jensen's subsurface scattering paper I am currently trying to implement Jensen's paper on subsurface scattering, but I am getting confused when I am trying to implement it in my pathtracer. I have questions about the data types of certain variables.

On page 3, $z_r$ is defined as $1 / \sigma_t'$, where $\sigma_t' = \sigma_s' + \sigma_a$. Based on Figure 5, $\sigma_s'$ and $\sigma_a$ are RGB values, which are Spectrum types in my pathtracer. However, $z_r$ is expected to be a distance above the surface, which I presume must be a float, not a Spectrum. How do I handle this discrepancy? Does each RGB wavelength have its own $z_r$?
There is a similar issue in the single scattering term in section 4. $s_o'$ is a distance along the refracted outgoing ray, which is presumably a float. However, the formula for this term uses $\sigma_t$, which is a Spectrum.
What about $\sigma_{tr}$? This value is $\sqrt{3\sigma_a\sigma_t'}$. Is this also a Spectrum or a float?

The heading row of Figure 5(b) says that the units of $\sigma_s'$ and $\sigma_a$ are inverse millimetres, that is, inverse distance. Hence, $1/\sigma_t'$ has units of distance.

If you send a beam of photons through a medium with scattering coefficient $\sigma$ and measure it at distance $l$, then the probability that a photon has made it that far without being scattered is $e^{-\sigma l}$. Since $l$ has units of distance, $\sigma$ must have units of inverse distance.


How is this normals map supposed to work? I have come across this image that's labelled a bump map to be used with a 3D model. It looks like this:

In this map, which is an RGB map, it seems we have pixels whose normals face upwards shaded lighter, and normals pointing downward shaded darker. I have isolated just one part of this map, a little mound with a whole in the middle:

I've tried to recreate this shape in an image editing program just using white colour for height, it's essentially a heightmap, or bumpmap. It looks like this:

It's basically a circular mound that rises gradually, and then falls off steeply into a hole.

So I know that this is one way normals and height can be stored. This is essentially a heightmap, but some people and programs call this a bumpmap. In 3DS Max the bumpmap essentially expects a heightmap.

Then there's the normal map, which looks like this:

Inside the red channel the information about how flat or inclined the surface is from left to right:

And same as in the green, except from up to down:

So I understand this, I think. However in the original image, this strange "bumpmap", it only seems to contain the information of the flatness of the surface from up to down, not left to right. What I've shown is how it appears as an RGB image. Here is each of the channels:

They all contain the same information, that is, facing of the surface from up to down.

How is this map supposed work? I've come across a few maps like these but I don't see how they're supposed to work. Can someone explain if this is possible?

Yeah, that doesn’t look like any bump, height, or normal map I’ve ever seen—as you’ve identified, there’s only information about the surface contour along a single axis. If anything, it looks like it’s meant to be overlaid on a texture as a cheap form of fake bump-mapping in an era where it was too expensive to do it for real.


OpenGL C++ Camera Rotation Problem Currently for a university assignment I have been given a 3D rendering engine, capable of rendering a 3D world. One task is to create a first person camera for it (I know with OpenGL you don't have a camera, you have to effect the modelview).

Anyway, I have managed to get a first person camera working, however rotation is becoming a bit of a problem. I have spent a good few hours trying to fix the problem, but can't seem to find a solution.

Rotation works perfectly, until you move forward about -10 into the z axis. When you do, the camera rotation seemingly rotates around a set point, so it orbits the actual model, rather than just changing the direction the camera is looking at.

So if you move out, then rotate, the camera will move around the object, not just move the object out of sight.

Not sure what code you'll need to see to help, but here's some:

Camera Rotation:

void Camera::RotateCamera(float yaw, float pitch, float roll)
{
//left and right rotation
Quaternion a;
Quaternion b;
Quaternion c;
Quaternion up;
Quaternion right;
Quaternion temp;
float s;
float vx;
float vy;
float vz;

if (yaw != 0)
{
    s = cos(yaw / 2);
    c.SetQuaternion(m_direction, s);
    vx = m_upVector[0] * sin(yaw / 2);
    vy = m_upVector[1] * sin(yaw / 2);
    vz = m_upVector[2] * sin(yaw / 2);
    a.SetQuaternion(vx, vy, vz, s);
    b.SetQuaternion(vx, -vy, -vz, s);

    up = a*c*b;

    m_direction.SetVector(up[0], up[1], up[2]);
}

//up and down rotation
if (pitch != 0 && curDown < maxDown && curDown > -maxDown)
{
    curDown += pitch;
    s = cos(pitch / 2);

    a.SetQuaternion(m_direction, s);
    vx = m_rightVector[0] * sin(pitch / 2);
    vy = m_rightVector[1] * sin(pitch / 2);
    vz = m_rightVector[2] * sin(pitch / 2);
    b.SetQuaternion(vx, vy, vz, s);
    c.SetQuaternion(-vx, -vy, -vz, s);

    right;
    temp;
    right = b * a * c;
    m_direction.SetVector(right[0], right[1], right[2]);
}
m_direction.Normalise();
m_rightVector = m_direction.CrossProduct(m_upVector);
m_upVector = m_rightVector.CrossProduct(m_direction);

m_rightVector.Normalise();
m_upVector.Normalise();
UpdateViewMatrix();
}

The quaternion class is correct (so I'm assuming as it was already created for us).

Camera rotation code

if (x - ox > 0)
{
    cam_pos->RotateCamera(0.05, 0, 0);
}
else if (x - ox < 0)
{
    cam_pos->RotateCamera(-0.05, 0, 0);
}

ox = x;
oy = y;

Applying the camera

modelview = (*cam_pos->GetViewMatrix()) * modelview;
modelview = modelview*translation;

float aspect_ratio = (float)m_width / (float)m_height;
projection.SetPerspective(60.0f, aspect_ratio, 1.0f, 1000.0f);

glUniformMatrix4fv( m_uniform_projection, 1, GL_FALSE, projection.ToPtr() );
glBindSampler(0, m_texDefaultSampler);

Where *cam_pos is the actual camera.

The code works, just rotation goes off.

This is what currently happens on rotation:

A common mistake that causes this behavior is to first apply the translation and then the rotation when rendering the scene. I think that's what's happening. If you apply the rotation first and then the translation it should behave as expected. Give that a try


Reconstruction filter of LCD monitor I'm very confused about sampling and reconstruction of images.
I read that in the framebuffer there is a sampled representation of an image, for visualization the sampled representation is converted into a continuous one by the video hardware.
Which filter is used by an LCD display ?

But isn't an LCD display already digital ?

The output of an LCD or other display is continuous by virtue of its being a physical device, with pixels that have some size and shape, and emit some physical spectrum of light.

from Apple Retina Display by Bryan Jones

My guess as to what's meant by the comment that "the sampled representation is converted into a continuous one by the video hardware" is just that the input to the hardware is a sampled digital signal that drives the physical pixels, while the output is a continuous light field such as the above. So the "reconstruction filter", so to speak, is the physical pattern of light emitted when the display is turned on.

(Also note that displays often do additional digital processing of the input image before displaying it, e.g. to adjust brightness, contrast, color temperature, sharpness, and so forth. But those things wouldn't be called a reconstruction filter.)


How does a PenTile display controller interpret its RGB framebuffer? On a PenTile display, the mapping from pixels in the framebuffer to individual subpixels is not trivial. Each pixel in the framebuffer may only have RG and BG subpixels, in some interesting geometric arrangement.

I'm trying to do some subpixel antialiasing of content that will be displayed on a PenTile display, so to get best results I'd like to understand how I can set the values of each pixel in the framebuffer to control the subpixels individually. (N.B. I know the subpixel layout of the display I'm targeting.)

I can't say for certain, but I was interested in this myself and trying to look it up (with little success). I did analyze the many macro photos you can find on the web of these kinds of displays, though.

RGB pixels      Pentile Subpixels (RGBG Layout)

--+---+--           --+---+--
X | X | X           G r G b G
--+---+--   -->     --+---+--
X | X | X           G b G r G
--+---+--           --+---+--

As far as I can tell, it looks like the green subpixels' locations exactly match the framebuffer pixel locations (X) and the intensity is transferred 1:1. The red and blue subpixels are shared between the two neighbouring pixels horizontally and their intensity seems to be the average of the red/blue channels of the two neighbouring framebuffer pixels left and right, effectively halving the horizontal resolution of R and B.

This strategy seems to carry over to the diamond Pentile layout even though the the red and blue subpixels are shifted downwards (or upwards) by half a pixel:

RGB Pixels       Diamond Pentile subpixels
                         _________
X   X   X                G | G | G
              -->        __r___b__
X   X   X                G | G | G
                         __b___r__

Due to the additional half pixel vertical shift for red and blue of this layout that isn't accounted for in driving the red and blue subpixels, you might be able to notice that a horizonal edge between a black area and a white area might appear green or magenta. You can see this effect in macro photos of diamond Pentile subpixel matrices rendering text. But there is actually no consistent shift. In some photos it's downwards and in others it's upwards.  I don't know if the displays were manufactured differently or just installed upside-down in some devices.

So, yes, you can easily control all subpixels of a Pentile display. Each color channel value in your frame buffer is used exactly once and for only one pixel. It's just that for the red and blue subpixels two frame buffer pixels' red and blue values are averaged.

Knowing this, you could try to do a more sophisticated subpixel rendering. But keep in mind that your typical color space like sRGB is nonlinear. The filtering should be done in a linear domain. For example, a checker board of alternating 0 and 255 will have the same brightness as a flat area with value around 185 (if I remember correctly) instead of 128.

And finally, I want to point out that it's possible to come up with anti-alias filters for non-rectangular sampling lattices. For example, for this checkerboard sampling lattice you have for the red and blue subpixels, the ideal lowpass filter would have a pass band in form of a diamond shape in the spectral domain. If you tried to reduce the resolution by dropping every 2nd sample in a checkerboard fashion, reasonable anti-alias filters would be these 5x5 and 3x3 kernels while the 5x5 version retains more detail:

 0   -1    0   -1    0
-1    0   10    0   -1              0  1  0
 0   10   32   10    0              1  4  1
-1    0   10    0   -1              0  1  0
 0   -1    0   -1    0

These can be applied in-place due to the zeros' locations.

You might also want to include a half-pixel shift for one or two dimensions in there to account for the relative positioning of the red blue and green subpixels. For example, for the diamond pentile layout you might want to use one of these filters:

  3    0    0    0    0    3
  0  -25    0    0  -25    0          -1   0   0  -1
  0    0  150  150    0    0           0   9   9   0
  0    0  150  150    0    0           0   9   9   0
  0  -25    0    0  -25    0          -1   0   0  -1
  3    0    0    0    0    3

to come up with the right red/blue subpixels instead of just averaging four values.

The RGBG (non-diamond) Pentile pattern would require only a half-pixel shift horizontally, leading to a filter like this:

 -3   3   3  -3
  2  30  30   2
 -3   3   3  -3

Some care must be taken with clipping due to the negative filter coefficients. For example, you could try to redistribute the residual to surrounding pixels to keep the average brightness.


How linear interpolation works between mipmaps? Nearest interpolation between 2 texels means to take the texel whose center is nearest to a given coordinate to draw or, in other words, the texel the coordinate is matching.
Linear interpolation between 2 texels means to mix the colors of them using more of the nearest texel and less of the others according to the distance to a certain coordinate.
Nearest interpolation between 2 mipmap levels consist in taking the mipmap whose size fits better the size of the image to draw
But I cannot understand how linear interpolation works with mipmap levels.

It's called trilinear interpolation. You first do a bilinear interpolation of the higher-res texture, then do a bilinear interpolation on the lower-res texture, then interpolate between the 2 results. The weight of the final interpolation is based on where between the 2 textures your Z-coordinate falls. If 0 is fully the low-res texture and 1 is fully the high-res texture, then you can use the standard glsl mix() function to combine the two:

result = (highres texture color * weight) + (lowres texture color * (1.0 - weight));


How are the three plane equations derived in the NVIDIA's CG Tutorial chapter 8? I'm reading the chapter 8 of the Cg tutorial and I could not understand 8.4.1 :

Because all these coordinates lie in the plane of the same triangle, it is possible to derive plane equations for x, y, and z in terms of s and t:
A_0 x + B_0 s + C_0 t + D_0 = 0
A_1 y + B_1 s + C_1 t + D_1 = 0
A_2 z + B_2 s + C_2 t + D_2 = 0

How are these equations derived?

In this context, deriving the plane equations does not refer to the equations themselves, but to the coefficients $A$, $B$, $C$ and $D$.

The equations are simply plane equations for a plane defined by three points, where the first plane is defined by the three points $(x_i,s_i,t_i)_{i=1..3}$, the second plane is defined by $(y_i,s_i,t_i)_{i=1..3}$ and the third plane by $(z_i,s_i,t_i)_{i=1..3}$. They can be seen as auxiliary objects in deriving equations for $x$, $y$ and $z$ in terms of $s$ and $t$.


Ray tracing geometries with same position, translation and scale this is my first post on the Computer Graphics Stackexchange forum. Apologies in advance if I am doing something wrong.
I have the problem that I have a scene with overlapping geometries. To be more precise, my scene is a Cornell Box, where the ceiling and the area light are quads that are lying in the same plane, e.g. are having the same height value:

This image is rendered by our "home renderer" and I am working on the integration of Intel's Embree framework into it, to accelerate the ray casting process.
When I render the scene with the help of Embree, it looks like this:

(There is nothing wrong with Embree itself, but probably with what I am doing with it)
What is immediately visible is that the ceiling looks more "grainy" or noisy and that pixels in the area light have the colour of the ceiling.
I strongly believe that this has something to do with the fact that the ceiling and the
area light are lying in the same plane.
So my question is: when such overlapping geometries are intersected with a ray, how does one choose which of the two geometries should be prioritized?
I did a short experiment in Blender with two cubes with different materials:

Then I moved the green cube "into" the red one, meaning I assigned the same position, such that the two cubes are overlapping.
This is what the Cycles renderer gave me:

Does anyone know what is going on? How can I handle the situation of overlapping geometry in my ray tracer?
Many thanks in advance for your help and support. Have a nice Weekend!
Kind regards
Sebastian

This is more of a comment then an answer:
I have resolved this problem in a ray tracer using Constructive Solid Geometry(CSG). Which was done by adding operators that allowed us to create objects that are the union, difference, and intersection of a combination of objects. Here is a wiki page on the subject.
CSG sounds difficult to implement, but it is actually surprisingly straight forward.
But, this system was added for other reasons, it just turned out that we could "subtract" the light from the box (using your post as an example), then the intersection for the light itself falls out naturally since it can no longer hit the box where the two intersect.
This really is just reiterating what everyone has been saying in the comments, every solution is either done by creating real geometry that avoids the problem (like CSG) or use some sort of hack. I posted the CSG answer because it resolves the problem so naturally if it is available.


Perturbed image texture implementation from renderman language I am trying to implement (in C#) an image perturbation algorithm presented in the book "Texturing and modeling - K. Perlin et al" (page 91 if anyone has it), which distorts an image.
The following code is in Renderman language:
The texture access

Ct = texture("example.tx", s, t);

is replaced by

point Psh;
float ss, tt;
Psh = transform("shader", P);
ss = s + 0.2 * snoise(Psh);
tt = t + 0.2 * snoise(Psh+(l.5,6.7,3.4));
Ct = texture("example.tx", ss, tt);

transforming the image on the left to that on the right.

From what I undestood, instead of accessing coordinate $(s,t)\in[0,1]$ we access slighty perturbed coordinates $(ss,tt)$ and display them at place $(s,t)$, thus creating an image that looks slightly perturbed.

$snoise(x)$ is defined as $(noise(x)*2)-1$, mapping noise from $[0,1]$ to $[-1,1]$, and in the RenderMan documentation $noise(P)$ where P is a point, returns a value based on some noise (most likely perlin or lattice).
(http://renderman.pixar.com/resources/current/RenderMan/noiseFunctions.html)

What I don't understand is what the transform function does, which is supposed to map the 3d point P into the "shader" space, and how can it be implemented.
Also, I'm not sure whether noise(x) returns a 3d point, a float (would make more sense) and if I can use a simple 2d implementation of Perlin's noise to reach the same desired effect.

As you've surmised, the transform() function transforms points from one co-ordinate space to another. (There are also vtransform() and ntransform() for transforming direction vectors and normal vectors, respectively.) The string argument names the co-ordinate space to transform into.

The Renderman Shading Guidelines have this to say about it:

  At the start of shader execution, all point, vector, normal, and matrix variables are expressed in the "current" coordinate system. Exactly which coordinate system is "current" is implementation-dependent. It just so happens to be that "current" is "camera" for PRMan*, but you should never count on this behavior - it is entirely possible that other RenderMan compliant renderers (including future renderers from Pixar) may use some other space (like "world") as "current" space.

It goes on to give a case like this as an example. Most lighting calculations should be done in camera space, but evaluating a noise function should be in the object's co-ordinate system, because you want the noise to stay the same as the object moves through world space.

In your C# implementation, you'll also need to transform the point being shaded from camera space to the object's co-ordinate system. Maybe you've done this already before computing the texture co-ordinates. If not, you'll need to multiply by the object's transformation matrix. Remember that the only use of this transformed point is as an input (like a seed) to the Perlin noise generator. It sets the domain that the noise varies over: world space co-ordinates.

In RSL, the noise() function can return any type you like: a float, a color, a point, or a vector. As you're adding it to another float (u or v), you'll get a float in this code. Really, the two noise() calls, added to s and t, are acting to generate a single 2D noise vector. In your own code, if you are using a 2D vector to store your texture co-ordinates, you can use a single noise function that returns a 2D vector, to get the same effect in one line of code.

If you're interested in making a nice noise generator, Shadertoy has a lot of noise shaders featuring different variants of Perlin noise with different properties (isotropic or not, configurable smoothness and bandwidth) and is worth looking at for inspiration as well as implementation hints.


Does premultiplied alpha give order independent transparency? I've heard that pre-multiplied alpha gives you order independent transparency but when I sit down and do the math, it doesn't seem to be working.

Is that untrue, or am I doing something incorrectly?

The formula I'm using is:

$out_{rgba} = in_{rgba} + out_{rgba} * (1 - in_a)$

where $in$ is premultiplied alpha. In other words, taking a "normal" color in RGBA, i multiply RGB by a.  30% opaque white would start out as (1, 1, 1, 0.3) but would become (0.3, 0.3, 0.3, 0.3) as premultiplied alpha.

After getting the wrong answers when working it out by hand, I wrote the C++ program below and am still getting the wrong results.

After execution:

$out1 = (0.738, 0.913, 0.3, 1.0)\\
out2 = (0.738, 0.875, 0.113, 1.0)$

Can anyone explain why?

#include <array>

typedef std::array<float, 4> RGBA;

void PremultiplyAlpha (RGBA& rgba)
{
    rgba[0] *= rgba[3];
    rgba[1] *= rgba[3];
    rgba[2] *= rgba[3];
}

RGBA BlendPremultipliedAlpha (const RGBA& dest, const RGBA& src)
{
    RGBA ret;
    ret[0] = src[0] + dest[0] * (1.0f - src[3]);
    ret[1] = src[1] + dest[1] * (1.0f - src[3]);
    ret[2] = src[2] + dest[2] * (1.0f - src[3]);
    ret[3] = src[3] + dest[3] * (1.0f - src[3]);
    return ret;
}

int main(int argc, char **argv)
{
    RGBA greenGround = { 0.0f, 1.0f, 0.0f, 1.0f };
    PremultiplyAlpha(greenGround);

    RGBA red25PercentOpaque = { 1.0f, 0.0f, 0.0f, 0.25f };
    PremultiplyAlpha(red25PercentOpaque);

    RGBA white30PercentOpaque = { 1.0f, 1.0f, 1.0f, 0.3f };
    PremultiplyAlpha(white30PercentOpaque);

    RGBA yellow50PercentOpaque = { 1.0f, 1.0f, 0.0f, 0.5f };
    PremultiplyAlpha(yellow50PercentOpaque);

    // one way
    RGBA out1;
    {
        // start with the green ground and blend in 25% opaque red
        out1 = greenGround;
        out1 = BlendPremultipliedAlpha(out1, red25PercentOpaque);

        // then blend in 50% yellow
        out1 = BlendPremultipliedAlpha(out1, yellow50PercentOpaque);

        // then blend in 30% opaque white
        out1 = BlendPremultipliedAlpha(out1, white30PercentOpaque);
    }

    // other way
    RGBA out2;
    {
        // start with the green ground and blend in 30% opaque white
        out2 = greenGround;
        out2 = BlendPremultipliedAlpha(out2, white30PercentOpaque);

        // then blend in 25% red
        out2 = BlendPremultipliedAlpha(out2, red25PercentOpaque);

        // then blend in 50% yellow
        out2 = BlendPremultipliedAlpha(out2, yellow50PercentOpaque);
    }

    return 0;
}

Premultiplied alpha itself does not give you order independent transparency, no.

This page talks about how it can be used as part of an order independent transparency solution however: http://casual-effects.blogspot.com/2015/03/implemented-weighted-blended-order.html

Other benefits of premultiplied alpha include:

better mipmaps for textures that contain alpha: https://developer.nvidia.com/content/alpha-blending-pre-or-not-pre
Being able to do both transparency and additive blending within a single texture without needing to change render states or issue multiple draw calls: http://amindforeverprogramming.blogspot.com/2013/07/why-alpha-premultiplied-colour-blending.html
Being able to represent more realistic lighting situations, such as the fact that being non opaque doesn't always mean transparency: http://distantsoulsdev.blogspot.com.ar/2013/02/premultiplied-alpha-online-interactive.html
Being able to more naturally / correctly / effortlessly composite images that have alpha channels.
Being able to get better results from image processing (blurs / convolution). No black halo: https://stackoverflow.com/questions/4854839/how-to-use-pre-multiplied-during-image-convolution-to-solve-alpha-bleed-problem
Premultiplied alpha also allows transparent objects to be drawn front to back instead of back to front, possibly saving pixel writes.


Why does an affine transform work only on three of the corners? I am trying to use a linear transformation to straighten out the camera angle on this image:

In other words, I want to transform the image into a head-on view of the whiteboard.

I applied an affine transformation to that image, identifying the top left, top right, and bottom left corners, thinking that it would turn the image into a 1080x720 rectangle. However, here's what I ended up with:

As you can see, it is NOT a rectangle, but some sort of misshapen quadrilateral. It's close, but not exactly a head-on view of the whiteboard!

What exactly happened here? I really thought an affine transformation would work.

An affine transformation doesn't have enough freedom to do what you want. Affine transforms can be constructed to map any triangle to any other triangle, but they can't map any quadrilateral to any other quadrilateral.

One way to see this is that the matrix for a 2D affine transform has only 6 free coefficients. That's enough to specify what it does to 3 points, but not 4.

For this task you likely want a projective transformation. A 2D projective transformation has enough freedom to map any convex quadrilateral to any other.


What's the main difference of pipeline process between Vulkan and DX12? DX12 introduces a new feature of pipeline named 'Bundle'. It seems that can optimize the command list, and send it to final pipeline. Vulkan invent some different pipeline: The graphic pipeline and the compute pipeline. The graphic pipeline seems just the same with the original, the compute pipeline is used as an accessor of image memory.

My CG knowledge is pretty basic, and can not confirm that cognition is correct. Would someone kindly provide some explanations for it - The main difference of pipelines between them ?

References:

https://msdn.microsoft.com/zh-cn/library/mt617309
https://www.khronos.org/registry/vulkan/specs/1.0/xhtml/vkspec.html#pipelines

D3D12 has 4 separate kinds of command lists: direct, bundle, compute, and copy.

Vulkan has similar concepts, but they happen in a different way. Command buffers are allocated from command pools. And command pools are directly associated with a queue family. And command buffers allocated from a pool can only be submitted to the queue family for which the pool was built.

Queue families specify the kinds of operations that the queue can take: graphics, compute, or memory copying operations. D3D12's command queues have a similar concept, but D3D12's command list API has you to specify the list type. Vulkan's gets this information from the queue family the pool is meant for.

The D3D12 "bundle" command list type seems similar on the surface to Vulkan secondary command buffers. However, they are quite different.

The principle difference is this: bundles inherit all state from their executing direct command list, except for the bound PSO itself. This includes resource descriptor bindings.

Vulkan secondary command buffers inherit no state from their primary command buffer execution environment, except for secondary CBs that execute in a subpass of a renderpass instance (and queries). And those only inherit the current subpass state (and queries).

This means you do different things with them, compared to D3D bundles.

Bundles are sometimes used to modify descriptor tables and render stuff, on the assumption that the direct command list they're executed within will have set those tables up. So bundles are kind of like light-weight OpenGL display lists, only without all of the bad things those do. So the intent with bundles is that you build them once and keep them around. They're supposed to be small things.

Vulkan secondary CBs are essential for threaded building of commands intended for a single render pass instance. This is because a render pass instance can only be created within a primary CB, so for optimal use of threads, there needs to be a way to create commands meant to execute in the same subpass in different threads. That's one of the main use-cases of secondary CBs. So the intent is that you'll probably build secondary CBs each frame (though you can reuse them if you want).

So in the end, bundles and secondary CBs are intended to solve separate problems. Bundles are generally dependent on the executing environment, while secondary CBs are more stand-alone.

At the same time, Vulkan secondary CBs can do something bundles cannot: they can execute on compute/copy-only queues. Since Vulkan makes a distinction between the primary/secondary level of the command buffer and the queues where that CB can be sent, it is possible in Vulkan to have secondary command buffers that execute on compute or copy-only queues.

Direct3D 12 can't do that with bundles. The ExecuteBundle function can only be called on a direct command list. So a copy-only command list cannot execute bundles.

Granted, because Vulkan doesn't inherit anything between secondary CBs except for subpass state, and compute/copy operations don't use render passes, there isn't much to be gained from putting such commands in a secondary CB rather than a primary one.

D3D12 has the same separation between compute pipelines and graphics pipelines that Vulkan does. However, when issuing commands, D3D12 has only one pipeline binding point, to which you can bind any kind of pipeline. By contrast, Vulkan has separate binding points for compute and graphics pipelines. Of course, Vulkan doesn't have different descriptor binding points for them, so the two pipelines can interfere with one another. But if you design their resource usage carefully, it is possible to invoke a dispatch operation without disturbing the needs of the graphics pipeline.

So overall, there's no real difference in pipeline architecture here.


What effects does path tracing capture that recursive ray tracing does not? So I have been wanting to make a rendering engine as a summer project. I'd like to shoot for photo-realism of static scenes rather than real time dynamic game like graphics. So in my research I came across two major techniques for going about this. One is to attempt to directly solve the rendering equation with specialized numeric methods like bidirectional path tracing. Another is the well known ray tracing technique. Ray tracing is a) much simpler and b) more efficient. Additionally it seems as though some kind of interpolation/de-noising is needed to make the images look properly smooth when path tracing techniques are used.

I understand recursive ray tracing to be an algorithm in which rays are shot out from the eye and collide with objects in the space. When they do more rays are shot from that point to calculate the lighting at that point.

I understand path tracing to be trying to find paths to the viewing plane from light sources. If enough of these paths are sampled without bias then you can use these paths to perform monte carlo integration over the space using the found paths as your sample points.

What effects are captured by path tracing that are not captured by ray tracing

Generally speaking, path tracing removes a number of assumptions that ray tracing makes. Ray tracing usually assumes that there is no indirect lighting (or that indirect lighting can be approximated by a constant function), because handling indirect lighting would require casting many additional rays whenever you shade an intersection point. Ray tracing usually assumes that there is no such thing as non-glossy reflection, because while it is fairly easy to handle glossy/mirror reflection (you reflect the ray and continue with the raycast), if you have non-glossy specular reflection you have a problem very much like indirect lighting: you need to trace a bunch more rays and then do lighting wherever those rays hit. Ray tracing also usually assumes that there are no area lights, because it is very cheap to cast a single ray towards a light to determine if an intersection point is in shadow, but if you have an area light you generally need to cast multiple rays towards different points on that light to determine the amount that the intersection point is shadowed.

And as soon as you need to break any of those assumptions with ray tracing, you now have a sampling problem: you need to decide how to distribute the many additional rays you need to trace in order to get an accurate result that doesn't have noticeable aliasing.


Rasterizing spheres? I want to start developing a hybrid renderer: it first applies a rasterizer, then stores for each pixel the surface pointer, normal vector, and distance to the camera.

Then it raytraces reflections, refractions, lambertian diffuse, etc using those values.

The first problem however is rasterizing a sphere: while there are many tutorials for rasterizing quads and triangles, there aren't any on spheres, and I don't know how to do this myself while keeping perspective intact.

So how do you perfectly rasterize an explicit sphere (with radius, center)?

Jim Blinn's book Jim Blinn's Corner: Notation, Notation, Notation has a couple of chapters which go through this in detail with all of the edge cases. This book is a collection of essays from his column in IEEE Computer Graphics and Applications. Highly recommended.


Can we use anisotropic filtering on 3D textures? Do modern GPUs support anisotropic filtering for 3D textures? If yes, how can one use it? The OpenGL spec doesn't seem to be very precise on this. From this link:

Should anything particular be said about anisotropic 3D texture filtering?

  Not sure.  Does the implementation example shown in the spec for
  2D anisotropic texture filtering readily extend to 3D anisotropic
  texture filtering?

I am not aware of hardware support for 3D anisotropic filtering.  I could be wrong about its existence though.  I believe it has been tried.

The motivation for 2D anisotropic filtering is to prefilter the texture function over the screen area it falls under in a more accurate way than doing a box filter in texture space (usually, anisotropic filtering does several box filters in texture space).

This definition is key.  2D anisotropic filtering tries to:

  Integrate over the part of a flat surface that you can see(within a single pixel on the screen).

What does 3D anisotropic filtering mean?

  Integrate over the part of  a . . . volume? . . .  that you can . . . see?(within a single . . . voxel? . . . on the screen)

The best analogue I can think of would be doing some sort of 3D texture lookup anisotropically along the viewing ray.  This isn't quite a direct analogy.  Anyway, the problem with this is that when you're doing volume rendering, you actually care about how this integration is done a lot more than in the 2D case.

Do you want absorption?  Emission?  (You need to do some kind of exponential to integrate.)  Do you want scattering?  (You need to do some kind of recursion or approximating blur to integrate).  And this doesn't even get into what kind of data the 3D texture stores.  Does this texture mean opacity, or does it mean density?  Is it diffuse or is it emission?  All of these need to be integrated differently.

New developments in volume rendering are happening all the time, and in any case there are so many variously suboptimal ways you could try to generalize 2D anisotropic filtering, the doubt expressed by the specification makes sense to me:

  Does the implementation . . . for 2D anisotropic texture filtering readily extend to 3D anisotropic texture filtering?

No.