Downscaling texture via mipmap [DirectX 11] I am implementing a post-processing effect in my DirectX 11 pet renderer. The post-processing pass is implemented by rendering a full-screen quad covered with texture containing original rendered image, which works as it should, but I have problems with downscaling the texture.

The non-processed testing scene looks like this (three very bright emmissive spheres):

I see no problem at this stage, but when I run the first post-processing pass, which just down-scales the image by the factor of 8 using the texture sampler, the result is very flickery (up-scaled for clarity):

I expected a mipmap would solve or at least reduce the flickering, but it didn't change a thing. What am I doing wrong?

RenderDoc Update

After investigating the issue using RenderDoc I found that the mipmap is being generated successfully and it's third level looks like this:

However, the output of the down-scaling pass looks like this:

As if the sampler didn't use the mipmap at all. Don't get distracted by coloured object instead almost white ones. I lowered the sphere brightness a bit while investigating the bug.

Even if I choose the mipmap level explicitly

float4 vColor = s0.SampleLevel(LinearSampler, Input.Tex, 3);

it changes nothing :-/

RenderDoc also says "LOD Clamp 0-0" for the used sampler. What is it? Couldn't this be the problem?

DirectX details

Samplers

D3D11_SAMPLER_DESC descSampler;
ZeroMemory(&descSampler, sizeof(descSampler));
descSampler.AddressU = D3D11_TEXTURE_ADDRESS_CLAMP;
descSampler.AddressV = D3D11_TEXTURE_ADDRESS_CLAMP;
descSampler.AddressW = D3D11_TEXTURE_ADDRESS_CLAMP;

descSampler.Filter = D3D11_FILTER_MIN_MAG_MIP_LINEAR;
mDevice->CreateSamplerState(&descSampler, &mSamplerStateLinear);

descSampler.Filter = D3D11_FILTER_MIN_MAG_MIP_POINT;
hr = mDevice->CreateSamplerState(&descSampler, &mSamplerStatePoint);

...are set right before rendering the screen quad

ID3D11SamplerState* aSamplers[] = { mSamplerStatePoint, mSamplerStateLinear };
mImmediateContext->PSSetSamplers(0, 2, aSamplers);

...and used within the down-scaling PS shader

SamplerState PointSampler  : register (s0);
SamplerState LinearSampler : register (s1);

Texture2D s0 : register(t0);

float4 Pass1PS(QUAD_VS_OUTPUT Input) : SV_TARGET
{
    return s0.Sample(LinearSampler, Input.Tex);
}

Texture

D3D11_TEXTURE2D_DESC descTex;
ZeroMemory(&descTex, sizeof(D3D11_TEXTURE2D_DESC));
descTex.ArraySize = 1;
descTex.BindFlags = D3D11_BIND_RENDER_TARGET | D3D11_BIND_SHADER_RESOURCE;
descTex.MiscFlags = D3D11_RESOURCE_MISC_GENERATE_MIPS;
descTex.Usage = D3D11_USAGE_DEFAULT;
descTex.Format = DXGI_FORMAT_R32G32B32A32_FLOAT;
descTex.Width  = width;
descTex.Height = height;
descTex.MipLevels = 0;
descTex.SampleDesc.Count = 1;
device->CreateTexture2D(&descTex, nullptr, &tex);

...it's render target view

D3D11_RENDER_TARGET_VIEW_DESC descRTV;
descRTV.Format = descTex.Format;
descRTV.ViewDimension = D3D11_RTV_DIMENSION_TEXTURE2D;
descRTV.Texture2D.MipSlice = 0;
device->CreateRenderTargetView(tex, &descRTV, &rtv);

...it's shader resource view

D3D11_SHADER_RESOURCE_VIEW_DESC descSRV;
ZeroMemory(&descSRV, sizeof(D3D11_SHADER_RESOURCE_VIEW_DESC));
descSRV.Format = descTex.Format;
descSRV.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
descSRV.Texture2D.MipLevels = (UINT)-1;
descSRV.Texture2D.MostDetailedMip = 0;
device->CreateShaderResourceView(tex, &descSRV, &srv);

Explicit generation of mipmap is called after the scene was rendered into the texture and another texture was set as a render target.

ID3D11RenderTargetView* aRTViews[1] = { mPass1Buff.GetRTV() };
mImmediateContext->OMSetRenderTargets(1, aRTViews, nullptr);

mImmediateContext->GenerateMips(mPass0Buff.GetSRV());

ID3D11ShaderResourceView* aSRViews[1] = { mPass0Buff.GetSRV() };
mImmediateContext->PSSetShaderResources(0, 1, aSRViews);

The code is compiled in debug and the D3D device was created with D3D11_CREATE_DEVICE_DEBUG flag and I get no runtime errors on the console.

I found the problem. By zeroing the sampler description I implicitly set LOD bounds to 0. After adding

descSampler.MinLOD = 0;
descSampler.MaxLOD = D3D11_FLOAT32_MAX;

the mipmap works as expected. Thank you for your attention :-)


Rotation matrix for a 3D object in space This is the follow-up question from here: Minimum requirements to uniquely represent a 3D object in space

Assume I have 3 original points in a 3D object (in 3D space) as A1=<xa,ya,za>, A2=<xa,ya,za>, and A3=<xa,ya,za> (assume we have all the requirements mentioned in our previous question). The 3D object is moved and rotated in the 3D space, and the new destination points become B1=<xb,yb,zb>, B2=<xb,yb,zb>, and B3=<xb,yb,zb>.

What is the formula for the transformation matrix? Basically, I need a matrix that if applied to all points of the origin object, I get the displaced object.

Skimming the math.  Starting with a quaternion $Q=w+\left(x,y,z\right)$ then we can rotate $\mathbf{v}$ by:

$$\mathbf{v}' = Q\mathbf{v}Q^{-1}$$

and if $Q$ is unit magnitude this reduces to:

$$\mathbf{v}' = Q\mathbf{v}Q^*$$

To create a matrix we need to apply the rotation to the basis set to form our three equations:

$$
\mathbf{x} = \left(1,0,0\right) \\
\mathbf{y} = \left(0,1,0\right) \\
\mathbf{z} = \left(0,0,1\right) \\
$$

which expanded and reduced gives:

$$
\mathbf{x}' = Q\mathbf{x}Q^{-1} = \left(1 - 2 \left(y^2+z^2\right), 2\left(xy+wz\right), 2\left(xz-wy\right) \right) \\
\mathbf{y}' = Q\mathbf{y}Q^{-1} = \left(2 \left(xy-wz\right), 1 - 2\left(x^2+z^2\right), 2\left(wx+yz\right) \right)  \\
\mathbf{z}' = Q\mathbf{z}Q^{-1} = \left(2 \left(wy+xz\right), 2\left(yz-wx\right), 1 - 2\left(x^2+y^2\right) \right)
$$

Sticking to the math convention of column vectors, then we shove the three equations into the first three columns and to add a translation by $\left(t_x,t_y,t_z\right)$ we shove that into the last column giving:

$$
{
\left( \begin{array}{ccc}
1 - 2 \left(y^2+z^2\right) & 2\left(xy-wz\right)       &  2\left(xz+wy\right)  & t_x\\
2 \left(xy+wz\right)       & 1 - 2\left(x^2+z^2\right) &  2\left(wx-yz\right)  & t_y \\
2 \left(wy-xz\right)       & 2\left(yz+wx\right)       & 1 - 2\left(x^2+y^2\right) & t_z \\
0 & 0 & 0 & 1
\end{array} \right)
}
$$


What is exactly the third component in homogeneous coordinate system? I was reading the wiki article about homogeneous coordinates , I learned that it has it's advantages when it comes to performing affine transformation, since you can represent it only matrices. But I couldn't understand what is the additional third component compared to  Cartesian coordinates.   In the image below  we see additional 1(which  I am trying to understand)  added to convert it to  homogeneous coordinates.

The relationship between standard coordinates $(x,y)$ and homogeneous coordinates $(X,Y,Z)$ is $x = X / Z, y = Y/Z$. Homogeneous coordinates are a type of projective coordinates. All points on the line $(Z x, Z y, Z)$ are equivalent to the point $(x,y)$ in 2D (the same thing can be applied for 3D). So technically 2D points are represented as 3D lines passing though the origin, and 2D lines are represented as 3D planes passing through the origin.
In computer graphics the above interpretation is largely irrelevant, however, since homogeneous coordinates are used mostly for convenience. A translation is of the form $x' = x+t_x, y' = y+t_y$. If one uses homogeneous coordinates, then this can be achieved through $X' = 1 \cdot X + 0 \cdot Y + t_x \cdot Z$, $Y' = 0 \cdot X + 1 \cdot Y + t_y \cdot Z$, and $Z' = 0 \cdot X + 0\cdot Y + 1 \cdot Z$.  If you set $Z=1$ then you get exactly the expression for translation, and you can use the matrix induced from the 3 equations above. For directions you can set $Z=0$ so that translation would not be applied to those.
In 3D the homogeneous coordinates $W$ also plays a role in the perspective projection, since the projection matrix modifies it to hold the relevant value by which the other coordinates must be divided, and the division is typically carried out by the hardware.


Machine Learning and Computer Graphics Is there currently anyone working on a link between these two fields? I have a hunch that, due to the repetitive nature of sampling, there could be a good bit to gain in the area of ray tracing using Machine Learning techniques, but I'm interested in finding out more and getting involved if there's a group out there doing this.

A group from UC Santa Barbara published the Siggraph 2015 paper A Machine Learning Approach for Filtering Monte Carlo Noise describing a technique which trains a neural network to select filtering parameters for path tracing. The paper details the structure of the neural network, the filter parameters used and how several secondary features are pre-computed to feed the neural network.


Why smoothed meshes in 3D studio end up with the same number of vertices/triangles? How then can they be smoothed with the same geometry? I am trying to understand why meshes that are smoothed in 3D studio (Modifiers/Smoother) end up having the same amount of vertices/faces before or after that process, as well as the exact same geometry.

In the following example, both meshes have 32 vertices and 60 faces.

Although I have experience working with programming (c++ and c#), I am quite a beginner with computer graphics. Thus my expectation was that the smoothed look of a smoothed mesh would require additional vertices, i.e. subdivision of triangles, in order to end up having a more detailed final mesh. However, that seems to be not the case.

Therefore I ask:

1) how is the smoothing possible without increasing the detailing of the mesh geometry?

2) does the smoothing at least increase the memory allocated/used for storing the new mesh in comparison to the original one?

Explanations are super welcome, but references (academic or not) are also appreciated.

Smooth in this case just makes the surface normals at vertices point the same way, when interpolated it looks smooth. Meshsmooth would add vertices.

1) how is the smoothing possible without increasing the detailing of the mesh geometry?

Human eyes cant actually see curvature except on the edges of objects. All they can do is approximate the smoothness and process the gradient slope. So having a continuous field does give a air of smoothness. The eye however is extremely sensitive to abrupt changes in color, and interprets that as a hard crease.

By interpolating the vertex normals your surface will get the appearance of smooth flowing. Since this normal is used to calculate the final reflected color you get a smooth color field.

Image 1: a flat shaded normal versus the normals of a smooth interplation. The black normal's lie on a vertex. The colored ones are interpolated.

There is nothing that says we need to do a linear interpolation. In fact by perturbing the normals we can cause the flat surface to change appearance. This is how bump mapping and normal mapping works. The effect can be convincing unless the surfaces edge plays a too big part in which place the illusion breaks.

Image 2: a flat shaded surface (back), Smooth shaded (middle) and a mapped smooth normal. The illusion of a wavy surface breaks because the edge plays so prominent part in the image, you could instead increase the normals

2) does the smoothing at least increase the memory allocated

Hard to say definitive things about the underlying graphics engine. The normals need to be emitted to the graphics card anyway, most likely this data is cached, but could be calculated on the fly (in both cases).

Since Max uses smoothing groups it seems to me that memory usage is constant regardless. Hard to say, even if its not cached then it wouldn't make a big difference. It makes the shader tiny a bit more complicated, but only just most likely this complexity is present use it or not.


Screen space coordinates to Eye space conversion I'm trying to figure out how to reverse Eye space -> projection -> clip(divide by w) -> screen space conversion.

Basically, I want to get Screen space (still have z value for z-buffer) -> undivide? by w -> inverse projection.

I found the inverse matrix for projection but can't figure out how to get the coordinate "undivided" by w.

Thank you!

Object Space → World Space → Eye Space → Clip Space → Normalized Device Space → Window Space

You get from Object Space to World Space by multiplying by the "World
Matrix" (This may actually be multiple matrices depending
on the parent-child object hierarchy of the scene.  Compose them all together to get a single "World Matrix" for a given object.)
You get from World Space to Eye Space by multiplying by the "View Matrix".
You get from Eye Space to Clip Space by multiplying by the "Projection Matrix".
You get from Clip Space to Normalized Device Space by performing the "perspective divide" (divide by w in clip space coordinates).  This is a non-linear transformation requiring a divide, and is not represented by a matrix.  Information can be destroyed in this step.  If w is divided by w you just get 1, and there's no way to recover the original w.

You get from Normalized Device Space to Window Space by performing the Viewport and Depth Range transform. (Simple linear relationship)

The World Matrix, View Matrix, and Projection Matrix are all just 4x4 matrices and can be inverted to get the reverse transformation.

So, to go backwards...

You get from Window Space to Normalized Device Space by performing the inverse of the Viewport and Depth Range transform.
You can get from Normalized Device Space to Clip Space by performing the inverse of the perspective divide (multiply by w).  This is a non-linear transformation requiring a multiply, and is not represented by a matrix.  In general this operation doesn't magically recover 3D data from 2D data.  You must know something about the depth externally to be able to recover the 3D position of a point on the 2D screen.
You can get from Clip Space to Eye Space by multiplying by the inverse of the "Projection Matrix"
You can get from Eye Space to World Space by multiplying by the inverse of the "View Matrix".
You can get from World Space to Object Space by multiplying by the inverse of the "World Matrix".

May I suggest reading this explanation of the OpenGL Transformation.


What is the basic difference between Radar Screen, CRT and Oscilloscope displays? What is the basic difference between Radar Screen, PC CRT Monitor, and, Oscilloscope displays?

All of them use CRT. So, what is the difference?

At many places in CG hardwares and algorithms you have the choice between vector description and raster description. Oscilloscops, radar, lasers shows, plotting tables, laser cuts directly follow lines and curves; these are the vectorials devices. LCD, inkjet and laser printers, stereolithographic 3D printing, follow pixels and voxels; these are the raster devices. The CRT, but also some laser shows, use the oscilloscope technology to implement raster: the "curve" is a repeated Z narrowly sweeping the screen line after line, with intensity modulation making the "pixels". Radars are indeed between this and pure vectors since rotation angle is swept. Details about various forms of radar display and oscilloscope is available on the wikipedia page.

( About representations, the vector/raster dichotomy corresponds to inkscape/illustrator vs gimp/photoshop, 3D meshes vs 3D voxels, list of lobes vs angular or SH decomposition, Lagrangian animation vs Eulerian animation, etc. )


calculating size of rectangle which fully obscures a sphere I am learning GLSL and want to raytrace a sphere. I am rendering rectangles which sit between the camera and sphere, and do a ray-sphere intersection test in the fragment shader. It almost works:

As you can see, the rectangles are a little too small. Here is how I calculated them.

Point $C$ is the centre of the rectangle, a distance $R$ from the radius of the sphere. Taking basis vectors $i,j$ I can construct the corners as $C+(\pm i, \pm j)$

Point $C$ is a fraction $1-\frac{R}{||V||}$ along the vector $V$ from the camera.

I can direct $i$ and $j$ given the vector $V$ and the "up" vector.

How long should $i$ and $j$ be? By similar triangles, they should have length $R\times (1-\frac{R}{||V||})$.

Here is the code (I'm new to GLSL so please ignore bad habits, for now I just want to get the geometry right).

A vertex array containing four $(\pm 1, \pm 1, 0)$ vectors forms a triangle strip.

The vertex shader:

#version 300 es
in  vec2 position;
out vec3 vs_fs_position;

uniform vec3 i;
uniform vec3 j;
uniform vec3 centre;

uniform mat4 pvm; // projection * view * model

void main ()
{
    vec3 p = centre + position.x * i + position.y * j;

    vs_fs_position = p; // world space coordinate sent to fragment

    gl_Position = pvm * vec4 (p, 1); // to screen space
}

The fragment shader

#version 300 es
precision mediump float;
out mediump vec4 out_colour;
in  mediump vec3 vs_fs_position;

uniform mediump vec3 camera_position;
uniform mediump vec3 camera_to_sphere;
uniform float radius;

void main ()
{
    vec3 direction = vs_fs_position - camera_position;

    float A = dot (direction, direction);
    float B = 2.0 * dot (direction, camera_to_sphere);
    float C = dot (camera_to_sphere, camera_to_sphere)
            - radius * radius;

    float det = B * B - 4.0 * A * C;

    if (det < 0.0)
        out_colour = vec4 (0.0, 0.0, 0.0, 1.0);
    else
        out_colour = vec4 (0.5, 0.5, 0.5, 1.0);

}

Some C++ to draw it

shader .camera_position = m_camera_position;

const float EPSILON = 0.0;

for (auto & s : SPHERES)
{
    auto R = s .position - m_camera_position;

    float ratio = 1.0 - (s .radius + EPSILON) / length (R);

    float width = ratio * s .radius;

    auto & up = m_camera_up;

    const float W = width * (1 + EPSILON);

    glm :: vec3 i = normalize (cross (up, R)) * W;
    glm :: vec3 j = normalize (cross (R,  i)) * W;

    shader .i = i;
    shader .j = j;

    shader .centre = s .position;
    shader .radius = s .radius;

    shader .camera_to_sphere = s .position - m_camera_position;

    glDrawArrays (GL_TRIANGLE_STRIP, 0, 4);
}

If it runs with EPSILON=0.5, I get this output.

Theoretically, I should be able to get a perfect fit with EPSILON=0, that way I can be confident it will work at all scales/positions.

What's wrong with my calculation of the width parameter?

How long should $i$ and $j$ be? By similar triangles, they should have length $R\times (1-\frac{R}{\lVert V\rVert})$.

I believe this is where you went wrong. If you draw the 2D version of this on a sheet of paper, it will be obvious that the edge of your triangle which touches the side of your square goes through the sphere. More so as the camera comes closer.

That line needs to be tangent to the sphere. So your large similar triangle will have its right angle at the contact point with the sphere, not at the sphere's center as you did. If my trigonometry is not too rusty, this gives

$\frac{R}{\sqrt{\lVert V\rVert^2-R^2}} \times \left(\lVert V\rVert-R\right)$


GL_DEPTH_CLAMP has no effect (reverse z) In an effort to improve cascaded shadow maps, I have looked into using GL_DEPTH_CLAMP and moving the near and far plane just around the actual view frustum instead of the global bounding box. But the use of GL_DEPTH_CLAMP appears to have no effect and the near plane clips the geometry.

I use reverse z as per Reversed-Z in OpenGL:

The tl/dr version is:

Change clip control to 0/1

glClipControl(GL_LOWER_LEFT, GL_ZERO_TO_ONE);

Use a floating point depth buffer.
Clear the depth buffer to 0

glBindFramebuffer(GL_FRAMEBUFFER, fbo);

glClearDepth(0.0f);
glClear(GL_DEPTH_BUFFER_BIT);

Change depth test to grater

glDepthFunc(GL_GREATER);

Change the projection matrix to match

The reverse z works really fine, but apparently it messes with GL_DEPTH_CLAMP. If I switch off reverse z GL_DEPTH_CLAMP appears to work as designed; but many other bits in the code get messed up. (It's just not switchable anymore.)

If you look at the documentation of GL_DEPTH_CLAMP it states the following:

  If enabled, the -wc ≤ zc ≤ wc plane equation is ignored by view volume clipping (effectively, there is no near or far plane clipping).

My guess is that GL_DEPTH_CLAMP's implementation is just not compatible with reverse z. But I feel like I am missing something. Any idea on how to get GL_DEPTH_CLAMP to working with reverse z?

See the following image of the depth buffers:

As you can see in the the first cascade, the near plane clips the geometry, it should be all white.

It is compatible, but instead of clipping it now clamps (hence the name). If the depth of your primitives is outside the [0,1] range it ends up behaving as if you have no depth-test at all. The only way to mitigate that is to either scale down the depth range so it fits in [0,1] or to use GL_NV_depth_buffer_float to disable clipping and clamping completely.


Path tracing with Next event estimation and area lights auto occlusion I am working my way through my first path tracer and I am currently designing how to deal with area lights. I have two questions:

Each object in my scene can have a Light attached to it, this is how I represent Area Lights. Given this I have two questions:

1) Shadow ray: When doing shadows, if I use a blind IsOccluded function that I use successfully for point lights, I will obviously have always an "occluded" result, as the light vector by construction ends up at the light source itself which is an object in my scene and therefore will be hit by the shadow ray.

To rephrase, how do I deal nicely with the light being an object in my scene when casting the shadow ray for that light?

I would like to keep my IsOccluded function simple, returning only a boolean and not a position/interaction record. At first glance PBRT manages to do so, but I don't understand how they get around the problem. Am I forced to use a different scene intersection function? If so, what is the most graceful way to solve the problem?

2) Secondary rays: If I have more than one bounce, I often see this black dots appearing (even if I disable shadows). I know this is hard without more context, but I wonder if someone has encountered something like this or can guess:

With multiple bounces

With only first hit

I thought it could be related to the way I sample the sphere, but if that was the case, shouldn't I get black dots even with only one event? I think these are NaNs, will double check, but if you know common scenarios when this happens it'd be appreciated. I am guessing I am trying to lit the light with the light itself, causing a NaN that gets then propagated. Does it make sense? I am more interested to understand the cause than how to fix this.

3) Bright pixels: Somewhat unrelated? (I can open separate question), but I get lots of unnaturally bright pixels all over the place if I have more than 2 bounces. What is the cause of this? How do I fix this?

Also, do you have any suggestion on how to handle area light sources that I am clearly missing here?

Thank you and sorry for the probably confused question, I am happy to edit it if you have suggestions.

I am no expert in offline renderers, but I'll give it a go until someone else comes with a better answer :)
Shadow rays intersecting area light
It is very common to assign an extent to your rays. How to do so? Think about the parametric formula of a ray:
$$R(t) = O + t\vec{D}$$
it should immediately jumps out that how far $R(t)$ can be from the origin $O$ depends on how much we can travel in the direction $\vec{D}$, and this travelling is clearly governed by the parameter $t$.
Your problem here is that you don't want to let the ray travel indefinitely, but you want to stop just before it hits the source of the light, because otherwise, as you correctly noticed, everything will result occluded.
So how do you limit how much your ray can travel? You just have to limit the maximum value that $t$ can take. You have two option:

Probably the most intuitive one is to compute at what $t_{light}$ your light source lies. By using the same parametric formulation of the ray, if $L_p$ is the point you've chosen on the area light, then $t_{light}$ will be:

$$t_{light} = \frac{L_{p}[i] - O[i]}{\vec{D}[i]}$$
where $[i]$ represents the coordinate of your choice.  Given this, you want to stop your shadow ray to travel just before $t$ reaches $t_{light}$. If it goes over this value, report an unoccluded result.

Now while the option above is probably the first thing that comes to mind, it is wasteful as you'd need to both have $L_p$ around and perform some unneeded arithmetic operations if you have your light vector $\vec{D}$ as unnormalized around. In fact, this is none other than $L_p - O$ and in a case like this, the Light lies at exactly $t == 1$:

$$ O + 1 \times (L_p - O ) = L_p $$
So if you trace your shadow ray with an unnormalized direction $\vec{D}$, you can just set your maximum $t$ to be just under $1$. This is in fact what PBRT does, and as far as I am aware many other renderers.
Black dots (or NaNs)
Nothing to add over @Dan Hulme comment, so I'll just add it here for completeness of the answer:

"I think these are NaNs, will double check, but if you know common scenarios when this happens it'd be appreciated." The common scenario is that you're using floating-point and you have a bug in your code :-) You have to expect that you'll get NaNs during development, so you should either (a) make sure they get filtered out instead of propagated by the filter, or (b) show all NaNs as bright pink so it's immediately obvious when they're there. – Dan Hulme♦

and if I may add, you should expect some inf as well from time to time, so it's always good to account for these two special cases.
"Bright pixels"
With that, I assume you are talking about what is commonly referred to as fireflies. Fireflies avoidance is indeed quite a big topic and probably warrant its own question. However, just briefly.
Fireflies typically occur when you lots of your paths end in areas with low light and very few paths end up in very bright light.
There is lots of material on how to fight fireflies, a short google search will lead to a lot of techniques depending on your case.
Just to name drop a few:

Radiance clamping
Multiple Importance Sampling (especially if in presence of glossy/shiny surfaces)
Denoising techniques
Increment your sample count :-)
etc.

Also, generally increasing the size of the light source tends to help as well. Remember, the goal is always to hit those bright spot with higher probability!
A better solution to the problem is probably to use something like Bidirectional Path Tracing as this will have sub-paths starting from the light source as well, increasing the likelihood of sampling the bright areas.
Another cause of fireflies are caustics, but I don't want to make this answer longer, so if you are interested this is well worth an additional question.


How do I replace a mesh's texture with vertex colours? I have a textured 3D mesh. I want to deduce a 3D mesh without texture, but with an RGB vector color on each node of this 3D mesh. The colors have to be picked from the initial texture. Is there a way to do this ? My question isn't related to art work. It's related to 3D mesh processing.

Thanks

You can sample the texture in your vertex shader and set the color appropriately. You don't say what system you're using, but for something like an OpenGL vertex shader, it would look something like this (this is a little old school, so it may need updating for modern GL) :

uniform sampler2DRect texture;
varying vec4 color;
void main ()
{
    color  = texture2DRect(texture, gl_MultiTexCoord0.xy);

    vec4    newVertex   = vec4(gl_Vertex.xyz, 1.0);

    gl_Position = gl_ModelViewProjectionMatrix * newVertex;
}

Your fragment shader would then just set the color of the fragment to the interpolated color coming from the vertex shader. Something like this:

varying vec4 color;
void main()
{
    gl_FragColor = color;
}


How to get started writing a low-level GPU profiler? I have been studying hardware corporation GPU profilers in recent days (Qualcomm, PowerVR, Intel).  I've noticed that these tools seem to give more low-level details  than the GPU profilers I have used in the past -- XCode's OpenGL ES frame capture and apitrace -- which only listed which OpenGL calls were made and what the state of current resources are.

How do I get started if I want to make a low-level tool that displays things like sampler cache misses and shader assembler code?

For basic GPU timing data, you can use D3D timestamp queries or the equivalent OpenGL timer queries.

Any low-level hardware data like cache misses is going to be extremely vendor-specific. Each GPU vendor has its own custom API or extension for giving access to low-level performance data on its hardware. The APIs vary in how they work, and they don't necessarily all expose the same details. The available data may also vary between different chip models within the same vendor, so you probably need to know a bit about how the hardware works to make sense of it.

Here are links to the relevant APIs for most of the main GPU vendors.

AMD: GPUPerfAPI; see also AMD_performance_monitor
Intel: Performance Counter Monitor (note: it's not clear to me whether this includes access to GPU counters, or only CPU ones); see also  INTEL_performance_query
NVIDIA: PerfKit
PowerVR: PVRScope
Qualcomm: QCOM_performance_monitor_global_mode


Avoiding z-fighting with coincident surfaces When rendering two overlapping co-planar surfaces, a common issue is "z-fighting", where the renderer can't decide which of the two surfaces is closer to the camera, giving visual artifacts in the area of overlap.

The standard solution to this is to give the surfaces a slight offset when designing the model.  Is there any other solution?

If the surfaces are exactly co-planer, your fate is up to the FPU gods; you'll more than likely have Z-fighting. If the triangles are identical and you do the exact same math to each, you will end up with the same Z-values for both. But again, this will only happen if your math operations are identical for both. (Since, in general, FPU operations are not commutative)

One possible solution is to not use the Z-buffer at all. Rather you can use the Painter's Algorithm. Granted, this comes with all the problems of the painters algorithm as well. But, it would solve the Z-fighting.

In your example case of Screen Space Decals (SSDs), the common solution is to use an offset, aka a simple thin cube. See Warhammer 40k's presentation about SSDs for reference. Or Bart Wronski's post which addresses some other issues with decals, but also links to a few other presentations about SSDs


What are sample gaps during scan conversion? In this tutorial,

It is very often the case that triangles are rendered that share
edges. OpenGL offers a guarantee that, so long as the shared edge
vertex positions are identical, there will be no sample gaps during
scan conversion.

1 What exactly are sample gaps during scan conversion?
2 Under what circumstances, will it happen?
3 Does it happen when the line cross the center of the fragment?

Sample gaps will occur if you don't do things "properly".

In OpenGL or D3D, assuming a consistent winding order, if you have triangles  ABC and CBD, then when a sample point - that is a test during scan conversion to determine if a point (read pixel) is inside a triangle - lies exactly on the shared edge BC, then that sample will belong to exactly one of those two triangles.

This avoids both gaps and double filling. The latter would be problematic with, say, translucency or stencil operations.


How to record N-body simulation as a video? I have implemented an N-body simulation using the Barnes-Hut optimisation in Python which runs at a not-unacceptable speed for N=10,000 bodies, but it's still too slow to watch real time.

A new frame is generated each time-step, and to display the frame we must first calculate the new positions of the bodies, and then draw them all. For N=10,000, it takes about 5 seconds to generate one frame (this is waaay too high as Barnes-Hut should be giving better results). The display is done through the pygame module.

I would thus like to record my simulation and replay it once after it's done at a higher speed.

How can I accomplish this without slowing down the program or exceeding memory limitations?

One potential solution is simply to save the pygame screen each timestep, but this is apparently very slow.

I thought also about storing the list of positions of the bodies generated each time step, and then redrawing all the frames once the situation finishes. Drawing a frame still takes some time, but not as much time as it takes to calculate the new positions.

user1118321's suggestion is the correct one but i'll elaborate a bit.

You run your simulation giving it a fixed time step between frames.  This time step would be 1/30th of a second (33.3333 ms) for instance if you wanted a 30 fps movie.

You let your simulation calculations and your rendering run, taking as long as they need to, and then write the rendered frame out to disk, with the frame number in the name (like frame0.bmp, frame1.bmp, etc).

After you have all these frames, you can use ffmpeg to combine them into a video of the desired frame rate.

If you want audio in your video (like sounds when things collide), you can actually keep an audio timeline (what audio events happened on which frames, or at what points in time) while doing your simulation, and then after your rendering is finished, you can generate the audio stream that goes along with your video, which ffmpeg will gladly combine with your video.

I've used this on a few projects in times past and it's worked great.  I also wrote up some more details about this process a while back:

Recording lagless demo videos of a laggy game | The blog at the bottom of the sea


Generating the end caps of a cylinder using triangle strips for use in a geometry shader I know how to generate the walls of a cylinder with triangle strips and the caps with triangle fans, but since I am working inside a geometry shader, I can only output triangle strips, so I am kind of lost here.

This here is my current code, which only generates the hull of the cylinder based on a line segment:

#version 330 core

    layout(lines) in;
    layout(triangle_strip, max_vertices = 32) out;

    uniform mat4 projection;
    uniform mat4 view;
    uniform mat4 model;

    in Vertex
    {
        vec4 color;
        float width;
    } vertex[];

    out vec4 vertex_color;
    out vec3 normal_vector;

    vec3 createPerp(vec3 p1, vec3 p2)
    {
        vec3 invec = normalize(p2 - p1);
        vec3 ret = cross( invec, vec3(0.0, 0.0, 1.0) );
        if ( length(ret) == 0.0 )
        {
            ret = cross(invec, vec3(0.0, 1.0, 0.0) );
        }
        return ret;
    }

    void main()
    {
        mat4 mvp = projection * view * model;

        vec3 axis = gl_in[1].gl_Position.xyz - gl_in[0].gl_Position.xyz;

        vec3 perpx = normalize(createPerp(gl_in[1].gl_Position.xyz, gl_in[0].gl_Position.xyz));
        vec3 perpy = cross(normalize(axis), perpx);

        float r1 = vertex[0].width / 1000.0;
        float r2 = vertex[0].width / 1000.0;

        int segs = 16;
        for(int i=0; i<segs; i++) {
            float a = i/float(segs-1) * 2.0 * 3.14159;
            float ca = cos(a); float sa = sin(a);
            vec3 normal = vec3( ca*perpx.x + sa*perpy.x,
                            ca*perpx.y + sa*perpy.y,
                            ca*perpx.z + sa*perpy.z );

            vec3 p1 = gl_in[0].gl_Position.xyz + r1*normal;
            vec3 p2 = gl_in[1].gl_Position.xyz + r2*normal;

            gl_Position = mvp * vec4(p1, 1.0);
            vertex_color = vertex[0].color;
            normal_vector = normal;
            EmitVertex();

            gl_Position = mvp * vec4(p2, 1.0);
            vertex_color = vertex[0].color;
            normal_vector = normal;
            EmitVertex();
        }
        EndPrimitive();
    }

Is there some simple way to integrate the generation of the endcaps here?

If the end of your cylinder is convex, with vertices $V_0, V_1, V_2,...V_{N-1}$ then just replace your fan order (i.e. as above) with this strip $V_0, V_1, V_{N-1}, V_2, V_{N-2}, V_3...$, i.e. just zig-zag across the polygon.


Bitmap-images or SVG I'd like to create a GUI in my first Unity-game (2D). I want my GUI appear sharp in all resolutions (e.g. 1920x1200 or 3840x2160).

The problem I am facing now is that I don't know if I should save my bitmap-images of my GUI in the highest resolution (so that it scales down when my game is played with a lower resolution) or should I use SVGs instead.

This may be a stupid question but it's the first time that I create a GUI in Unity and I'm really struggling with this kind of problem...

Neither of your suggestions is a solution for crisp 2D graphics at all resolutions. SVGs are great and infinitely scalable, but since you don't know where the pixel boundaries will lie in the final render, you can't get crisp edges at all resolutions. For example, say your GUI is a red box on a green background. If the edge between the red and the green falls in the middle of a pixel, and the SVG renderer does anti-aliasing, you'll get a row or column of half-red, half-green pixels, which will look like mush. If the renderer doesn't anti-alias, you instead get the edge in the wrong place, which might be fine for a single rectangle, but for a repeating pattern each instance of the pattern might come out a different width, which is just as noticeable.

Downscaling a bitmap to the display resolution is just as bad. Again, imagine your green-red edge is on a pixel boundary that will not be a pixel boundary in the downscaled bitmap. You're again going to get a row or column of pixels that are a mushy red-green mix. The lowest resolutions are the most important to have crisp edges, but they're going to be the worst-affected.

You can make either one of these techniques work somewhat, if you can accept a restriction: instead of handling any resolution, you're willing to handle a set of resolutions, ideally powers of two apart (e.g. icon sizes of 512x512, 256x256, 128x128, etc.) When you're designing your UI, draw a large bitmap (512x512) or a vector image, but use guides in your editor to show the minimum size (e.g. 64x64). Make sure all your important edges line up with those guides, but you can still use curves, gradients etc. That way, you know you won't get the blurry mush at low resolutions, but you can still get the benefit of smooth diagonals and curves at high resolutions. This is a quick way to design scalable UI.

Mobile OSes (which are designed for a range of screen dpis) use a similar technique, but with two differences. First, they "bucket" the screen's actual dpi to the nearest "standard" dpi. This means a UI might show slightly larger or smaller on different screens, but you know exactly how many pixels will be in each icon or widget. To make this work out, they don't limit the resolutions to powers of two, but have some intermediate ones as well.

Second, they let designers specify a different bitmap for each resolution. This means that you can make a higher-detail design for a higher-dpi screen, without it being too busy on a low-dpi screen. (For example, you can use a fine checkerboard or ruler pattern, and have it show up correctly at each resolution.) You could consider doing this too, but it makes designing and managing the bitmap assets a little more time-consuming.

If your buckets are powers of two apart, you can use mipmaps to store the images (instead of using the built-in mipmap generation), whether you're creating different images for each resolution or just downscaling. That way you avoid the runtime memory bandwidth cost of downscaling.


Polarising reflection and refraction for a wrap around water surface I want to render realistic images of water in an orbiting space habitat. The image does not need to be generated in real time, although I wouldn't want it to take weeks either. I'm looking for an approach that can generate realistic images in hours or days.

The habitat is cylindrical with the curved inner surface being the living space. Rotation of the cylinder about it's axis provides an approximation of gravity. I'm not looking for details of simulating the physics of this, just the rendering of an image.

The specific aspect I want to know about is polarisation. Light reflected from the surface of water is polarised, leaving the light that passed into the water polarised perpendicularly to the reflected light. Ignoring this effect and simply modelling the proportions of light that are reflected and transmitted works reasonably well when there is only one water surface, but if the cylindrical habitat has bodies of water that take up large proportions of the curved surface then a given ray will make multiple reflections at a wide range of different angles. This means the proportion of light that is reflected will depend on the polarisation angle previously applied to it.

Are there existing approaches that incorporate such effects that could give realistic images of multiple reflections from a curved water surface? They would also need to model refraction with polarisation. The water will be shallow in places so I'm expecting polarised refraction to influence the results.

If not, could I adapt an existing ray tracer or would this need an approach starting from scratch?

I'm looking for realism in order to discover unexpected effects, not just to pass as realistic to a casual observer. Obviously most observers (including me) won't know the effects to look for since they aren't familiar from everyday life, so I'm looking for "reasonably physically correct" rather than just "convincing".

The most commonly suggested method seems to be Mueller calculus, which boils down to tracking the Stokes parameters of a light ray to represent the polarization of light transmitted along that ray. A ray might be unpolarized—Stokes parameters of (1, 0, 0, 0)—or it may be circularly or linearly polarized in various directions, which is a property of light in the aggregate. At the surface light is scattered according to the polarization and the Stokes vector is propagated by multiplying it by the Mueller matrix of the surface.

Here's a writeup by Toshiya Hachisuka about ray tracing while tracking light polarization. It seems like a good introduction, and there are several references that seem promising. The article argues for direct tracking of the polarization state of the ray: instead of an aggregate representation, individually tracking the direction and frequency of the two harmonic oscillations of a given light ray. This may have the disadvantage that you need more samples to reproduce the polarization effects accurately, but it may be able to reproduce more effects (in the article, thin-film interference).


Remove orphaned geometry after Constructive Solid Geometry operation I have used a library to perform a Constructive Solid Geometry difference operation, which has left me with orphaned geometry as well as imperfections.

Is there an approach/algorithm that will allow me clean the geometry so all that remains is the two cuboids?

I've not done CSG per se, but I did work on a, not-unrelated, problem of triangulation of arbitrary, self-intersecting, multi-contoured polygons. If you don't get the implicit intersections correct, all sorts of nasty artefacts can result.

Initially, I tried using standard, floating-point maths to compute intersections etc, but this is fraught with precision problems and no amount of trickery of ordering etc eliminated these accuracy issues.

In the end, I replaced all intersection calculations with rational maths (high precision integer numerator and denominators) with sufficient precision that all intersections would be represented exactly. Once the process was completed, then these high precision results could be reduced to back to floating point.

Perhaps a quick way to start would be to use a package such as GNU's GMP which supports arbitrary rational representations.


translate coordinates relative to line segment I need to write a function (in Javascript) that accepts the endpoints of a line segment and an additional point and returns coordinates for the point relative to the start point. So, based on this picture:

function perpendicular_coords(x1,y1, x2,y2, xp,yp)

returns [xp',yp'] where xp' is the distance from (xp,yp) along a line perpendicular to the line segment, and yp' is the distance from (x1,y1) to the point where that perpendicular line intersects the line segment.

What I've tried so far:

function rotateRad(cx, cy, x, y, radians) {
    var cos = Math.cos(radians),
        sin = Math.sin(radians),
        nx = (cos * (x - cx)) + (sin * (y - cy)) + cx,
        ny = (cos * (y - cy)) - (sin * (x - cx)) + cy;
    return [nx, ny];
}

[xp', yp'] = rotateRad(x1,y1, xp, yp, Math.atan2(y2-y1,x2-x1));

I don't really understand the function; got it from https://stackoverflow.com/a/17411276/1368860

I got a good answer on StackOverflow: https://stackoverflow.com/questions/41701713/translate-coordinates-relative-to-line-segment/41702475#41702475 from https://stackoverflow.com/users/1579780/giladd:

function perpendicular_coords(x1, y1, x2, y2, xp, yp) {
  var dx = x2 - x1,
      dy = y2 - y1;

  //find intersection point
  var k = (dy * (xp-x1) - dx * (yp-y1)) / (dy*dy + dx*dx);
  var x4 = xp - k * dy;
  var y4 = yp + k * dx;

  var ypt = Math.sqrt((y4-y1)*(y4-y1)+(x4-x1)*(x4-x1));
  var xpt = distance(x1, y1, x2, y2, xp, yp);
  return [xpt, ypt];
}

// Distance of point from line
function distance(x1, y1, x2, y2, xp, yp) {
  var dx = x2 - x1;
  var dy = y2 - y1;

  return Math.abs(dy*xp - dx*yp + x2*y1 - y2*x1) / Math.sqrt(dy*dy + dx*dx);
}


How to count DPI of an image (challenge)? a) I have used an online service where you can upload an image.

b) Afterwards, you can select a print size and it gives you information on what is actual DPI based on that print size.

c) When it gives you information if that particular DPI (dots per inch) for that print size is enough quality to print, so basically it outputs "bad", "average" or "good".

I want to built this functionality on my own site, but I lack logic's behind how they count DPI.

This is my two tests where I upload slightly different photos. As site outputs DPI it is easy to try to get how it counts it, but as for different photos it does differently. I just can not grasp the pattern on why it does so.

Here is my two different photos uploaded:

Product: Poster
Size: 1000x1200

I will just explain the first value - 8x10 -> 1200 / 10  -  you get DPI by subtracting photo height with print size height.

8x10 -> 1200 / 10   ->>> why this DPI is counted differently from other image?
10x10 -> 1000 / 10
12x12 -> 1000 / 12
12x16 -> 1200 / 16  ->>> why this DPI is counted differently from other image?
14x14 -> 1000 / 14
16x16 -> 1000 / 16
12x18 -> 1200 / 18
18x18 -> 1000 / 18
16x20 -> 1200 / 20  ->>> why this DPI is counted differently from other image?
18x24 -> 1200 / 24  ->>> why this DPI is counted differently from other image?
24x36 -> 1200 / 36

Product: Poster
Size: 1000x1333

8x10 -> 1000 / 8    ->>> comparing to this value
10x10 -> 1000 / 10
12x12 -> 1000 / 12
12x16 -> 1000 / 12  ->>> comparing to this value
14x14 -> 1000 / 14
16x16 -> 1000 / 16
12x18 -> 1333 / 18
18x18 -> 1000 / 18
16x20 -> 1000 / 20  ->>> comparing to this value
18x24 -> 1000 / 18  ->>> comparing to this value
24x36 -> 1333 / 36

Just do not get it why for a very similar image, counting of the same print sizes are different for particular values?

Any ideas?

They are not "very similar images". They have quite different aspect ratios. The first one has a 5:6 aspect ratio, while the other has a 3:4 ratio.

If you try to print a 5:6 image at 8x10, what you find is that if the width is 8, then a 5:6 image would have a height of 9.6, which is less than 10. So that means the 5:6 image can fit into the 8x10 region with its full width; some of the height will go unused.

However, a 3:4 aspect ratio image at 8x10 doesn't do that. If the width is 8, then the 3:4 aspect ratio would require a height of 10.6. That would exceed the maximum available height, so the width cannot be full width. Instead, it uses the full height of 10, and the width used will only be 7.5.

You will find that this is the case for all of the ones where you noted such a difference.

The "DPI" its computing is essentially the number of pixels in the dimension that isn't the maximum. For the 3:4 image at 8x10, it takes the image width (1000) and divides by the area width (8). For the 5:6 image at 8x10, it takes the image height (1200) and divided by the area height (10).

This number only makes sense if we assume that the image is being scaled independently in width and height to fit the page. That is, the aspect ratio of the image is not being preserved when printed. So a 5:6 image is being transformed into a 4:5 aspect ratio image when printed at 8x10.

Since each dimension is being scaled differently, each dimension has its own DPI. The lowest DPI will be for the dimension being stretched the most. And that's what this formula seems to be computing.

Of course... who cares? Stretching images creates obvious visual distortion, and should pretty much never be done when printing something you actually want to have a paper copy of. So I would declare this computation algorithm to be bogus and move to something that actually makes sense with a reasonable printing methodology.


An idea for an implementation of continuous collision detection of circles So I'm making a 2D pool game in OpenGL (you might've guessed as soon as I've mentioned collision of circles). I'm having a little trouble with collision detection. What I mean by that is that I have an algorithm that currently works but I'm thinking of improving it.

So far I've used discrete collision detection, disregarding the possibilities that it might not work at high velocities. However, I've noticed that I in fact sometimes do need the circles (pool balls) to travel at velocities at which the discrete collision detection might fail. The reason for that is that some of those velocities are quite realistic in a real-life game of pool (I've calculated that an average professional break shot in pool can make the ball travel as fast as six times the ball's diameter per 33 milliseconds which is the time interval that one frame lasts on 30FPS machines). Therefore in every such shot my discrete collision detection is very likely to fail.

This is the idea I've had:

The dashed circles represent the positions of the balls in the previous frame. The lines from them represent the velocity vectors and, obviously, the circles that they're connected to are their positions in the current frame.

One thing I would do immediately is check whether the velocity vectors are collinear in which case there's no need to check for any kind of collision. If they're not collinear, I'd find the intersection point of the line segments represented by the points of previous position and the next position (if they don't intersect I just disregard everything) . After that I'd manually translate the balls "before" the point of intersection (I was too lazy to actually compute where the balls would touch so I just put them somewhere near that point for the purposes of illustration, but it doesn't really matter). After I've done all that I would just let the discrete collision detection algorithm, which I already have, do the work.

Couple of things I'm worried about here:

(1) Since I have more than 2 balls on the table (16 to be precise) can the manual translation step somehow impact eventual collision of one of the balls that have been translated and some other, third ball? It seems to me like it shouldn't, but I'm not sure.

(2) On one hand I'm thinking that this might cause some glitching since I'm translating the balls "back" after I've already drawn them at their positions. On the other hand since this will only be happening at high velocities it might not matter that much. This also causes me to wonder whether I should do draw - collide or collide - draw. That is, should I use the principle I've described here (first draw the balls, then check whether they've been drawn too far) or maybe I should "look ahead", as in check whether the next position could cause problems instead of whether the previous position has caused them?

Not looking for any code, just ideas and thoughts.

Thanks in advance.

I think you should determine the "next collision point" in your whole 16-ball-and-table-border system and adjust your simulation 'Delta t" depending on it. Rendering should be de-coupled from this anyway, or it will show irregular ball movements... .

Some further thoughts:

Collinear velocity vectors do not mean there is no collision - if v1 != v2 ...
In good pool games you can define the 'hit point', where the queue hits the ball. Then "curvy" ball trajectories are generated, etc. etc.
Your system should also consider friction, therefore: v->0 over time. This should also affect your collision calculations.


How to adapt diffuse/GI light mapper to directional light maps? I have a working light mapper that captures global illumination. I evaluate a hemisphere when collecting the data and store the result in the texel. However, I want to support directional lightmaps so that I can more realistically light normal maps at runtime. What strategies are there to adapt my light mapper's final gather operation to capture this data? Is it as simple as performing three hemisphere samples per texel in orthogonal directions? Or should I uniformly sample the hemisphere once, and project the results onto the basis vectors? Or do I sample the hemisphere and project each sample on to the basis.

I've done some tests with the latter - projecting the each sample on the hemisphere onto the three basis vectors - and this works mostly, but I'm not sure about whether the projected samples should be clamped when their dot product is facing away from the basis or not.

When reading all of Valve's HL2 Radiosity Normal Mapping articles, the need to generate three lightmaps for the bases vectors is clear, as are the coordinates for the bases vectors, as is the reconstruction in a shader of the data - this is straight forward enough. See Valve SIGGRAPH06, Valve Tutorial and Peter Houska on Directional Lightmaps).

The complication comes from how to generate those three samples:

For example - from a Valve GDC talk:

  • Traditionally, when computing light map values using a radiosity
  preprocessor, a single color value is calculated

  • In Radiosity Normal
  Mapping, we transform our basis into tangent space and compute light
  values for each vector

Given that I am currently sampling on a hemisphere, do I simply project each sample onto the bases and store those accumulations? If I do that, how do I handle the fact that data within my original hemisphere, when projected onto the bases, could create a projection that actually subtracts energy, given that a sample ray pointing away from the bases gives a negative dot product.

Take a look at Valve's paper on combining Radiosity and Normal Mapping. It had a lot of useful insights.

If I'm not mistaken, it looks to me like you're trying to compute normal mapping using the energy value at the texel that is a result of Radiosity - e.g. it came from all directions through almost-endless bumping around the room across thousands of surfaces.

But your point light, for the normal mapping, has a very specific position, direction, distance and intensity. Meaning, you're combining apples and goats.

What I believe you should do is:
- compute the normal mapping separately for those surfaces
- blend the GI solution with normal mapping
- experiment with the blending factor till it looks 'good enough' for you


Can V-Sync avoid tearing? I think it can't. Am I wrong? 1) Update the front buffer will cause screen tearing, every one knows it.

2) But V-Sync, I think it still get the same issue. Why I get this idea?

2.1) When GPU flips to the back buffer, current screen displays the old image which was in previous front buffer.

2.2) Now GPU sends image from "current front buffer" (previous back buffer) to monitor. This will last 16.7ms (60Hz). During this period, Human eyes will see half "current front buffer" and half "previous front buffer". Is it right?

2.3) If it is right, it would be same as non-V-sync.

3) But, in fact, when I switch "V-sync" on, I can't see screen tearing. Why?

Thanks a lot for the answers.

V-sync means that the GPU won't switch buffer until the current frame is done showing.

When you tell to swap the buffer with V-Sync enabled the actual swap doesn't happen until V-Blank, a small window between frames where nothing is being sent to the screen.

The reason I think you cannot see the update tear is that it moves across the screen during the update. While a non v-sync tear stays in the same place on the screen for 16 milliseconds.


Data structures for tile-based (deferred) rendering Tiled rendering is used in modern mobile GPU architectures to increase the coherency of memory access by subdividing image space into a regular grid of small (e.g., 32x32 pixel) tiles. Information is scarce on the types of data structures used to track the primitives that are associated with each tile, considering that arbitrarily many primitives may overlap any given tile.

From the perspective of a driver developer, what data structures are commonly used to represent the primitive sets that belong to a tile, and are such structures dynamically allocated/resized according to the geometry that overlaps a particular tile?

That blog post that John mentions is a pretty good start (if I do say so myself!), but there's a bit of extra detail that might be helpful.

For the PowerVR architecture, the intermediate data structure -- variously called the primitive list or parameter buffer (PB) -- that stores the per-tile data, after all vertex shading and the tiling process is complete, is actually mostly generated and managed by the hardware, rather than the driver.

The PB's in-memory structures are physically split into two. First, blocks of transformed vertex data, including vertex attributes. The blocks are compressed, and as you can imagine they're just packed and compressed floating point data for the most part. The second in-memory structure is the tiling data, which is effectively a list of lists.

The top level list in that data structure is called a region, and it can encode a set of tiles rather than single tile at a time, for a given primitive block. A region is therefore a set of locations of screen tiles, tile states, and then a list of the compressed blocks that hold geometry in that region. Regions are what the rasteriser works on, and you can imagine that empty tiles are just automatically skipped, although there's a good reason in some cases for the rasteriser to visit empty regions.

The memory the GPU uses for the PB is dynamically allocated in all modern PowerVR implementations. A pointer to that memory is provided by the driver, and the driver, with the GPU's help, will size it as required. That mechanism is a tradeoff between having to reallocate frequently and minimising the amount of allocated PB space.

Modern GPUs try really hard to minimise memory indirection, but walking the PB to feed the rasterisation stage is one of those cases where it's really difficult and there's no other choice. Thankfully the pointer chasing wraps up large blocks that cache well and are streamed into the core.

Other architectures don't work exactly the same as PowerVR, because part of the reason the PB is the way it is in our architecture is to help the fully deferred pixel shading concept we implement, but the general concept applies to all other tilers in the mobile space that I'm aware of.


How can I create a bounding volume hierarchy for constantly moving objects? I'd like to be able to render a large population of small independently moving objects in real time. They may move in a swarm-like manner, but their relative positions will not be coherent - their position may change arbitrarily within a swarm and swarms may break up and reform at any point.

What approach to building a bounding volume hierarchy would best suit this situation? Is there a way to maintain a hierarchy which is sub-optimal but good enough, that only requires a partial update each frame? Or is there a way of building a hierarchy from scratch each frame that is fast enough for smooth animation?

The number of objects will be too large to render without a hierarchy, but for the same reason I expect building the hierarchy to be time consuming.

Following the comment from John Calsbeek, if my focus on bounding volume hierarchies is misguided, and there is a better space partitioning approach for this situation, please answer accordingly. I'm looking for something that can deal with what I describe, including anything I haven't thought of.

Consider using spatial hashing, especially if your objects are similarly sized.

Basically, divide your world into uniformly-sized grid cells (2D and 3D are both valid possibilities depending on the amount of vertical motion). Each update, assign your object to each bin that it overlaps—if the cells are decently sized relative to the objects, most objects should end up in a single bin.

Each bin is inserted into a hash table, with the key being the coordinates of the bin. (You can also think of it as a hash table with multiple values for the same key, and inserting an object once for every cell that it overlaps.)

There's no hierarchy to rebuild in this scheme, which makes it well suited for dynamic scenes. You can still test the cell's dimensions against the frustum or against occluders at a coarse level and discard many objects at once. Also, it's easier to manage this structure incrementally—you can keep the hash table the same from frame to frame and only move objects from one bin to another when they cross the boundary of a cell.


Is there an analogue of OpenGL's ImageLoadStore in DirectX 12? I'm trying to port part of an OpenGL program into an existing DirectX 12 software, at some point in their shaders they use OpenGL ImageLoadStore in order to store information in 3D Textures without having to use an FBO.

Is there something similar in the DirectX 12 API?
Or would I have to render to a custom FrameBuffer?

You should be able to use RWTexture3D.
The hlsl documentation page about RWTexture2D also has sample code that you might find useful.


Why is AO considered Post-Processing? Ambient Occlusion is a physical approximation to occlude ambient light. If ambient light is not considered post-processing, why is AO? Ambient occlusion has the same inputs as regular deferred lighting, plus kernels and a noise map, but essentially, it can be handled the same way. Is there a reason why it's considered post-processing and deferred lighting/shading isn't?

This is a potentially confusing re-use of terminology. Ambient Occlusion is not in itself a post processing effect. Screen Space Ambient Occlusion is a post processing effect using the depth buffer to approximate the effects produced by Ambient Occlusion.

Ambient Occlusion is a relatively expensive global method, while Screen Space Ambient Occlusion is the cheaper alternative for real time animation. Only the latter is considered post processing.


Is a branched discard to be avoided When a simulating a camera, I may run into the situation that a black elliptic border is needed (rather than a rectangular image).

In this case I have the option to discard the fragment in the fragment shader or draw it with a black color. Discarding obviously needs branching, so now I am not sure which option is faster. Generally I assume avoiding branching in shaders is the way to go, but does that hold true for discarding, especially if the branch contains only one call (the discard one)? Or is there no rule of thumb for this case at all and just benchmarking is the way to go?

You can use a stencil instead, This allows the gpu to do an early out test before the fragment shader even starts.

You only have to build the stencil once (possibly with a discard in a simple shader) and you can reuse the stencil in future draws.


How does notch's new graphics engine work? @notch has recently been posting on an engine he is developing. He claims that it is voxel-based and supports deformable terrain, with a render distance of almost 1000 000 voxels and real-time reflections, running at 99fps in WebGL. Even with heavy LOD reduction, such a thing sounds too good to be true. He has not published any source code as of now, so does anyone have an idea how this is done? Is it just many small optimizations or could this be a new approach to real time 3D?

EDIT: He also mentions a "signed density field". What does that mean exactly?

Notch's engine most likely works using volume raymarching in a volume field. This means that you shoot rays that move a certain distance and check whether they are inside and object or not. Once they are, they return the position of the colission and some other data.

You can either advance rays by a set amount per step until you hit something or refine the final collision position by doing a binary search along the last walked ray segment once you hit something. Both of these aproaches have the problem that they can overshoot, missing small geometry, or take a large amount of steps to traverse empty space where there is no geometry to hit.

Binary search can add precision and reduce overshoots somewhat but comes at an extra runtime cost for doing a binary search per ray.

Signed distance fields are essentially 3d arrays  tell you how far away from any surface you are. This allows you to do raymarching with ray step sizes equal to that number and being sure that you wont overshoot, as well as stepping through large empty spaces quite quickly. This greatly enhances performance and reduces overshoots from a regular step size or binary search based approach.

This last technique trades off runtime speed for precompute time and extra memory. So it is not always adequate for rendering exteriors or large scenes. Its memory usage an runtime speed can be further improved by using a hierachical data structure that gets re built every so often with more detail and data density near the camera and less further away from it.

EDIT: Oh it seems i misread density for distance

I really have no idea what a signed density field is. I've never heardthe term before. A quick goolge search throws results that talk about signed distance fields. I would guess it was either a mistake to write that, some obscure tech or something that Notch made up himself.


Can gl_WorkGroupSize be set outside the compute shader? I am migrating code from CUDA to OpenGL compute shaders.
I could not find a way to set the local workgroup size outside the shader; is there a way to set gl_WorkGroupSize in the calling program like one can do for gl_NumWorkGroups?
Thanks in advance

Core OpenGL has no mechanism for this. The ARB_compute_variable_group_size extension provides such a feature. It doesn't allow for indirect compute operations though.

It doesn't seem to have much AMD support though (outside of open source Linux drivers). And Intel support is nil.


Why is accessing textures much slower when calculating the texture coordinate in the fragment shader? When using textures in GLSL, it is best to calculate the final texture coordinates in the vertex shader and hand them over to the fragment shader using varyings. Example with a simple flip in the y coordinate:

// Vertex shader
attribute vec2 texture;
varying highp vec2 texCoord;
// ...
void main() {
    texCoord = vec2(texture.x, 1.0-texture.y);
    // ...
}

// Fragment shader
varying highp vec2 textureCoordinates;
uniform sampler2D tex;
// ...
void main() {
    highp vec4 texColor = texture2D(tex, texCoord);
    // ...
}

If the flip in the y coordinate, or an even simpler operation like adding vec2(0.5) to the texture coordinate is performed in the fragment shader the texture access is much slower. Why?

As a note, e.g. blending two textures, using a weighted sum of them, is much cheaper in terms of time and also needs to be done for each pixel, so the computation of the texture coordinate itself does not seem to be that costly.

What you're talking about is commonly called "dependent texture reads" in the mobile development community. It's an implementation detail of certain hardware, and therefore it really depends on the GPU as to whether or not it has any performance implications. Typically it's something you see brought up for PowerVR GPU's in Apple hardware, since it was explicitly mentioned in both Imagination and Apple documentation. If I recall correctly, the issue basically came from hardware in the GPU that would start to pre-fetch textures before the fragment shader even began running, so that it could do a better job of hiding the latency. The docs I linked mention that it's no longer an issue on Series6 hardware, so at least on newer Apple hardware it's not something that you have to worry about. I'm honestly not sure about other mobile GPU's, since that's not my area of expertise. You should try and consult their documentation to find out for sure.

If you decide to do some Google searches on this issue, be aware that you'll probably find some older material that's talking about dependent texture fetches on older desktop hardware. Basic in the early days of pixel/fragment shaders, the term "dependent texture fetch" referred to using a UV address that relied on a previous texture fetch. The classic example was bump-mapped environment map rendering, where you wanted to use a reflection vector based on the normal map in order to sample the environment map. On this older hardware there was some major performance implications, and I think it wasn't even supported on some very old GPU's. With modern GPU's the hardware and the shader ISA is much more generalized, and so the performance situation is much more complicated.


Ray Tracing Bounding Volume Hierarchies nodes I'm new to computer graphics programming so I apologize in advance if my question is stupid.

I'm trying to implement a BVH for my ray tracer and I looked at some lectures explaining BVH. They all mention that a BVH takes in primitives which are then divided until they end up as nodes of the tree, but when I looked at scratch-pixel, they were talking about having the meshes as nodes. I'm a bit confused here because I'm not sure exactly which is the right way of doing it. Should the nodes contain the meshes as a whole or should I just feed the BVH construction algorithm the primitive triangles (ignoring the fact that they are a part of a mesh by just grouping them into a list/array of primitives)?

Bounding Volumes
A bounding volume is for example an Axis-Aligned Bounding-Box (AABB), a generaal Bounding-Box not aligned with the major axes or a Bounding-Sphere. They differ in

how much memory they take (e.g. a sphere takes only 4 floats, AABB 6 floats)
how good they enclose the object's geometry
how fast are the related computations (mainly intersection tests).

You will find an extensive overview at Wikipedia.
Hierarchy of volumes
So in the hierarchy of bounding volumes, instead of whole meshes you will store only their bounding volumes. You choose a single type, e.g. a bounding sphere, and will use it to build the whole BVH. At the bottom level each bounding volume will probably enclose only a single object. You will build the hierarchy up by enclosing pairs or smaller groups of volumes in a larger volume.
So using spheres you would enclose each object into a bounding sphere at first. Then at the higher levels of the hierarchy, you would always create a new sphere enclosing a few spheres from the previous level which are close to each other. You proceed until a single root sphere encloses all 3D objects in your scene.
There are different strategies to create BVH. But all have in common that complex structures like triangle meshes are hierarchically clustered using very simple entities.


Avoid having an opaque and transparent variant of each pixel shader Transparency can be achieved (or actually approximated) by using alpha-to-coverage (in case of MSAA) or basic alpha/transparency blending using the following approach:

Bind a rasterizer state with a Less-Equal comparison function.
Bind no blend state.
Render opaque geometry.
Bind alpha-to-coverage or alpha/transparency blend state.
Render transparent geometry.

Here, the sets of opaque and transparent geometry are mutually exclusive. The benefit of this approach is that geometry is only drawn once and the rasterization state does not need to change. Unfortunately, it could be possible to observe, through the semi-transparent holes of some double-sided object, that object itself. In this case there is no guarantee that fragments of this object would be blended with other fragments of this object, since the order in which the fragments are resolved is and should not be known.

Another approach:

Bind a rasterizer state with a Less-Equal comparison function.
Bind no blend state.
Render all geometry and clip transparent fragments
Bind a rasterizer state with a Less comparison function.
Bind alpha-to-coverage or alpha/transparency blend state.
Render transparent geometry.

Here, all transparent geometry is drawn twice. First, opaque fragments are processed by the pixel shader whereas transparent fragments are clipped inside the pixel shader. Second, opaque fragments will be discarded by the early-z-test of the rasterizer whereas transparent fragments are processed by the pixel shader (and blended afterwards).

Both approaches have some differences, but the difference that concerns me the most is the latter approach having two variants of the pixel shader: one that clips transparent fragments and one that does not clip transparent fragments. In case of multiple pixel shaders, this approach will always result in a doubling of the number of pixel shaders. Is it possible to have the benefits of the second approach while having only one variant of the pixel shader?

You can use a threshold set through a constant buffer to clip pixels (ie an alpha test):

float threshold; // in constant buffer

float4 color = myTexture.Sample(...);
if (color.a < threshold)
    discard;

Set threshold to 1.0 to discard all transparent fragments; set it to 0.0 to discard nothing. There will be some minor inefficency in the latter case compared to having a separate pixel shader with the alpha test removed entirely, but it shouldn't be a big deal on most GPUs nowadays.


Estimating the position of vertexes in a 3D model Introduction to my project:

We are machine learning scientists and working on a biomedical system for tracking tongue for speech pathology using a technology called EMA (Electromagnetic articulography). In EMA a coil is attached to tongue surface using a glue and its 3D pos and orientation (roll and pitch) are read by EMA measurement system. So, you can track the coil and thus a point on the surface of tongue while speech is made. Also let's assume we have a low density 3D model of tongue for the patient is obtained by 3D MRI and we know exactly that which vertex in 3D model corresponds to the point that coil is attached.

It would be very valuable for our research if we can visualize the tongue for the patient. But the issue is that we just track 1 or 2 points on the tongue but our 3D model has more than 100 vertices which are not tracked. Now, we think it might be possible to estimate the position of those point given these two points.

Here are some of the facts that we think it should be possible:

Although tongue is not a solid object, it has a degree of rigidity thus the position of these 100 vertexes are correlated by physics For example how much they can be stretched or bent (Something like spring and mass model in physics).
by looking at a training set of 3D models for tongue during production of different sounds, we might say that the position of vertexes are not pure random and there is a distribution for them which might be estimated using the training dataset (sequence of MRI 3D models).
we might assume that in our presentation the tongue can have just a bent toward the palate to simplify the modeling problem. or a 2D profile of tongue (like the line at the center) is enough for modeling. or any simplification that we can make.

Now my question:

What do you know in computer graphics literature which resembles to our problem (any model, method, tool, software, library,....)? What is your suggestion for us to do this task in more efficient way?

Thanks,

Mike

If you can construct a mesh that is representative of the tongue, like a spring mass system, then you can attach 'hooks' at your two control points and get a close approximation of the deformation of the tongue.


Tessellation Speed in terms of interpolated points per second For a research project, I have written a Tessellation Shader to interpolate solution points of a 3-Dimensional mathematical equation. Therefore I don't use any techniques for displacement, I merely use the tessellation hardware to interpolate a lot of data points in parallel. I want to write a paragraph about the potential speed benefits compared with trivial interpolation on a CPU, but I can't find any resources about how fast tessellation is in terms of "generating new points per second" or "subdividing triangles per second". Even more interesting would be "new points per GPU clock cycle". But all resources I find are from the video game perspective talking about FPS-performance, which is useless in my case, because first I tessellate my model only once and second I don't use any displacement techniques which are normally used to increase visual quality but will lower the performance again.

Obviously the performance depends on the actual GPU, but a single data sheet as an anchor point would already help a lot.

If you’re only tessellating the model once and never changing the density of the mesh then tessellation is probably a waste of time here—any measurements you make would include all the extra work the GPU has to do to generate new points / triangles, which is orthogonal to what it sounds like you’re trying to measure. Tessellating the mesh ahead of time (in your tool of choice) and doing the interpolation / evaluation in a vertex shader would probably give you a better benchmark against a CPU implementation working on the same mesh.


What algorithm is used in this animation from Tatami Galaxy? Is there a name for this type of effect where many area sizes are increased/decreased in relation to each other? I'm looking for some type of algorithm, formula, logic, etc. for recreating this. Here is the effect (which I believe was created in After Effects):

Source: The Tatami Galaxy (Yojouhan Shinwa Taikei / 四畳半神話大系) - Clear END

It looks like a voronoi diagram with a non-Euclidean distance metric. Probably not Manhattan L1 but something close related, but maybe Mahalanobis with some kind of restriction on seed point generation and movement.

A similar result may be calculated with Weight-proportional Space Partitioning Using Adaptive Voronoi Diagrams when reducing spatial resolution of the domain.

I guess yo'll find similar examples in ShaderToy if you look for voronoi examples, which code you could study, and could be much simpler than the algorithm in the paper.


How are hard and soft shadows cast? I have been studying computer graphics, from the book Fundamentals of Computer Graphic (but the third edition), and I lastly read about texture mapping and shadow maps. Though, I didn't exactly understand how they function. I know there is also another type of texture map called projective texture map which allows to cast hard shadows, but how they function? On the other hand, how are soft shadows cast? And what's exactly the difference between hard and soft shadows?

Hard shadows are simple that only needs a point light.

How it's done is by rendering the scene from the point of view of the light and only keep only the depth information. This is the shadow map.

Then when doing the actual rendering you calculate the point on the triangle in world space and find where it would be on the shadow map. Then you sample the depth from the shadowmap at that point and compare it to the distance to the light.

Soft shadows are harder. They require figuring out how close you are to the edge of what would be a hard shadow and adjusting the light value based on that. There are several techniques for that. One of which is simply sampling multiple times.


Path Tracing & Materials I'm wondering about the technique used when sampling a layered material for the next event in a path tracer. I have a material with three layers: a base diffuse, specular and sheen lobes. How does one go about choosing a discrete layer to sample for the next ray direction? Is this just a random selection? If so, are equal weights assigned to the layers or are weights used based on final contribution of a lobe? Thanks for any insight into this stage of sampling.

~M

This question is somewhat related to this one.

As Alan has already said, following the actual light path through each layer leads to more physically accurate results. I will base my answer on a paper by Andrea Weidlich and Alexander Wilkie ("Arbitrarily Layered Micro-Facet Surfaces") that I have read and partially implemented. In the paper the authors assume that the distance between two layers is smaller than the radius of a differential area element. This assumption simplifies the implementation because we do not have to calculate intersection points separately for each layer, actually we assume that all intersection points over the layers are just the same point.

According to the paper, two problems must be solved in order to render multilayered material. The first one is to properly sample the layers and the second is to find the resulting BSDF generated by the combination of the multiple BSDFs that are found along the sampling path.

UPDATE: Actually I have adopted a different method to implement the evaluation of this layered model. While I have kept with the idea of considering the intersection points to be just the same point along the layers, I have computed the sampling and the final BRDF differently: for sampling, I have used ordinary ray tracing, but through the layers (using Russian Roulette to select between reflection/refraction when that's the case); for the final BRDF evaluation, I just multiply each BRDF traversed by the ray path (weighting the incident radiances according to the cosine of the incident ray).

Sampling

In this first stage we will determine the actual light path through the layers. When a light ray is moving from a less dense medium, e.g. air, to a denser medium, e.g. glass, part of its energy is reflected and the remaining part is transmitted. You can find the amount of energy that is reflected through the Fresnel reflectance equations. So, for instance, if the Fresnel reflectance of a given dielectric is 0.3, we know that 30% of the energy is reflected and 70% will be transmitted:

When the light ray is moving from a denser to a less dense medium, the same principle described by the Fresnel reflectance applies. However, in this specific case, total internal reflection (a.k.a TIR) might also happen if the angle of the incident ray is above the critical angle. In the case of TIR, 100% of the energy is reflected back into the material:

When light hits a conductor or a diffuse surface, it will always be reflected (being the direction of reflection related to the type of the BRDF). In a multilayer material, the resulting light path will be the aggregate result of all those possibilities. Thus, in the case of a 3-layer material, assuming that the first and second layers are dielectrics and the third layer is diffuse, we might end up, for instance, with the following light path (a tree actually):

We can simulate this type of interaction using recursion and weighting each light path according to the actual reflectance/transmitance at the corresponding incident points. A problem regarding the use of recursion in this case is that the number of rays increases with the deepness of the recursion, concentrating computational effort on rays that individually might contribute almost nothing to the final result. On the other hand, the aggregate result of those individual rays at deep recursion levels can be significant and should not be discarded. In this case, we can use Russian Roulette (RR) in order to avoid branching and to probabilistic end light paths without losing energy, but at the cost of a higher variance (noisier result). In this case, the result of the Fresnel reflectance, or the TIR, will be used to randomly select which path to follow. For instance:

As can be seen, TIR or Fresnel reflectance might keep some rays bouncing indefinitely among layers. As far as I know, Mitsuba implements plastic as a two layer material, and it uses a closed form solution for this specific case that accounts for an infinity number of light bounces among layers. However, Mitsuba also allows for the creation of multilayer materials with an arbitrary number of layers, in which case it imposes a maximum number of internal bounces since no closed form solution seems to exist for the general case. As a side effect, some energy can be lost in the rendering process, making the material look darker than it should eventually be.

In my current multilayer material implementation I allow for an arbitrary number of internal bounces at the cost of longer rendering times (well... actually, I've implemented only two layers.. one dielectric and one diffuse :).

An additional option is to mix branching and RR. For instance, the initial rays (lower deep levels) might present substantial contribution to the final image. Thus, one might choose to branch only at the first one or two intersections, using only RR afterwards. This is, for example, the approached used by smallpt.

An interesting point regarding multilayered materials is that individual reflected/transmitted rays can be importance sampled according to the corresponding BRDFs/BTDFs of each layer.

Evaluating the Final BSDF

Considering the following light path computed using RR:

We can evaluate the total amount of radiance $L_r$ reflected by a multilayer BSDF considering each layer as a individual object and applying the same approach used in ordinary path tracing (i.e. the radiance leaving a layer will be the incident radiance for the next layer). The final estimator can thus be represented by the product of each individual Monte Carlo estimator:

$$ L_r =  \left( \frac{fr_1 \cos \theta_1}{pdf_1}  \left( \frac{fr_2 \cos \theta_2}{pdf_2} \left( \frac{fr_3 \cos \theta_3}{pdf_3} \left( \frac{fr_2 \cos \theta_4}{pdf_2} \left( \frac{L_i fr_1 \cos \theta_5}{pdf_1} \right)\right)\right)\right)\right)$$

Since all terms of the estimator are multiplied, we can simplify the implementation by computing the final BSDF and $pdf$ and factoring out the $L_i$ term:

$$fr = fr_1 \cdot fr_2 \cdot fr_3 \cdot fr_2 \cdot fr_1$$

$$pdf = pdf_1 \cdot pdf_2 \cdot pdf_3 \cdot pdf_2 \cdot pdf_1$$

$$\cos \theta= \cos \theta_1 \cdot \cos \theta_2 \cdot \cos \theta_3 \cdot \cos \theta_2 \cdot \cos \theta_1$$

$$ L_r =  \left( \frac{fr \cos \theta}{pdf} \right) L_i$$

The paper by Andrea Weidlich and Alexander Wilkie also takes absorption into consideration, i.e. each light ray might be attenuated according to the absorption factor of each transmissive layer and its thickness. I've not included absorption into my renderer yet, but it is represented by just one scalar value, which will be evaluated according to the Beer's Law.

Alternate approaches

The Mitsuba renderer uses an alternate representation for multilayered material based on the "tabulation of reflectance functions in a Fourier basis". I have not yet dig into it, but might be of interest: "A Comprehensive Framework for Rendering Layered Materials" by Wenzel Jacob et al. There is also an expanded version of this paper.


Can a scene be efficiently pre-rendered and displayed along a fixed camera line? A scene can be pre-rendered from a single camera position.

Can a scene be efficiently rendered for a fixed set of positions and lines that the camera can move through?

Consider games like escape room games where the camera only assumes fixed locations. Can the transition between those positions also be rendered more efficiently than frame-by-frame?

Suppose I wanted the player to move freely along a fixed linear (or curved) track, how can I show the maximum render quality?

Would Diego Nehab et al's "Accelerating Real-Time Shading with Reverse Reprojection Caching" be the sort of thing you are looking for?

Paper Link and/or Slide presentation from Graphics Hardware 2007

IIRC, the idea is to reuse a previous frame's rendered image results in the current frame rather than running a, potentially expensive, shader for every pixel.

At the conference, I did ask Diego if he'd considered doing 'bidirectional' reuse (i.e. rendering frames out of order somewhat akin to h.264) but I can't recall his answer. I think in your proposed use case it might be of value.


What is the correct way to put different textures on different parts of an object? I tried passing a variable from the vertex shader to tell the fragment shader whether or not I wanted part of an object textured. That way I could write another shader for the other parts of the object. Unfortunately, what I have creates the undesirable effect seen here.

vertex shader for house

<script id="long_house-vs" type="x-shader/x-vertex">
    precision highp float;

    attribute vec3 vpos;
    attribute vec3 vnormal;
    attribute vec2 vtexCoords;

    varying vec3 outNormal;
    varying vec3 outPos;
    varying vec2 outTexCoords;
    // 1 if part of roof
    varying float roof;

    uniform mat4 view;
    uniform mat4 proj;
    uniform mat4 model;
    uniform vec3 lightdir;
    uniform vec3 long_housecolor;

    void main(void) {
        if(vpos.y > 1.2) {
            roof = 1.0;
        } else {
            roof = 0.0;
        }
        gl_Position = proj * view * model * vec4(vpos, 1.0);
        outTexCoords = vtexCoords;
        outPos =  (view * model * vec4(vpos, 1.0)).xyz;
        outNormal = normalize(view * model * vec4(vnormal,0.0)).xyz;
    }
</script>

fragment shader

<script id="long_house-fs" type="x-shader/x-fragment">
    precision highp float;
    varying vec3 outNormal;
    varying vec3 outPos;
    varying vec2 outTexCoords;
    varying float roof;

    uniform vec3 lightdir;
    uniform vec3 long_housecolor;
    uniform sampler2D usampler;
    uniform sampler2D urooftext;

    void main(void) {
        float diffuse = .5 + .5*abs(dot(outNormal, lightdir));
        vec3 outColor = long_housecolor * diffuse;
        vec4 textureColor;
        if(roof == 1.0) {
            textureColor = texture2D(urooftext, outTexCoords);
        } else {
            discard;
        }
        gl_FragColor = vec4(textureColor.rgb * diffuse, textureColor.a);
    }
</script>

When I rearrange the code a bit so that the bottom half of the house is textured instead of the top I get this

So what would be the correct way to write a shader that only affects part of an object?

In vertex, put  vPos as varying. Then instead of doing the height check in vertex, do it in fragment.

if(outVPos.y > 1.2)
{
   textureColor = texture2D(urooftext, outTexCoords);
}
else
{
   discard;
}

By the way, varying qualifier is deprecated and replaced with out in vertex and in in fragment.


What's a "texel coordinate"? So I'm reading some Ray-Sphere interaction codes and in one there was "texCoord", which I think means "texel coordinate". However, what does this refer to "visually"?

It stands for texture coordinate.

It's a 2D value between 0 and 1 to indicate where in the texture to sample from for the color.

For sphere that means creating a map projection to map the curved surface to the flat plane, a simple option is to use the lat/long remapped to 0-1 though that gives awful distortion at the poles.


Why the BRDF of specular reflection is infinite in the reflection direction? I know the BRDF of specular reflection is nonzero only in the reflection direction.

But why it is infinite?

A paragraph on page 36 of Advanced Global Illumination:

In the BRDF curve, the total area under the curve is the albedo: what fraction of incident light is reflected in total (as opposed to being absorbed or transmitted). For a perfect reflector, the area under the curve sums to 1, because it reflects all of the incident light. This area limits how high the curve can be. For example, a perfect diffuse reflector has a BRDF that's a flat line at $\frac{1}{2\pi}$, because it reflects the light evenly across the whole hemisphere, the solid angle of a hemisphere is $2\pi$, and the area has to be 1.

In a perfect specular reflector, the area under the curve is still 1: it still reflects all of the incident light. But this time, it's reflecting all of that light in one direction. The BRDF curve is a single spike. If you draw a graph with an infinitely thin spike, and the area under the graph has to be 1, then the spike must also be infinitely high. It's the only way to make the area correct.

Obviously a real material can't be a perfect specular reflector, so it will have two differences. First, the area under the curve will be less than 1,  because some light is absorbed. This doesn't matter much to us. More importantly, the reflection peak can't be infinitely thin: the reflection will be blurred ever so slightly. This means the peak won't be infinitely high: the wider the peak, the less high it has to be to still have an area of 1.

The remark about "$\delta$-functions" is because of a function (properly speaking, it's not a function but a distribution) called the Dirac $\delta$ (delta). This distribution is an infinitely thin, infinitely tall spike at $x = 0$, whose area is 1, just like our perfect specular BRDF.


adobe illustrator art brush, how does it work? I want to find out how the art brush in adobe illustrator works.

It looks like it's taking some vector art and applying it to a smooth line.

How can I go about implementing this?

These rely on a transformation from global coordinates to path local coordinates, where the local axes are along and out (or more often u and v). In essence this all works on a principle similar to how offset works.

Image 1: Depiction of local axes.

Now the first one ($A$) is a sweep of an oval. Which is technically the most challenging of the four. In essence You offset edges directionally and cap them with a fixed directional oval. Although this has all kinds of interesting edge cases. A lot of mathematical lore is known about sweeping shapes as this is somewhat hard to do in 3D and doing so with an arbitrary shape is useful for the cad crowd. At it simplest its just sampling many times, which allows for change in shape along line, but quite sophisticated approaches are available.

Image 2: Oval sweep.

The second is simply (bombing) a shape in random places in the new coordinate system. Mathematically this is pretty simple. Generate a number in along direction then find that point. Then generate a second number and multiply that by the normal direction vector of your path then add this to the point.

The third and fourth just do this for a lot of points read in form a Cartesian template object. Again you should familiarize yourself with how offset and graduated offset is done. You can find a lot of references but one of my favorite online resource for Béziers is in the Pomax Bézier primer. Its just lot of small offset lines to transform from.


MSI GTX 950 2GD5T triangles per second A friend got an MSI GTX 950 2GD5T graphics card for xmas.

I said "yeah, graphics cards are super fast now, that thing probably processes 2 billion triangles per second". Then I tried to look it up to check if that was even in the right ballpark, but I was unable to find a "triangles per second" stat for that card, and actually for many cards.

Is "triangles per second" even a meaningful stat? If so, what is the approximate number for this card? (given otherwise average conditions)

Yes, it's a meaningful stat: GPUs have dedicated triangle setup HW and the rate is measured in triangles/GPU clock. According to white papers available on NV's website, the 680 (Kepler) could issue one triangle per SM every other clock - with 8 SMs, this yielded 4 triangles/clock. The Maxwell white paper doesn't indicate a change in this rate per SM - the 980 has 16 SMs so, if there is really is no rate change per SM, it can produce 8 triangles/clock. While the 980 has 2048 CUDA cores, the 950 has 768, implying 6 SMs and 3 triangles/clock. The chip runs around 1 Ghz, so the 950 is probably limited to 3 billion triangles per second.