Sending programs from CPU to GPU How does a CPU know which piece of code should be executed on GPU (i.e how to recognize whether a code is shader or not) and how is it been sent to GPU VRAM?Is it via DMA?

Most, if not all, approaches to GPU programming require the developer to manually identify code that will be executed on the GPU. The code for the GPU is compiled separately from CPU code, so it is clear which binary code is for which processor type. Moving the GPU code to VRAM is often done with DMA, but may be moved to GPU accessible memory using other techniques such as CPU writes across PCIe, or direct CPU memory access by the GPU.

The GPU compiler often resides in userspace, and a system call will hand binary GPU code to the kernel, which can then make the code available to the GPU.


Is a cubic Lagrange interpolation tensor product the same as bicubic interpolation? I just implemented some interpolated texture sampling by sampling the 4x4 nearest pixels then doing Lagrange interpolation across the x axis to get four values to use Lagrange interpolation on across the y axis.

Is this the same as bicubic interpolation or is it different?  Or are there different kinds of bicubic interpolation, and this is just one of them perhaps?

Webgl Shadertoy implementation here and relevant GLSL (WebGL) code below: https://www.shadertoy.com/view/MllSzX

Thanks!

float c_textureSize = 64.0;

float c_onePixel = 1.0 / c_textureSize;
float c_twoPixels = 2.0 / c_textureSize;

float c_x0 = -1.0;
float c_x1 =  0.0;
float c_x2 =  1.0;
float c_x3 =  2.0;

//=======================================================================================
vec3 CubicLagrange (vec3 A, vec3 B, vec3 C, vec3 D, float t)
{
    return
        A *
        (
            (t - c_x1) / (c_x0 - c_x1) *
            (t - c_x2) / (c_x0 - c_x2) *
            (t - c_x3) / (c_x0 - c_x3)
        ) +
        B *
        (
            (t - c_x0) / (c_x1 - c_x0) *
            (t - c_x2) / (c_x1 - c_x2) *
            (t - c_x3) / (c_x1 - c_x3)
        ) +
        C *
        (
            (t - c_x0) / (c_x2 - c_x0) *
            (t - c_x1) / (c_x2 - c_x1) *
            (t - c_x3) / (c_x2 - c_x3)
        ) +
        D *
        (
            (t - c_x0) / (c_x3 - c_x0) *
            (t - c_x1) / (c_x3 - c_x1) *
            (t - c_x2) / (c_x3 - c_x2)
        );
}

//=======================================================================================
vec3 BicubicTextureSample (vec2 P)
{
    vec2 pixel = P * c_textureSize + 0.5;

    vec2 frac = fract(pixel);
    pixel = floor(pixel) / c_textureSize - vec2(c_onePixel/2.0);

    vec3 C00 = texture2D(iChannel0, pixel + vec2(-c_onePixel ,-c_onePixel)).rgb;
    vec3 C10 = texture2D(iChannel0, pixel + vec2( 0.0        ,-c_onePixel)).rgb;
    vec3 C20 = texture2D(iChannel0, pixel + vec2( c_onePixel ,-c_onePixel)).rgb;
    vec3 C30 = texture2D(iChannel0, pixel + vec2( c_twoPixels,-c_onePixel)).rgb;

    vec3 C01 = texture2D(iChannel0, pixel + vec2(-c_onePixel , 0.0)).rgb;
    vec3 C11 = texture2D(iChannel0, pixel + vec2( 0.0        , 0.0)).rgb;
    vec3 C21 = texture2D(iChannel0, pixel + vec2( c_onePixel , 0.0)).rgb;
    vec3 C31 = texture2D(iChannel0, pixel + vec2( c_twoPixels, 0.0)).rgb;

    vec3 C02 = texture2D(iChannel0, pixel + vec2(-c_onePixel , c_onePixel)).rgb;
    vec3 C12 = texture2D(iChannel0, pixel + vec2( 0.0        , c_onePixel)).rgb;
    vec3 C22 = texture2D(iChannel0, pixel + vec2( c_onePixel , c_onePixel)).rgb;
    vec3 C32 = texture2D(iChannel0, pixel + vec2( c_twoPixels, c_onePixel)).rgb;

    vec3 C03 = texture2D(iChannel0, pixel + vec2(-c_onePixel , c_twoPixels)).rgb;
    vec3 C13 = texture2D(iChannel0, pixel + vec2( 0.0        , c_twoPixels)).rgb;
    vec3 C23 = texture2D(iChannel0, pixel + vec2( c_onePixel , c_twoPixels)).rgb;
    vec3 C33 = texture2D(iChannel0, pixel + vec2( c_twoPixels, c_twoPixels)).rgb;

    vec3 CP0X = CubicLagrange(C00, C10, C20, C30, frac.x);
    vec3 CP1X = CubicLagrange(C01, C11, C21, C31, frac.x);
    vec3 CP2X = CubicLagrange(C02, C12, C22, C32, frac.x);
    vec3 CP3X = CubicLagrange(C03, C13, C23, C33, frac.x);

    return CubicLagrange(CP0X, CP1X, CP2X, CP3X, frac.y);
}

It turns out that no, while you can use bicubic Lagrange interpolation for bicubic texture sampling, it isn't the highest quality option, and probably not actually likely to be used.

Cubic hermite splines are a better tool for the job.

Lagrange interpolation will make a curve that passes through the data points, thus preserving C0 continuity, but hermite splines preserve the derivatives at the edges while also passing through the data points, thus preserving C1 continuity and looking much better.

This question has some great info on cubic hermite splines:
https://dsp.stackexchange.com/questions/18265/bicubic-interpolation

Here is the cubic hermite version of the code I posted in the question:

//=======================================================================================
vec3 CubicHermite (vec3 A, vec3 B, vec3 C, vec3 D, float t)
{
    float t2 = t*t;
    float t3 = t*t*t;
    vec3 a = -A/2.0 + (3.0*B)/2.0 - (3.0*C)/2.0 + D/2.0;
    vec3 b = A - (5.0*B)/2.0 + 2.0*C - D / 2.0;
    vec3 c = -A/2.0 + C/2.0;
    vec3 d = B;

    return a*t3 + b*t2 + c*t + d;
}

//=======================================================================================
vec3 BicubicHermiteTextureSample (vec2 P)
{
    vec2 pixel = P * c_textureSize + 0.5;

    vec2 frac = fract(pixel);
    pixel = floor(pixel) / c_textureSize - vec2(c_onePixel/2.0);

    vec3 C00 = texture2D(iChannel0, pixel + vec2(-c_onePixel ,-c_onePixel)).rgb;
    vec3 C10 = texture2D(iChannel0, pixel + vec2( 0.0        ,-c_onePixel)).rgb;
    vec3 C20 = texture2D(iChannel0, pixel + vec2( c_onePixel ,-c_onePixel)).rgb;
    vec3 C30 = texture2D(iChannel0, pixel + vec2( c_twoPixels,-c_onePixel)).rgb;

    vec3 C01 = texture2D(iChannel0, pixel + vec2(-c_onePixel , 0.0)).rgb;
    vec3 C11 = texture2D(iChannel0, pixel + vec2( 0.0        , 0.0)).rgb;
    vec3 C21 = texture2D(iChannel0, pixel + vec2( c_onePixel , 0.0)).rgb;
    vec3 C31 = texture2D(iChannel0, pixel + vec2( c_twoPixels, 0.0)).rgb;

    vec3 C02 = texture2D(iChannel0, pixel + vec2(-c_onePixel , c_onePixel)).rgb;
    vec3 C12 = texture2D(iChannel0, pixel + vec2( 0.0        , c_onePixel)).rgb;
    vec3 C22 = texture2D(iChannel0, pixel + vec2( c_onePixel , c_onePixel)).rgb;
    vec3 C32 = texture2D(iChannel0, pixel + vec2( c_twoPixels, c_onePixel)).rgb;

    vec3 C03 = texture2D(iChannel0, pixel + vec2(-c_onePixel , c_twoPixels)).rgb;
    vec3 C13 = texture2D(iChannel0, pixel + vec2( 0.0        , c_twoPixels)).rgb;
    vec3 C23 = texture2D(iChannel0, pixel + vec2( c_onePixel , c_twoPixels)).rgb;
    vec3 C33 = texture2D(iChannel0, pixel + vec2( c_twoPixels, c_twoPixels)).rgb;

    vec3 CP0X = CubicHermite(C00, C10, C20, C30, frac.x);
    vec3 CP1X = CubicHermite(C01, C11, C21, C31, frac.x);
    vec3 CP2X = CubicHermite(C02, C12, C22, C32, frac.x);
    vec3 CP3X = CubicHermite(C03, C13, C23, C33, frac.x);

    return CubicHermite(CP0X, CP1X, CP2X, CP3X, frac.y);
}

Here is a picture showing the difference between sampling methods.  From left to right: Nearest Neighbor, Bilinear, Lagrange Bicubic, Hermite Bicubic


Image based lighting, tangent space coordinates, and optimization When I saw an implementation of normal mapping that computed the TBN matrix in the vertex shader and converted everything (in particular the view vector and light vector) to tangent space at that stage, it seemed like the proper way to do. This way there is no matrix-vector multiplication left to do in the fragment shader.

Enters image based lighting: for each fragment we want to fetch from an image, typically one or more cube map textures, the light intensity coming from certain directions. I assume this data is in world space.

My question is: is it possible to use IBL while retaining the vertex shader transform optimization mentioned above?

It seems to me we still need to transform our fetch direction from tangent space to world space (or whichever space the cube map is in).
Conversely, if we want to work in world space, like with deferred shading, we need to transform the normal for each fragment.

Does IBL necessarily imply at least one matrix vector multiplication per fragment and the optimization I mentioned is an uncommon case, or am I missing something?

Yes, you do need to transform the fetch direction into the space of the cubemap. If you could somehow figure out the fetch direction in the vertex shader, then you could do the transformation there instead, but that would produce worse lighting.

It also may be worth optimizing for a smaller number of interpolants between the vertex and pixel shaders (rather than a smaller number of matrix-vector multiplications in the pixel shader). As soon as you have one transformed view vector and two transformed light vectors that must be interpolated (which would otherwise be constant), you've matched the size of the tangent/bitangent/normal matrix that you would otherwise be interpolating. That has the potential to have more performance impact than doing extra matrix multiplications in the shader.


is focal length equal to the distance from the optical center to the near clipping plane- openGL The part confusing  about opengl during studying the projection was that the focal length, top, left, bottom, right, wasn't declared, and it magically did it with FOV and image aspect ration.   I found the wonderful maths in this amazing site, and found out how it is calculated based on FOV and focal length.
Two points I noted for my confusion was:

near clipping plane is the plane which limits the projected coordinates, and
focal length is the distance from the point of projection to the optical center( (0,0,0) in our case )

Now, the question is, is the focal length always equal to the distance of the near clipping plane from the optical center, i.e. in other words, is the image always projected on the near clipping plane (or at least in openGL)? Yet in other words, is the near clipping plane and the plane of projection the same thing just with different terms.

For some reason most sites choose to leave the projection plane out of the visualization of the view frustum. The focal length often labeled "g" is the distance from the camera to the projection plane. The projection plane (which is blue in the attached image) is almost always placed at a distance from the camera such that the distance from the center to the top and bottom is 1. This image from the fged website shows it well:  This choice is a matter of convenience since it makes the math all work out nice and neat. As you can see the near plane can, and often is, behind the projection plane. (the green plane in the image is the near plane)
For your second confusion: One goal of the projection matrix is to force coordinates with a z value less then that of the near plane to fall outside of the near clipping region which is -1 in OpenGL. So any coordinate that has a z value less then near value will result in a z value below -1 after projection.


Is there a general rule with regard to using the inverse of the "camera" to world when using a homogeneous transform matrix? Not sure if this is just specific to this article but it says:

We learned how to convert points from world to any local coordinate
system. If we know the local-to-world matrix, we can multiply the
world coordinate of the point by the inverse of the local-to-world
matrix (the world-to-local matrix).

Nothing else is ever done with the local-to-world matrix - other than taking its inverse. So I'm wondering if it would be easier just to define the world-to-local matrix directly? Does taking the inverse make things like "moving" the camera more intuitive?
Thanks

Is there a general rule with regard to using the inverse of the “camera” to world when using a homogeneous transform matrix?

No, there is no general rule. You should always think about which approach is the best one for your specific use-case.

Nothing else is ever done with the local-to-world matrix - other than taking its inverse. So I'm wondering if it would be easier just to define the world-to-local matrix directly?

Don't forget, that the source you linked is a tutorial. It has a limited scope and avoids confusing the reader with additional information he might not understand at the time of reading. That said, if I have an isolated look at the rendering process, I can't think of a reason why you absolutely need the cam-to-world matrix there because you want to transform everything into camera space before continuing with other transformations (projection, clipping, etc.). Even though it seems to make no sense to transform something out of camera space just to transform it back later, there still might be some reasons to do exactly this. For example, if some renderable object is attached to the camera coordinate system. Using an extra shader or a branch statement just for objects defined in the camera system usually costs you much more performance than two obsolete transformations.
If we now consider other operations, not just rendering, you might find some useful applications of the camera to world matrix, for example, the movement of a camera that always moves into view direction or orthogonal to it (free camera mode in an FPS). The target point of the movement can be easily determined in camera space. It is just a vector of the movement speeds in each principle direction. Use the camera to world matrix to transform the target point into world space and you have your new location (if no collisions occur).
Lastly, another reason to take the "inverse-route" might be that it is just easier, less work, or more convenient. All objects, including the camera, usually have a world space coordinate and orientation. The transformation to world space is always the same series of operations. Scale-rotate-translate. Since it so common and frequently used, you probably have an optimized function that does not multiply 3 individual matrices but calculates all the entries of the resulting matrix directly. You can still do the same for the world-to-local transformation, but it is less frequently used and if you only need it for your world-to-camera transformation it might be just easier to calculate the inverse since every linear algebra library you might use should provide this function. If you do this not too frequently each frame, it shouldn't affect performance in any notable way.
As I said in the beginning, you should always focus on what your program needs. If you have no use for the camera-to-world matrix other than to calculate the world-to-camera matrix, you can definitely skip this and calculate the world-to-camera matrix directly. Just ask yourself, if it is worth it to add some extra lines of code for that.


Maximal and minimal no. of angles obtained from clipping a convex polygon with n angles I'm trying to figure this out for my Computer Graphics course, but I find it very hard to understand.

I believe the minimum number of angles obtained from clipping a convex polygon with n angles is 0, for example if the polygon is outside of the clipping rectangle, but I'm not sure if this makes sense.

As for the maximum number of angles, I believe it is 2n, as in if every angle tip gets outside of the rectangle. However, I'm not sure about this either.

Since this is a homework question, I'll give hints rather than a numerical answer.

Clipping a convex polygon

Think about how many times the polygon can cross each edge of the rectangle. In general, a polygon can cross one of the edges of the rectangle an arbitrarily large number of times. How is this number reduced if the polygon is convex?

This will give you the maximal number of new angles that can be added per rectangle edge, which should lead you to the answer.

Clipping a non-convex polygon

As for your guess of 2n, even for a non-convex polygon it may not be able to reach this high. If one vertex is outside the rectangle, then in order to create 2 new vertices both of its neighbouring vertices must be inside the rectangle. This means you cannot create 2n new vertices this way.

The exception is when an edge of the polygon crosses 2 edges of the rectangle, for example at a corner of the rectangle. This allows creating two new vertices from a single edge. So for small enough n, it is possible to clip to a polygon with 2n vertices, but for larger n this is not possible.


Computing camera front direction from Euler angles I am new to OpenGL and Computer Graphics in general.

Lately I was learning how to model a camera, specifically how to model the rotation of camera. I was introduced to Euler angles for this purpose.

I got a rough idea about how to use Euler angles to model camera rotation from this post. Based on that, I wrote something like below:

GLfloat yaw, pitch;
...
void mouse_callback(GLFWwindow* window, double xpos, double ypos)
{
        /*
           Here I am updating the yaw, and pitch values
           based on mouse movement and previous mouse position
        */

        glm::mat4 yawPitchRotMat;
        /* We want to do yaw * pitch. */

        //y-axis as yaw axis
        yawPitchRotMat = glm::rotate(yawPitchRotMat, glm::radians(yaw),
                         glm::vec3(0.0f, 1.0f, 0.0f));

        //x-axis as pitch axis
        yawPitchRotMat = glm::rotate(yawPitchRotMat, glm::radians(pitch),
                         glm::vec3(1.0f, 0.0f, 0.0f));

        cameraFrontDirection = glm::normalize(-1.0f *
                 glm::vec3(yawPitchRotMat[2].x, yawPitchRotMat[2].y, yawPitchRotMat[2].z));
}

The above code seems to work. However, I would like to hear your opinion about this approach. Specially as my math is not really very strong, so any mathematical insight regarding Euler angles would be very helpful. Also appreciate any reference to online resource on the same, that is easy to understand.

Note: Though later I also came to know about certain limitation of modelling rotation using Euler angles and use quaternions instead. However I would like to save that topic for future discussion, as I am yet to read about that in details.

This kind of navigation works well for situations that are centered about objects. It keeps the up vector always pointing upward which makes it less possible for confusion.

Bringing quaternions into this mix is not necessarily a good idea. People often totally exaggerate the problems of Euler angles (or taint Tait–Bryan angles). Its true that Euler did not even consider the implications in interpolating the angles. True you can get into big problems if you rotate angles into certain positions and interpolate angle by angle. But this didn't bother Euler so much as he did mathematical modeling where the problem cancels itself out.

So Euler angles are perfectly fine for storing the rotation, also ok in differential rotation. And in case as no gimbal lock can really happen anyway, you get other problems before it. Euler angles are just not good at linear interpolation in a curved space.

In this case there should not be any bigger problems. Euler angles are much easier to understand. Unless you need to interpolate between 2 recorded positions. However this is a no issue as your collection and interpolation dont need to be based on the same model.

Its possible to look at this model in another light, you can think of it as a vector that records camera position in relation to a center as long as you keep the vector equal length you achieve same thing. You can then just form the matrix by multiplying by up vector and then that by front to get new up. This is even easier to interpolate unless you move trough the actual center but again you can keep normalizing the vector to a length.


Why is there a difference between the CIE XYZ colour gamut vs CIE RGB? Yet another question about colour space...
In my research on CIE XYZ system, I found that it is based on the CIE RGB colour matching experiments, and that because the RGB system needed occasional negative values, XYZ was developed in order to have an entirely non-negative system. From this I infer that the x(), y() and z() colour matching functions are just transformations of the original r(), g() and b() functions, using exactly the same data. I also found online that the XYZ and RGB systems were completely interchangeable, and XYZ is just preferred for the lack of any negative components.
If that is incorrect please correct me!
However, if they are interchangeable and based off of the same data, why is it that Wikipedia's comparison of the two spaces shows a marked difference? Is the missing curvy section outside of that inner triangle just the areas where the CIE RGB system would have to be negative?
https://en.wikipedia.org/wiki/CIE_1931_color_space#CIE_RGB_color_space

Correct, the missing curvy section in the green-cyan-blue area represents where the red component would have to go negative to express those colors in CIE RGB coordinates.
RGB and XYZ are, at one level, just different coordinate systems covering the same color space—the space of all colors visible to typical human vision. In a mathematical sense, when used as coordinates, there's nothing wrong with negative RGB values (as long as the overall luminance of the color remains positive). But it does present a problem for storing or transmitting such values, as conventional image formats and display signal protocols like HDMI etc only allow for positive values.
On another level, various RGB color spaces are used because they more or less directly represent the actual red, green, and blue subpixels on the display. Those can't emit negative light of course, and so the RGB triangle in color space represents the gamut of colors that can be produced by the display.
Unfortunately because the spectral locus is curved, there's no way that 3 primaries can cover the whole thing. All RGB spaces inevitably cut off a big chunk of the highly saturated green/blue colors.
The XYZ space sort of has the opposite problem. All visible colors can be represented using only positive values in XYZ, but the XYZ primaries themselves are not physically possible colors—they're well outside the visible gamut. So, there's a big chunk of XYZ space that is not valid as a color. And it's not trivial to determine exactly what values are or aren't valid, as you have to test whether they fall inside or outside of the curved spectral locus. This also means that you need more bits per component to get good precision, if you store/transmit images expressed in XYZ—8 bits won't do, probably not 10 bits either, maybe 12 bits would do the job.
If we want to make displays that cover more of the visible gamut, we will eventually need to move to 4, 5, or more primaries. However, that doesn't mean we need 4- or 5-dimensional color spaces. A futuristic display device could be fed by images in XYZ space, for instance, and the device would decide how best to generate each color using the primaries it has available.


Wrong tangential directions for cubic bezier curve I am well aware that there are several ways to find the tangents to a Bezier curve. However, I choose to just substitute the result of differentiating the Bezier curve equation.

So, given four control points, ptOne, ptTwo, ptThree and ptFour:

// Differentiate x with respect to t
float dxdt = -3*(1 - t)*(1 - t)*ptOne.x
+ 3*(1 - 4*t + 3*t*t)*ptTwo.x
+ 3*(2*t - 3*t*t)*ptThree.x
+ 3*t*t*ptFour.x;

// Differentiate y with respect to t
float dydt = -3*(1 - t)*(1 - t)*ptOne.y
+ 3*(1 - 4*t + 3*t*t)*ptTwo.y
+ 3*(2*t - 3*t*t)*ptThree.y
+ 3*t*t*ptFour.y;

// Find the tangent, dy/dx
float dydx = dydt * (1/dxdt);

// Find the rotation angle in degrees
float angle = atan(dydx) * 180/M_PI;

The strange thing is that this does not seem to be working for all kinds of curve. For example, if I draw a curve like this, with the first control point being the second leftmost point and then in a clockwise direction:

and attempt to draw the tangents:

You can see that some tangents (in particular, the first two and very last one) are not oriented in the right direction. Why might this be happening? I triple checked my differentiation and can't find anything wrong...

Using what Rahul kindly suggested in the comments, I realised that my tangent can be rotated in the opposite direction for two cases:

if dy/dt and dx/dt are both negative, yet dy/dx ends up being positive
if dy/dt is negative and dx/dt is positive, so the overall dy/dx is negative

This is because my angle calculation, which is done by float angle = atan(dydx) * 180/M_PI; will only calculate acute angles from the horizontal, and hence miss out the opposite cases.

Thanks a bunch!


What happens when you display a high resolution image on a low resolution screen? For a non-computer graphics expert, what happens technically when you do this? Are multiple pixels merged to one to compensate for the lower resolution? And in how far will this affect the image quality?

Would a high resolution image look sharper on a low resolution display than if the same image would have the exact same resolution as the screen?

If you scale an image down to a screen that has a lower resolution than yout image, you have to descide what you do, there is no single way of doing this.

There are many different techniques that yield very different results.
This is very similar to displaying a low resolution image on a high resolution display where do the inverse thing by desciding how to fill the "more" pixels on the screen. There are also many techniques for doing this.

The image shows two upscaling-techniques and shows how different the results can look if different techniques are used. (A nearest-neighbor scaling on the left and a 2*Sal on the right.) Imagine that there are also such differences if you downscale an image.

You can find a nice overview on different scaling techniques on the wikipedia page for Image scaling.


What part of the graphics chain rotates computer displays when in portrait mode? When the display hardware is physically rotated 90 degrees, your graphics can still be rendered with the correct side up. Where is this rotation accomplished?
I can imagine a few possibilities:

A) The code for every low-level graphics primitive contains something to converts its arguments, like

if (portrait) {
   swap(x,y);
}

(or perhaps multiplying every coordinate with a matrix)

B) The part of the graphics card that continuously scans its memory to create a video signal reads the data in a different order.

C) The monitor paints the received data column by column instead of row by row

A and B sounds like they would be inefficient and C means that it would only work on monitors that has special hardware-support for rotation.
Which one is it?

On typical GPUs today it's option B: monitor rotation is done by the video output hardware ("display engine") during scan-out. In addition to rotating/flipping, the display engine can also scale the image within certain limits, do simple color/gamma correction, and even composite multiple image layers ("planes") together, all dynamically during scan-out.


Loop performance in a shader I am wondering what's the best way to integrate a dynamic loop function in a shader?

First, it seems that dynamic arrays are not possible. So, is it better to create a maximum size array and only fill a part of it or define arrays with predefined sizes?

Then, what's the best way to iterate over this array?

Is it better to use an unrolled loop or a dynamic loop for something between 4 to 128 iterations?
I've also seen that it's possible to unroll it to a maximum predefined number of iterations then stop it with a condition such as if (i == myCurrentMaximumIterationNumber).

Shader compilers are extremely aggressive about unrolling since early HW often didn't have flow control and the cost on more recent HW can vary. If you have a benchmark you are actively testing against and a range of relevant hardware, then try things to see what happens. Your dynamic loop is more amenable to developer intervention than a static loop - but leaving it to the compiler is still good advice unless you have a benchmark available. With a benchmark, exploration is worthwhile (and fun).

BTW, the biggest loss with a dynamic loop on a GPU is that individual "threads" in a wavefront/warp will finish at different times. The threads that stop later force all the ones that finish early to execute NOPs.

Nested loops should be carefully thought through: I implemented a block based entropy decoder that encoded runs of zeros (for JPEG like compression). The natural implementation was to decode the runs in a tight inner loop - which meant often only one thread was making progress; by flattening the loop and explicitly testing in each thread if it was currently decoding a run or not, I kept all threads active through the fixed length loop (the decoded blocks were all the same size). If the threads were like CPU threads, the change would have been terrible, but on the GPU I was running on, I got a 6 fold increase in performance (which was still terrible - there weren't enough blocks to keep the GPU busy - but it was a proof of concept).


How to calculate vertex normals on a mesh with non-planar polygons If I have a mesh consisting of polygons that are not necessarily triangles and not necessarily planar. As answered in the previous question I asked, there's no correct answer to calculating normals for such polygons. So I'm wondering instead no matter how wrong, what are some of the strategies used in 3d modeling to calculate the vertex normals for such a mesh, specifically for smooth shading?

If you're interested in vertex normals specifically, there's an easy answer even for non-planar polygons that avoids the question of defining what the exact surface is: for each vertex, calculate the normal of the plane formed by the two edges entering and leaving that vertex.
More formally, given vertices $\mathbf{v}_1, \mathbf{v}_2, \ldots \mathbf{v}_n$ with counterclockwise winding, define the normal at the $i$th vertex as:
$$
\mathbf{n}_i = (\mathbf{v}_i - \mathbf{v}_{i-1}) \times (\mathbf{v}_{i+1} - \mathbf{v}_i)
$$
(where the indices wrap around).
You can then proceed to accumulate normals calculated this way from all the faces that share a given vertex, as usual for smooth shading.
As has been discussed, there are multiple ways to define a smooth surface corresponding to a non-planar polygon, but any reasonable way of defining such a surface must converge to the normals as defined here near the vertices, or else the surface can not both be smooth (locally flat) and meet the straight-line edges between the vertices.
A caveat to this approach, though, is that it won't define a normal for collinear vertices (where the entering and exiting edges are parallel), since the cross product goes to zero there. If this is a problem, it might work to patch up such vertices by interpolating normals to them from the surrounding non-collinear vertices.


What does "muted" mean in the context of color? I want to extract colors from images like in Adobe Kuler. When I select colorful or muted corresponding colors are shown. But what is the definition for colorful or muted colors? What do the words "colorful", "deep" and "muted" refer to?

In terms of the HSV color space, "muted" colors are those with lower saturation and/or value.

"Deep" colors are saturated but not too high in value (e.g. deep red) while colors with both high saturation and value might be called "bright" (e.g. bright red).

"Colorful" is somewhat more vague; it might mean bright colors, but it also might mean having a range of different hues within an image or palette, such as including complementary colors or analogous colors rather than just variations on a single hue.

Note that these are general terms that would be understood by any graphic designer or artist, but the specific algorithms that Adobe Kuler attaches to these words are proprietary and only known to Adobe.


Understanding material .mtl files I am reading this piece of documentation to understand mtl files

So far I am stuck trying to understand the difference between ambient, diffuse and specular reflectivity (Ka, Kd, Ks).

If I am correct specular is "straight line" reflectivity (the kind of reflectivity of mirors), diffuse is the reflectivity of scattered light (the kind of reflectivity of many rough plastics). But then what is ambient reflectivity?

@PaulHK explained it very well in the comment section:

  Ambient is a (rough) approximation of indirect lighting. For example, an outdoor object would be lit by the sun but the surfaces facing away from sun would be black if using only lambert shading, we can use dark gray for the ambient colour to mimic the effect of indirect lighting from the ground and sky

If you were to visualize this with a shading model, say Phong, the Ka, Kd and Ks values would correspond to the following lighting calculations.


Is there a way I can make this easier for openGL to draw? I have a very basic grasp of computer graphics however I threw it all away when I wrote this simple application that I am using for scientific purposes. I didn't have a lot of time to write it correctly because my "bosses" didn't see the promise of it but now that it is done they have found it quite useful.

The problem is I built it stupidly.

Basically it takes a TON of points stored in a big array and draws them to the screen. The screen can be moved and zoomed.

Instead of using a matrix to accomplish this I am simply passing in the width, height and center of the screen making my average vertex shader look like so

#version 330 core
layout (location = 0) in float ex;
layout (location = 1) in float ey;

uniform float sizeX;
uniform float sizeY;
uniform float centerX;
uniform float centerY;
uniform float size;

out vec4 color;
out float passsize;
void main()
{
    gl_PointSize = size;
    passsize = size;
    float x = (ex - centerX) / (sizeX / 2.0);
    float y = (ey - centerY) / (sizeY / 2.0);
    gl_Position = vec4(x, y, 0.0, 1.0);

    color = vec4(1.0f, 0.5f, 0.2f, 1.0f);
}

and an average draw call look about like this

glUseProgram(program);
glGenVertexArrays(1, &VAO1);
glBindVertexArray(VAO1);

glGenBuffers(1, &VBO1);
glBindBuffer(GL_ARRAY_BUFFER, VBO1);
glBufferData(GL_ARRAY_BUFFER, sizeof(data1), data1, GL_STATIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, VBO1);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 0, 0);
glEnableVertexAttribArray(0);

glBindBuffer(GL_ARRAY_BUFFER, 0);

glGenBuffers(1, &VBO2);
glBindBuffer(GL_ARRAY_BUFFER, VBO2);
glBufferData(GL_ARRAY_BUFFER, sizeof(data2), data2, GL_STATIC_DRAW);
glVertexAttribPointer(1, 1, GL_FLOAT, GL_FALSE, 0, 0);
glEnableVertexAttribArray(1);

glUniform1f(glGetUniformLocation(program, "centerX"), centerX);
glUniform1f(glGetUniformLocation(program, "centerY"), centerY);
glUniform1f(glGetUniformLocation(program, "sizeX"), scrwidth);
glUniform1f(glGetUniformLocation(program, "sizeY"), scrheight);
glUniform1f(glGetUniformLocation(program, "size"), dotSize);

glDrawArrays(GL_POINTS, 0, dot count);
glfwSwapBuffers(window);
glfwPollEvents();

This works very well but I would love to get rid of the low frame rate this causes when I am drawing a ton of points (generally over 10,000 on my small laptop that doesn't actually have a GPU).

Is there any way I can get the GPU to more intelligently clip points that are going to be off screen?

I don't have a lot of time to change things around so it would be great if it were something simple to achieve.

You only need to create the VBOs and VAO once and then you can reuse them every frame.

void init(){

    glGenVertexArrays(1, &VAO1);
    glBindVertexArray(VAO1);

    glGenBuffers(1, &VBO1);
    glBindBuffer(GL_ARRAY_BUFFER, VBO1);
    glBufferData(GL_ARRAY_BUFFER, sizeof(data1), data1, GL_STATIC_DRAW);
    glBindBuffer(GL_ARRAY_BUFFER, VBO1);
    glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 0, 0);
    glEnableVertexAttribArray(0);

    glBindBuffer(GL_ARRAY_BUFFER, 0);

    glGenBuffers(1, &VBO2);
    glBindBuffer(GL_ARRAY_BUFFER, VBO2);
    glBufferData(GL_ARRAY_BUFFER, sizeof(data2), data2, GL_STATIC_DRAW);
    glVertexAttribPointer(1, 1, GL_FLOAT, GL_FALSE, 0, 0);
    glEnableVertexAttribArray(1);

    uniformLoc_centerX = glGetUniformLocation(program, "centerX");
    uniformLoc_centerY = glGetUniformLocation(program, "centerY");
    uniformLoc_sizeX = glGetUniformLocation(program, "sizeX");
    uniformLoc_sizeY = glGetUniformLocation(program, "sizeY");
    uniformLoc_size= glGetUniformLocation(program, "size");
}

void draw(){
    glUseProgram(program);
    glBindVertexArray(VAO1);

    glUniform1f(uniformLoc_centerX, centerX);
    glUniform1f(uniformLoc_centerY, centerY);
    glUniform1f(uniformLoc_sizeX, scrwidth);
    glUniform1f(uniformLoc_sizeY, scrheight);
    glUniform1f(uniformLoc_size, dotSize);

    glDrawArrays(GL_POINTS, 0, dot count);
}

If the data ever changes then all you need to do is:

glGenBuffers(1, &VBO1);
glBufferData(GL_ARRAY_BUFFER, sizeof(data1), data1, GL_STATIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, VBO2);
glBufferData(GL_ARRAY_BUFFER, sizeof(data2), data2, GL_STATIC_DRAW);


Path Tracer - implementation questions so I am currently in the middle of implementing a path tracer for my bachelor thesis. And so far I have understood and mapped the general flow of the program except for 2 things. Note that I'm using OpenCL and will be using OpenGL-OpenCL interop for rendering the path traced image to the window created by openGL.

Ok so here are my questions.

1)  First of all, I know that GPU memory is quite limited as compared to the CPU and it's one of the reasons we don't see GPU path tracers that much in professional industries like Film industry etc.

What I am interested in, is there any way to know the exact amount of memory available?

I'll be loading model files and sending this whole model data to the GPU probably in a data structure (K-D tree etc). Now these models will probably contain millions of triangles. How is the GPU supposed to store that much amount of data? Or is there any other efficient way of doing this?

Secondly I also read that accesses to global memory within a work-group can be coalesced/combined if they are done at the same time. So I was thinking to use a memory barrier and once all the work items have reached it, I can copy the whole K-D tree from global to local memory for faster processing ofcourse.

Again can the local memory hold such amount of data?

2) The second part I'm confused with is that, you usually hear that you turn on the path tracer and leave it for a few minutes and the Image gradually starts looking better and better as the samples are increased.

I don't understand yet, how should I be implementing this feature from coding point of view.

As far as I have planned out, there will be a single main kernel that will loop over the Image (shared between openGL and openCL) sending out a fixed number of rays (suppose 8) per pixel. After this kernel finish executing I can either blit the opengl FBO to the default or draw the texture on quads whichever is faster.

But all of this is just one instance. How can I gradually improve my image by increasing samples (1,2,4,8,16...) and each time blit the FBO to the default FB?

The only thing I can think of at this moment is to relaunch the kernel, compute a new color value for a specified pixel and then take the average of this new one and the old one. Is this feasible or is there a better approach?

I'm not an expert on path tracing, maybe someone with more experience can come up with a better answer, but I'll have a go at answering your questions.

Firstly, for the purposes of a college-project scale path-tracer, VRAM capacity shouldn't really be an issue. A vertex with position, normal, tangent and UV fits in 48 bytes, so with a GB of buffers you can hold ~20 million vertices, not counting the space for your index buffers and kd-trees of course. Even my crappy 5-year-old midrange laptop card has 2GB, and modern cards have significantly more, so i doubt you're gonna exhaust VRAM with model data.

As for the local memory question, short answer is no. Group shared memory for shader cores is tiny, you shouldn't be allocating more than a few KB per shader group, so there's no way you'd get your acceleration structures in there.

One of the main issues with path tracing is coherence, which is what makes it difficult to do on GPU. Since ray directions are chosen at random, each group of shaders processing a nearby group of pixels can end up sampling radically different parts of the scene, and hence memory. This can introduce a lot of branching and cache misses if not handled carefully, and negate a lot of the advantages of running on GPU.

The current state of the art for production path tracers is probably Disney's Hyperion. This is a CPU path tracer that runs on absolutely enormous scenes (think San Fransokyo in Big Hero 6), that are potentially too big to fit into even main RAM, and need to be streamed in from disk on demand ('out-of-core' rendering). They mitigate the coherence problems by first generating rays en-masse, then sorting them into bundles with similar position and direction, so that consecutive ray solves are likely to touch the same part of the scene and utilize the cache. Even so, they're bound by memory and I/O speeds, not compute power. Good write up on it here including a technical PDF.

For your second question, that is more or less the way it's done. You just accumulate the light from consecutive passes onto your colour buffer, and divide by the number of passes to normalize - you probably want to use 32-bit float colour channels for this. Naive path tracers (which is probably what you're building to start with) just choose a truly random sample distribution and weight each sample by its probability. Smarter approaches bias the sampling towards the more significant directions then remove the bias when summing the color. For example, for a diffuse surface, you could (a) pick truly random directions in the hemisphere and weight each sample by $N.L$ or (b) sample in a cosine distribution (more samples around the normal and less around the horizon) and apply full weight to each sample.

If you haven't seen it already, Ray Tracey's blog has some good tutorials on optimizing path-tracing for GPU. Good luck with your project.


Extract 3D seam coordinates from I have a .obj file with accompanying .mtl file and texture files. I want to extract the coordinates of the seams on my mesh. I then want to plot these seams in a 3D matrix in Python. Is there a fast way to do that?

Further clarification: I have found a way recreate my mesh in my Python IDE using a library called Pymesh. I now want to plot the seams of my mesh (the ones that will give me the unwrap that I have in my texture files) in a 3D matrix as well. One way I could do this is to write a script that takes the edge points of all my UV islands, and then searches the .obj file for the 3D coordinates that they map to (and translate that into coordinates appropriate for my 3D matrix in Python).
That will take a long time though - is there a faster way?
I also asked this in the Blender StackExchange as I wasn't sure which was more appropriate.

Thank you!

A seam can be defined as a line along the mesh where the vertices are doubled each with different texture coordinates.

So that's what you should look for. The set of vertices where there is a twin in the same position with different texture coordinates.

Depending on how the obj file was built that will be simple if they didn't duplicate the position when a vertex became part of a seam. Or it's not as simple because you need to deduplicate positions again.


Gamma correction is done in an image or a monitor? pretend the images and the monitor have same color profile

Is every image from the internet commonly got gamma corrected?
Or our monitor is automatically gamma corrected all the image showed up?

Is every image from the internet commonly got gamma corrected?

Every image is encoded with a color profile. Most color profiles do have a gamma correction, but not every color profile has that (raw for example). Most images on the internet are encoded in sRGB, which is gamma corrected. If you would download a random image from the internet, you will more than likely have an image that is in sRGB and thus gamma corrected.

  Or our monitor is automatically gamma corrected all the image showed up?

The monitor receives its image in the color profile of the monitor (let's just say sRGB), it then decodes the image from sRGB to the actual intensity of the pixels and then displays it. Basically, the monitor receives a gamma corrected image and removes the gamma correction just before it displays it.

Images that are saved to a file or that are sent to the monitor, are generally gamma corrected. This is for a very simple reason. We only have 256 values per RGB component to define the color of a pixel. Our eyes are very good at picking up small differences in dark parts of an image, but relatively terrible at picking up small differences in bright parts.

If we try to save an image without any gamma correcting as an 8-bit image (thus 256 values per RGB component), we end up with having the same amount of precision for dark parts in the image and bright parts in the image. However, our eyes cannot see small differences in bright parts of the image, so all that precision in the bright parts is simply wasted. Gamma correction fixes this. Before we convert the image to 8 bits, we first gamma correct it so that we allocate more precision to that darker parts of the image than the brighter parts of the image. When displayed, the monitor will remove that gamma correction.

Left is an image that was gamma corrected before being saved to 8 bits and right is an image that was not gamma corrected before 8 bits. You can clearly see the lack of precision in the darker parts without gamma correction, but not in the bright parts.

Gamma correction is a very useful trick to make our images with finite bits, look better.

If you want to read more about it, you can do so here.


Blend Shape, Pose Blend Shape, and Shape Blend Shape Since there is a lot of cross-section between computer vision and computer graphics for 3D pose and shape estimation of non-rigid objects, I would like to know what "blend shape", "pose blend shape" and "shape blend shape," exactly mean in 3D modeling of human shape and pose?

Blend shapes, also known as morph targets, are a widely used character animation technique that helps get around the limitations of linear-blend skinning (skeletal animation). A blend shape is defined as a set of per-vertex offset vectors, usually in a localized region of a character mesh, such as a neck or upper arm. The idea is that during animation, the basic pose produced by skinning is supplemented by adding these offset vectors, scaled by an animated weight factor, prior to applying the skinning transformations. This allows character artists to model such motions as the bulging/thinning of muscles when a limb is extended or contracted, or the shifting of soft tissue around the underlying rigid skeleton, which are not captured by skinning.
Blend shapes are also commonly used for facial animation, with different blend shapes representing different motions of the facial muscles, such as raising/lowering eyebrows, squinting the eyes, flaring the nostrils, moving the lips in various ways, etc. These are very difficult to model realistically using linear-blend skinning alone. By manipulating the weight factors of all these blend shapes, the animator can produce various facial expressions. A captured performance of an actor can be "solved" into animation curves for a predefined set of facial blend shapes, as a means of compression, or for transferring performance to a different face.
In the most basic version of blend shapes, their weight factors are animated by hand. But their weights can also be driven programmatically by some other set of parameters—for example, by the pose of the skeleton. Essentially, you can say something like "whenever the elbow is bent, apply the bicep-bulging blend shape", so that will happen automatically based on whatever the skeletal animation is doing, without having to animate it manually. This is broadly called "pose space deformation", since the blend shapes (deformations) are associated with regions in "pose space", the high-dimensional space of all the joint rotations/translations. The slide deck you posted refers to this as "pose blend shapes", however the more standard term is "pose space deformation" (and there's plenty of literature on it).
The "shape blend shapes" in that slide deck appear to be something specific to the pipeline they're describing, for creating a parameterized human model that can output many different body shapes and proportions. The "shape" parameters in the model presumably index the range of individual variation in body shape—different heights, limb lengths, hip to waist ratios, etc. They attempt to do this by starting with a single generic human mesh and applying various blend shapes (as well as moving joints around) to parameterize this space of body shapes.


OpenGL GLSL - Sobel Edge Detection Filter With respect to this topic  I've successfully implemented the Sobel Edge Detection filter in GLSL. Here is the fragment shader code of the filter:

#version 330 core
in vec2 TexCoords;
out vec4 color;

uniform sampler2D screenTexture;

mat3 sx = mat3(
    1.0, 2.0, 1.0,
    0.0, 0.0, 0.0,
   -1.0, -2.0, -1.0
);
mat3 sy = mat3(
    1.0, 0.0, -1.0,
    2.0, 0.0, -2.0,
    1.0, 0.0, -1.0
);

void main()
{
    vec3 diffuse = texture(screenTexture, TexCoords.st).rgb;
    mat3 I;
    for (int i=0; i<3; i++) {
        for (int j=0; j<3; j++) {
            vec3 sample  = texelFetch(screenTexture, ivec2(gl_FragCoord) + ivec2(i-1,j-1), 0 ).rgb;
            I[i][j] = length(sample);
    }
}

float gx = dot(sx[0], I[0]) + dot(sx[1], I[1]) + dot(sx[2], I[2]);
float gy = dot(sy[0], I[0]) + dot(sy[1], I[1]) + dot(sy[2], I[2]);

float g = sqrt(pow(gx, 2.0)+pow(gy, 2.0));
color = vec4(diffuse - vec3(g), 1.0);
}

And here is the result of a cube with Sobel edge detection:

If you enlarge the picture, you will see that there is a lot of "noise" produced by Sobel:
There are gray horizontal stripes all over the scene due to the blue/white gradient. Futhermore, the light cones produce an unwanted pattern on the cube. The black edges on the left of the cube also seem to fade because of the light cone on the left half of the cube.

So I read this article which stated that one should grayscale the image first and use a Gaussian blur filter to make the edges more apparent. On the bottom of the article, there is also the canny edge detection filter which seem to produce better results.

Now I have two questions:

Are the following steps correct to produce the best possible edge detection results:

Grayscale
Gaussian Blur
Sobel/Canny edge detection

If yes, how would I merge the original image with the processed image? I mean, after processing the steps stated above, I get an image that is either completely black with white edges or vice verca. How would I put the edges on my original image/texture?

Thanks for your help!

The best results strongly depend on your use case. They also depend on what effect you want to achieve. Sobel is just an edge detection filter: the edges will depend on the input signal, choosing that input signal is up to you.

Here you are using the color image as an input, and the filter rightfully detects faint edges in the blue gradient, while the edges of the cube get interrupted where its color is too close from the background color.

Since I assume your program is also responsible for drawing the cube, you have access to other information that you can feed to your Sobel filter. For example depth and normals are good candidates for edge detection. The albedo before lighting could be used to. Test with different inputs and decide which ones to use depending on the results you get.
Regarding your question about how to combine the edge information, I suggest you filter g a little before using it. Then you can use it to interpolate between the original color and the wanted edge color.

For example you could try something like this:

    float g = sqrt(pow(gx, 2.0)+pow(gy, 2.0));

    // Try different values and see what happens
    g = smoothstep(0.4, 0.6, g);

    vec3 edgeColor = vec3(1., 0., 0.2);
    color = vec4(mix(diffuse, edgeColor, g), 1.);

Addendum

To use depth or normals, you would need to save them in a texture if that's not done already. When you create the frame buffer for your regular rendering pass, you can attach various textures to it (see glFramebufferTexture2D) and write other information to them than just the color of the scene.

If you attach a depth texture (with GL_DEPTH_ATTACHMENT), it will be used automatically for depth. If you attach one or several color textures (with GL_COLOR_ATTACHMENTi), you can write to them by declaring several outputs to your fragment shaders (this used to be done with gl_FragData; either way, see glDrawBuffers).

For more information on the topic, look up "Multi Render Target" (MRT).


How do you compute the winding number of a closed poly curve? Pretty much the title, given a closed curve in 2D, defined by a set of points, and a point. What's the algorithm to calculate the winding number of that curve, point pair?

Winding number as in https://en.wikipedia.org/wiki/Winding_number?

Unless I'm mistaken, one way is to fire, say, a horizontal 'ray' out from your point towards, say, $+\infty$. Each time you cross an edge, $(V_n V_{n+1})$ that is going up (i.e.$V_n[y] <V_{n+1}[y]$) , increment a counter, and decrement if the edge is going down.

The special cases of horizontal edges and vertices that lie on the ray are left as a (not too difficult) exercise for the reader.

If you want to do it quickly/approximately with a graphics API, you can use stencil buffers.


How does ID3D12Resource::Map work? How does Map() work? I would guess that it allocates memory to fit the resource size (or a range of it), but when are the bytes sent to the GPU?

There are two general ways the graphics driver can implement Map().

One way is to set up a virtual memory mapping that makes the resource's actual memory (could be system memory or GPU memory) visible as part of the application's address space. If it does happen to be GPU memory, any writes you do would typically be buffered up and sent across the PCIe bus to the GPU, using write combining. (Even on a unified-memory system, mapped resource memory probably has some different caching behavior from "regular" memory, since CPU and GPU caches aren't coherent with each other.)

The other way the driver can implement Map() is to allocate memory and create a local copy of the resource, let your app edit it, then copy the modified memory back to the resource when Unmap() is called. This might need to be done if the actual memory layout of the resource on the GPU is "weird" somehow, like having a nonstandard tiling scheme or some such.

The copying approach is clearly more expensive due to the extra copies required, so I'd expect the driver to use the memory mapping approach whenever possible.


How to raytrace Bezier surfaces? I tried this question on math.SE and surprisingly, the answer was "the equations are too nasty, just feed the function it to a numerical root-finder". But if you consider yourself "a graphics guy" like me, and have played extensively with Bezier curves for design work, I got to believe that better can be done. There is a published algorithm by Kajiya that I don't have the background to understand (Sylvester Matrices), but the related advice on math.SE was that the result is a degree-18 polynomial in t, and you still need to solve that numerically. I had another idea with similar result.

So, is it a total pipe dream to hope to solve the Ray/Bezier-surface intersection algebraically, thus making it possible to code explicitly and have super-fast super-smoothness?

Barring that, what's the fastest method for performing this calculation? Can you "find the wiggles" to get a tight bound (and target) for recursive subdivision? If you have to use a numerical root-finder (sigh), what properties does it need and is there a best choice for speed?

My original thought was about preparing for a specific surface, similar to Laplace expansion as described in the answer to my other math question about triangles. But I'd be interested in general methods, too. I'm just thinking of a fixed set of shapes, like the Utah teapot. But I'd be very interested in ways of optimizing for temporal coherence across animated frames.

First off, here's the Kajiya method I think you're thinking of: Kajiya, Ray Tracing Parametric Patches, SIGGRAPH 82. The tech report version might be more informative.

What I hope you get from that is that it's not impossible and it's not conceptually difficult if you don't mind getting your hands dirty with some algebraic geometry and complex numbers. However, doing it directly is absurdly expensive.

"Real" ray tracers tend to do some combination of two things:

Placing a bounding hierarchy (e.g. AABBs) on the patch to get a good "initial value" for a numeric root finder. If you do this well, you can avoid the "wrinkle" problem.
Tesselating the patch into DDG shells and ray tracing them like polygon meshes.

That last point sounds like it kills the "super-smoothness" requirement, but it isn't nearly as bad as that if you're using ray differentials. Matching the tessellation level to the "size" of the ray bounds the error nicely. Besides, you probably need differentials for texture coordinates anyway, so you might as well use it to control the accuracy of the intersection test too.

Exploiting temporal coherence isn't a bad idea, but exactly how you'd do that depends a lot on your scene graph representation. You might want to look at ray coherence. Ask your favourite search engine about ray packet tracing and ray reordering.


Calculate a rotation around an arbitrary axis Suppose I have two 3D triangles before & after a single rotation around an unknown axis.
How would I go about finding this axis?

Take a point $P$ and it's rotated point $P'$.

Find the plan that runs through the middle between them $C = \frac{P+P'}{2}$ and is perpendicular to the line connecting them.

Do this for all 3 of them and find the line of the planes' intersection. That will be the rotation axis.

If all mid-planes are coplanar then you can use the planes of the triangles themselves.

To get the angle you can take a $P$ and $P'$ again and project them onto the axis.

Take a point $A$ on the axis and the direction $v$ of the axis. The projected point is $P_p=A+\frac{v \cdot (P-A)}{v\cdot v} v$. And then the angle is $acos(\frac{dot(P-P_p, P'-P_p)}{|P-P_p|* |P'-P_p|})$


Why is the CMYK space four sided in chromaticity charts In chromaticity charts the CMYK color space is usually four-sided. I've lately had to read quite a few charts and it suddenly dawned to me that I don't really understand why. It has just not been something I've needed to think about until now.

Image 1: A sample CIE31 chromaticity graph showing the spaces*.

One of my working hypotheses is that it is actually a triangle. This triangle just clips for some reason. But there might be other reasons such as non-ideality of the color primaries, or bad plane. So why is it squarish?

* This image is not so clearly a polygon, like it's usually depicted as, but it's the best one I had at hand.

There are two aspects to this:

The traditional chromaticity graph is actually a two-dimensional slice of a three-dimensional volume.  The slice usually used is one that matches up (more or less) to the fully-saturated primaries of an RGB-based colorspace; the full RGB colorspace is a tetrahedron.
CMYK is a four-primary colorspace, with primaries that are not linearly independent.  This makes the full volume a complex shape.  Further, at least one (black) and possibly all four of the primaries don't lie on the plane of the 2D chromaticity graph.

The CMYK space is four-sided because it's a 2D slice of a complex 3D shape that (probably) doesn't lie on any of the surfaces of that shape.


Why does checkerboard pattern on a computer screen appear with a yellowish tint? This question was originally asked on Physics, then moved to Cognitive Sciences.

Consider the following image:

You might want to display the image in a new page, in case it gets resized for mobile displays.

On the top half, there is a pixel-sized checkerboard pattern with alternating black and white pixels; on the bottom half, there's a black to white gradient. Now, I don't know if you see the same, but for me, when viewing from distance or defocusing my eyes and the top half blending into one color, I can't find any color in the gradient below which would match it.

With simple arithmetics, I would guess at first guess the resulting color be either RGB $(0.5, 0.5, 0.5)$ or, with gamma, $(\sqrt{0.5}, \sqrt{0.5}, \sqrt{0.5})$. I cannot help myself, but the resulting color appears a lot warmer than the metallic gray which would appear if you for example zoomed this page or used a blur filter. I tried to add some yellow to the gradient, and the result looks more similar to the perceived color.

Now, based on the comments, it appears some people perceive yellowish tint and some don't. And I do on my LCD computer screen but not on my mobile display. Thus I guess it's based on some property of the display.

Why don't we perceive the resulting color like real gray? Where does the yellow color come from?

I have a theory: Based on the color arrangement in a typical LCD pixel, one white pixel would contain the red and green color together and blue on the right side. Blue color appears darker to human eye than colors with the same physical intensity, so a white pixel is more green than red than blue. Green and red have roughy the same perceptional insensity, and mixing red and green color addivitely gives yellow, thus the yellow tint? Well, shouldn't then all white pixels on the display appear a bit yellow?

Is this possible? Are there any other explanations?

A side question: Do you know any computer image scaling algorithm or blur filter that tries to mimic this, simulating a blurry vision of a human eye correctly?

Because your monitor is not properly calibrated.

On my screen at home the top and bottom parts have the same hue. At my office though, the top part tends to looks a bit yellow compared to the bottom part that looks more red.

The difference: my screen at home was from a series that was decently calibrated out of the factory, and on top of that I did calibrate it properly with a color calibration tool. That was quite some years ago and its colors have probably degraded since, but it still makes a difference.

A poorly calibrated monitor will not display exactly the color intensity requested for R, G and B, resulting in differences between the color that should be displayed and the one actually displayed. It can even vary depending the area of the screen (many consumer level LCD screens tend to have more light leaking near the edges for example).

The image you included is a good example to highlight a calibration issue, because it allows to compare directly the average color at 50% intensity (combining two halves at 0% and 100% intensity) with the color displayed when requesting 50% intensity. Moreover, the gradient allows to see if the hue is constant from 0% to 100% intensity.


How are we supposed to fix brightness with square roots? I recently watched this video that talked from a physics perspective how most of the ways we deal with color on the computer is incorrect because brightness is on a logarithmic not linear scale.  Being a novice graphics programmer I want to know how to deal with the information to make my graphics even better. I figure ray-tracers but for rasterizing, blur, grading, and light adjusting like HDR what am I supposed to do with this information?

So I guess I have some questions. (sorry its a lot)

Most texture data is stored in a format where the rgb brightness is incorrect because it has not been square rooted (right?) therefore if we want to display it correctly
When using textures is the gaussian formula technically incorrect? If so what would be the mathematically correct method.
What part of this problem is the monitor responsible for? Does it square or squareroot the values it gets?
What part of the problem is the GPU responsible for when you ask it to sample a color at a point? Does it square or squareroot the values it gets?
Do people know about this property and just not act on it because square rooting a number is an expensive computation? If so do we have a good way to quickly approximate the squareroot of numbers from 0-1?
How do you apply the knowledge from that video in your graphics?

What the video is talking about is called gamma correction and it's a very familiar topic for graphics programmers. The first 30 minutes of John Hable's talk on Uncharted 2 rendering is my favorite introduction to "why graphics programmers should care about gamma", as well as being a good introduction to HDR rendering. If you don't want to watch the talk (though I highly recommend it), The Importance of Being Linear is also a good article, and Charles Poynton has a Gamma FAQ that goes into more technical detail and explains the terminology used.
By the way, square/squareroot is only an approximation; the sRGB gamma curve is what's most commonly used in practice (see the formulas there mentioning $C_\text{srgb}$ and $C_\text{linear}$). See also this related question about gamma and color spaces. It's a bit more complicated of a topic than the video makes it sound. :)
To answer your specific questions:

Yes, most image data is stored in gamma format, so the pixel values do not represent linear luminance. I wouldn't describe it as "incorrect", since gamma encoding has the useful property of putting more values in the darks, which is better for perceptual precision (as mentioned in the video). Gamma encoding is absolutely necessary to make good-looking images with only 8 bits per RGB component; if you stored linear values you would need more like 12 bits per component to get enough precision in the darks.

To be more physically correct, when blurring an image, one should convert it to linear luminance, then do the blur (Gaussian or otherwise), then convert it back to the original color space.

The monitor accepts image data in gamma format. So the pixel values do not represent linear luminance, and the output brightness is not linearly proportional to the pixel values.
If it's a good-quality, calibrated monitor, then it accepts data in a specific color space such as sRGB or Adobe RGB and reproduces the colors accurately based on the definition of that color space. If it's a crap uncalibrated monitor, then who knows what you'll get, but it'll be something kinda like sRGB.

GPUs can be configured to do various things here. Modern GPUs have hardware support to automatically convert values between gamma and linear when sampling a texture, or storing a value out to the framebuffer, but the programmer has to explicitly enable that using an API such as Direct3D or OpenGL.

Computer graphics people certainly do know about this topic, and it's critical to get it right when building a realistic renderer for film, AAA games, and so on. However, a lot of more "everyday" apps like web browsers and image editors (even Photoshop, as the video mentions) don't really handle gamma correctly.
It's not an issue of processing power; maybe that was a problem 20 years ago, but today even mobile phones have ample processing power to convert images between gamma and linear. The reason why web browsers and image editors still don't get this right is some combination of (a) ignorance, (b) not believing that gamma is important enough to be worth getting right, and (c) simply being stuck with bad behavior because it's been that way for a long time and users have come to expect it, so they freak out if something changes.

As mentioned, we have to think a lot about gamma when building a renderer. Most computations (e.g. lighting) should be done in linear space, but the input textures and output image should be in gamma space. Also, maybe some parts of rendering such as UI compositing should happen in gamma space because it's more familiar to artists (it works more like Photoshop layer compositing and such). Also, we have to decide if internal render targets are going to be stored in linear or gamma space, as it will affect precision. And we have to get all the conversions right, so that data gets converted between linear and gamma at the right spots in the rendering process.


How to test if one object A intersects and is therefore partly or wholly inside object B I have the world point positions of two objects, and would like to test if there is an intersection between the two objects from this information.

What methods can I use to test this, especially at the boundaries.

Do you need exact triangle - triangle intersection test that is do you need to know exactly the points of intersection of each object? If you just need a simple yes / no they have intersected then you could get away with creating an AABB ( https://en.wikipedia.org/wiki/Bounding_volume ) that encompasses each object and do a AABB -> AABB ( good info here https://studiofreya.com/3d-math-and-physics/simple-aabb-vs-aabb-collision-detection/ ) intersection test? even if you need a more complex collision check Triangle -> Triangle it is always a good idea to do a simple collision check first to make sure its worth doing the complex Triangle -> Triangle intersection tests. Look here ( http://www.realtimerendering.com/intersections.html ) for all the normal object -> object intersection tests you could possibly need


When should quaternions be used to represent rotation and scaling in 3D? Quaternions (a four-dimensional extension of complex numbers) can used to represent rotation and scaling of a 3D vector, and the application of a quaternion onto a 3D vector involves two quaternion multiplications, thus requiring fewer operations than multiplication by the corresponding transformation matrix. However, linear and affine transformation matrixes are often used instead, especially in shader code.

When is it appropriate and preferable (due to speed, stability, etc.) to use a quaternion to represent scaling and rotation in three dimensions, instead of the corresponding transformation matrix?

I want to start with misconceptions:

Modern GPUs (NVIDIA for quite a while, and AMD since Southern Islands) do not meaningfully support vector/matrix operations natively in hardware. They are vector architectures in a different direction: each component of a vector (x, y, z) are generally 32- or 64-valued, containing values for each element in a lane. So a 3D dot product is not usually an instruction, it is a multiply and two multiply-adds.

Additionally, counting primitive operations like multiply-add, transforming a vector by a quaternion is more expensive than transforming a vector by a matrix. Transforming a vector by a 3x3 matrix is 3 multiplies and 6 multiply-adds, and transforming a vector by a quaternion is two quaternion multiplies, each of which consist of 4 multiplies and 12 multiply-adds. (You can get less naïve than this—here's a writeup on a faster way—but it's still not as cheap as multiplying a vector by a matrix.)

However, performance is not always determined simply by counting the number of ALU operations it performs. Quaternions require less space than the equivalent matrix (assuming you are only doing pure rotation/scale), and that means less storage space and less memory traffic. This is often important in animation (which is conveniently also often where the nice interpolation properties of quaternions show up).

Other than that:

Matrices use more space because they support more operations. A 3x3 matrix can contain nonuniform scale, skew, reflection, and orthogonal projection.
Matrices can be naturally thought of as basis vectors, and easily constructed from those vectors.
Multiplying one quaternion by another (composing two rotations) is less operations than multiplying one matrix by another.


Clashes when combining openGL and GUI For the fun of it, I've taken up a task of writing a level editor for a voxel game.

The idea is to have window that contains two main components. First component is openGL viewport placed in lower left or right corner of the window (or anywhere, really). The rest would be occupied with the second component, you guessed it, the GUI. It would consist of the menu bar, status bar and various options to select tools to use. It could also contain dialog boxes to manage metadata contained in the map. The GUI stuff. It would have this general look:

The question here is whether those two would get along. I've been googling for a while and have failed to find any specific info on openGL to GUI interaction and vice versa.

Am I free to just use any GUI library with openGL, limiting the latter to a part of the window, or are there some things I'd have to look out after. I know it's a bit of a general question, but I really didn't manage to find any specific information on this.

General answer is that creating a window as a GUI element with some GUI designer library involves subclassing some basic GUI element (like widget in QtCreator for example) with class providing handling GL context in more or less user friendly way (like QOpenGLWidget in QtCreator which provide some user friendly wrapper or on the other hand basic Win32 API handle) and override methods for handling its framebuffer drawing or resizing, swapping buffers etc. with usual gl extensions provided by some lib you use.

This will work perfectly fine with other GUI elements.

I don't know which GUI lib or IDE with GUI designer or language you prefer but most good libs can handle this. You can even do it in Assembler with WinApi GUI elements.


What are the parameters measure the performance of a rendering? Let's say I designed a housing model and rendered it using a rendering engine. how can I measure the performance of my rendering?

I assume you mean performance of the numerical algorithm responsible for generating the image?

Usually the performance of a rendering algorithm is assessed through a pixel-wise comparison with a ground truth, converged image. This image is typically rendered for a very long time using a standard and (ideally) unbiased technique like unidirectional path tracing. For a fixed time and sampling budget, you then run your custom algorithm and track a meaningful error (e.g. relative MSE) between your intermediate image and the ground truth image. Plotting the error over time on a logarithmic scale should give you a sense of how your algorithm is performing.

When comparing algorithms, you need to make sure that you are actually comparing images that were output by the same rendering engine. This is crucial if you want to report a faithful metric, otherwise the comparison is unfair and inherently depends on the implementation. For instance, if rendering engine $A$ takes twice as long as engine $B$ to fetch and load a texture, your reported error will be higher, even though the main algorithm you are testing is the same (assuming you are treating materials as blackboxes). The main downside is that rendering researchers are typically required to implement other algorithms in their engine to report the true performance of their new algorithm.


Calculate BRDF and PDF when doing direct light sampling I learned in PBRT that to sample direct lighting for a surface a integrator usually calculates the sample as
$$
Sample = \frac{L_d(p,\omega_r) f(p,\omega_o,\omega_r)|cos\theta_i|}{p(\omega_r)}
$$
where BRDF $f(p,\omega_o,\omega_r)$ is the sum of brdf from all the components (for instance microfacet specular & lambertian diffuse), and PDF $p(\omega_r)$ is the average of all the components. My question is why for $f$ we calculate sum while for $p$ it's average. I have the intuition that for $f$ we should also calculate average.
In addition, why can we use the distribution of importance sampling to evaluate $p(\omega_r)$ for the given $\omega_r$?

1) When we have multiple BRDFs, adding thier components together can break energy conservation, as I think you suspect. However most renderers will handle this problem when constructing the layered BRDFs. For example you would might weight each layer or use a fresnel term to decide the weight for each layer.

Here is the spec for arnolds standard shader:

https://autodesk.github.io/standard-surface/

Each layer is multiplied by 1 - previous layer weight, which the artist is setting. This is done inside the fr function, which means that energy conservation is solved here.

2) The pdf is an average, I think, if each bxdf has an equal chance of being sampled. You therefore average the pdf even though each layer may contribute different amounts.

Note: Becareful which value you use for p(wr), this will be the pdf you used to sample your direction, so either the bxdf pdf or the light pdf. (You still need to calculate both pdfs at some point for MIS though.)


Why is the z direction away from the screen/viewer? I wonder what is the reason that when z increase, the objects appear far away and when z is negative, the object is behind the camera?

If you want to use the traditional x is towards right and y is up you are used to in math and physics classes. Now if you follow a right handed coordinate space, which is also commonly used. Then the direction of the z follows from the x and y direction.  In this sceme positive z is pointing out from the screen.

Now you can also choose a left handed coordinate system in which case the z is reversed. Its just a matter of choosing which convention you want to use.

Likewise if you choose y to be down, ala traditional windowing math then z is reversed even if you use right hand rule.

None of these conventions are set in stone you can do whatever you want. A bit like you can choose row major or column major matrixes. You just need to know what comvention you follow.


How do I use texture coordinates of a .obj file in OpenGL? I have an obj file, with texture coordinates (indicated by vt). How would I use this to load textures in OpenGL?

The OBJ file itself doesn't contain the texture, only the texture co-ordinates (which describe how to map a texture onto the surface of the mesh.) The texture is an image file (such as a PNG) whose path is given by the accompanying .mtl file. You need to parse the .mtl file to get the path to the image file, then use an image-loading library (such as DevIL) to load the image into memory. Once you have the pixels of the image, you can use glTexImage2D to load that image into a texture, which can be accessed from your fragment shader using a sampler2D.


Path Integral Form with Path Tracing The description is going to be long and detailed, thanks in advance for your patience!
I am learning the path integral form of light transport equation(LTE) to be able to follow recent rendering papers. The materials I am using are PBRT(chapter 14.4-14.5: link) and SIGGRAPH 2013 course: link.
The path integral form of LTE is:
$$ I(x) = \int_{\Omega}f(\overline{x})d\mu(\overline{x})$$
From the SIGGRAPH course's slide, we know that we can estimate $I$ by randomly sampling a path, computing its contribution $f$, and dividing by its probability $p$:
$$ \overline{I}(x) = \frac{f(\overline{x})}{p(\overline{x})} $$
From the PBRT's notes and its implementation, it expands the path integral into an infinite sum of integrals on specific lengths of paths. This notation is well shown in the path tracing implementation since the i-th bounce computes the i-th integral and simply summing them up will the final estimate of $I$.
$$ \overline{I}(x) = \sum_{i=0}^{\infty}\overline{I_{i}}(x) $$
With these two notations based on the path form integral of LTE, my question is how to represent path tracing as the first one(the SIGGRAPH course notation) since most PT implementations I've seen so far is based either on the PBRT's notation(the sum notation) or other notations of LTE.
I have one possible idea that is to follow the formula by generating a path and computing its pdf, but I am not sure about its practical implementation. Should I generate paths for different lengths and do this procedure over and over(since this looks more like the PBRT's notation), or a single path with the correct pdf will suffice?
Or, from another perspective, I think that to apply the SIGGRAPH course's notation in practice, one has to randomly generate paths with different lengths and sum their estimate up since each length corresponds to one term in the expanded notation. Is this correct?

Q1: how to represent path tracing as the first one...
Your materials have brought enough information, the first formula, quoted from your material, presents as

The path integral formulation of light transport formalizes the idea
by writing the camera response as an integral over all light transport
paths of all lengths in the scene, which given path encompasses

the “amount” of light emitted along the path,
the light carrying capacity of the path, and
the sensitivity of the sensor to light brought along the path.

Can't interpret the formula better
Q2: Should I generate paths for different lengths and do this procedure over and over...
Q3: ...randomly generate paths with different lengths and sum their estimate up since each length...
Quoted from your material,

The Monte Carlo integration procedure consists in generating a
‘sample’, that is, a random x-value from the integration domain, ...

I can't say the second one is a formula, it is just the Monte Carlo estimator of the path integral formulation of light transport or it is just a method for programming the formulation; or course you could have other methods.


Creating a gently moving 2D fog effect I am trying to create a fog-like effect, similar to that found using perlin noise. I know where I can find information about that effect, but it's harder for me to tell if it's an efficient route for what I want to achieve. A similar effect can be seen here, although I'd imagine what I want is much lower in detail: Animated perlin noise | youtube.com

I am currently working in openframeworks. What I have done so far is build a grid of noise values, and have worked towards linearly interpolating in-between those. I feel like I am on the right track, but it very blocky, which makes me wonder if my approach is totally off-mark. This page on Perlin noise has been helpful: Understanding Perlin Noise | flafla2.github.io

for(int x = 0; x < grid.getWidth(); x++){
    if(x % g_size == 0){
        for(int y = 0; y < grid.getHeight(); y++){
            if(y % g_size == 0){
                noise = ofNoise(x,y,ofGetElapsedTimef());
                grid.setColor(x,y,ofFloatColor(noise));
            }
        }
    }
}

ofVec2f g1, g2, g3, g4;
float f1, f2, f3, f4, u, v, t_1, t_2;

for(int x = 0; x < grid.getWidth(); x++){
    for(int y = 0; y < grid.getHeight(); y++){
        //find neigboring grid points
        g1 = ofVec2f(x-(x % g_size),y-(y % g_size));
        g2 = ofVec2f(g1.x + g_size,g1.y);
        g3 = ofVec2f(g1.x,g1.y + g_size);
        g4 = g1 + g_size;

        //get noise values from the grid neighbors
        f1 = grid.getColor(g1.x,g1.y).r;
        f2 = grid.getColor(g2.x,g2.y).r;
        f3 = grid.getColor(g3.x,g3.y).r;
        f4 = grid.getColor(g4.x,g4.y).r;

        u = (float)(x % g_size)/g_size;
        v = (float)(y % g_size)/g_size;

        t_1 = ofLerp(f1,f2,u);
        t_2 = ofLerp(f3,f4,u);

        ofColor average = ofLerp(t_1,t_2,v);

        fog.setColor(x,y,average);
    }
}

Animated noise can be created by using time as an extra dimension. So instead of 2D noise, you'd use 3D noise with time as the z-axis position, like ofNoise(x, y, time).

To control the level of detail, you'd use octaves of noise: multiple noise layers with different scales and amplitudes, mixed together. The basic Perlin routines just generate a single octave, so you can re-invoke it with scaled coordinates to add more detail, something like:

float noise = ofNoise(x, y, time);          // first octave
noise += 0.5 * ofNoise(2*x, 2*y, 2*time);   // second octave
noise += 0.25 * ofNoise(4*x, 4*y, 4*time);  // third octave

and so on. The more octaves you put in, the more detailed the noise will be. The article you linked describes this in more detail if you scroll down to Working with Octaves.


Can I have lower color bit depth in sample buffer than target framebuffer? I´m looking into uses of high amount of samples in multisampling. If I have an RGBA8 framebuffer and render to it using multisample with many samples, say 32, dithering on per-sample level should yield a reasonably accurate 8-bit color from the 32 samples in 4-bit color when resolved. The spec (4.5) seems a bit scary though (glBlitFrameBuffer):

  If the read framebuffer is multisampled (its effective value of
  SAMPLE_BUFFERS is one) and the draw framebuffer is not (its value of SAMPLE_BUFFERS is
  zero), the samples corresponding to each pixel location in the source are converted
  to a single sample before being written to the destination. filter is
  ignored. If the source formats are integer types or stencil values, a
  single sample’s value is selected for each pixel. If the source
  formats are ﬂoating-point or normalized types, the sample values for
  each pixel are resolved in an implementation-dependent manner. If the
  source formats are depth values, sample values are resolved in an
  implementation-dependent manner where the result will be between the
  minimum and maximum depth values in the pixel.

What does that even mean?

Is there a way to do it in a cross implementation defined manner?

What does that even mean?

It means that multisampling is resolved.

A multisample framebuffer contains multiple samples for each pixel. The combination of samples represents the color of that pixel. The process of converting from multiple samples to a single color is multisample resolving.

By blitting from a multisampled buffer to a non-multisampled buffer, you are telling OpenGL to resolve each pixel's worth of samples into a single sample. Typically, this is done by adding all the sample values up and dividing by the number of samples, but implementations can take weighted averages or something else, depending on the precise nature of each sample.

Generally speaking, it's best that you let the implementation do its job.

  Is there a way to do it in a cross implementation defined manner?

You can program your own resolution by rendering with a multisample texture to a non-multisampled texture. In the FS, you read each fragment's worth of pixels, doing whatever operation you would like to combine them.

But that's probably not going to be as fast as letting the hardware do the multisample resolve itself.


SSBO get block of data instead of iterating I have a mesh, each face (triangle) of the mesh needs to read some information out of a SSBO object.
Is there a way to read out a block (lets say 100 floats) at once? Or do I need to iterate over the 100 floats? The number of floats is not fixed. They vary for each triangle.
During this stage the 100 floats are not changing at all... So to iterate over them would make no change in the logic. But I think it takes to mush time to read them after each other.
I also have a second question:
when reading lets say 128 floats from SSBO iteratively does it takes longer than reading 32 vec4 iteratively? in principle both is the same (128 floats)

yes, and no. Loading the floats into the SSBO is loading them for the shader so yes you still have to iterate over them. And GPU's generally are good at latency hiding so whether or not you read floats or vec4's the GPU should be able to read them in just as fast. But processing them as vec4's instead of 1 float at a time will be faster since the code is taking advantage of the GPU's vector processing.