World coordinates, Normalised device coordinates and device coordinates Can I get to know the difference between the three? A good example would add up too.

World coordinates are at the bedrock of your game world. The 3D positions of all objects are ultimately specified in world space, either directly or through a node hierarchy. The ground, buildings, trees, and other stationary things will be fixed in world space. Gameplay calculations and physics will be done in world space (perhaps with some local recentering for precision purposes, if the world is large). The axes of world coordinates are often taken to represent compass directions, such as X = east, Y = north, Z = up.

Local coordinates are attached to an object (there is one local coordinate space for each object). The axes may represent something meaningful to the object, for instance X = forward, Y = left, Z = up for an object such as a character, vehicle, gun, etc. that has an inherent orientation. As the object moves around, the relationship between local and world space (as expressed by a transformation matrix) will change. For instance if you flip your car upside down, its local Z axis ("up" in local space) will now be pointing "down" in world space.

Camera or view coordinates are local coordinates attached to the camera. This is still a 3D coordinate system, without any projection or anything, but with the axes aligned to match the screen orientation: usually X = right, Y = up, Z = backward. The transformation from world to view space is often known as the "view matrix".

Clip space coordinates are the coordinates output by a vertex shader: coordinates to which the projection matrix has been applied, but not the perspective divide. This is a 4D (homogeneous) space. (World, local, and view space are 3D with an implicit w = 1.) It's so named because this is the space in which view frustum clipping and culling takes place (conceptually, at least).

Normalized device coordinates, also commonly known as "screen space" although that term is a little loose, are what you get after applying the perspective divide to clip space coordinates. The 3D coordinates now represent the 2D positions of points on screen, with X and Y in [−1, 1], together with the depth within the depth buffer range, Z in [0, 1] for D3D or [−1, 1] for OpenGL. The axis orientation is X = right, Y = up, and Z can be either forward or backward depending on the depth buffer configuration.

Device coordinates are 2D pixel coordinates within the render target, with (0, 0) in the upper-left corner, X = right, Y = down. One arrives at device coordinates by applying the viewport transform to the normalized device coordinates; the viewport controls at what pixel offset and resolution the image appears in the render target. One thing that's important to note is that device coordinates are not integer values; they aren't indices of pixels. They're a continuous 2D coordinate system, using pixel-sized units, but in which fractional (subpixel) values are perfectly valid. Pixel centers lie at 0.5 offsets (like 0.5, 1.5, 2.5, ...) in this coordinate system.


How to crop with edge-detection using imagemagick Let's say I have an image of an object against a white background. I want to crop so that the image fills the available space.
This takes 10px off:

convert original.png -shave 10x10 shaved.png

How would I implement something like this that detects the edges of the object, thus cropping a variable number of pixels from each side?

Try -trim instead of -shave:

convert original.png -trim trimmed.png

See the docs for this and other options.


When to disable perspective correct interpolation ( noperspective ) In GLSL, perspective correct interpolation of vertex attributes is the default setting - one can disable it for specific vertex attributes by using the noperspective qualifier. Other than in post-processing shaders, I've never seen the perspective correct interpolation disabled - are there any other use cases? Also, does it even make a difference, performance-wise?

Use cases are only limited by your imagination! noperspective means that the attribute is interpolated across the triangle as though the triangle was completely flat on the surface of the screen. You can do antialiased wireframe rendering with this: output a screen-space distance to the nearest edge as a noperspective varying and use that as coverage in the pixel shader.

Or if you're doing non-photorealistic rendering and want a pattern in screen-space like halftoning, you can enable noperspective on your UVs used for texturing.

Does it make a performance difference? Probably, but you probably won't notice (with the potential exception of less powerful graphics hardware). Most GPUs are composed of a series of pipeline stages that execute in parallel, and in some sense you only pay the cost for the most expensive stage. If rasterization is the most limiting part for you, then you may see a difference from the divisions that you're skipping per-pixel. I would guess that is most likely when rendering a shadow map or a depth prepass, but those also have the fewest attributes to interpolate.


Should not specular reflection on a rough surface be similar to diffuse reflection? Different BRDFs are usually used to compute diffuse and specular reflection.
Some of the most often used include for example the Lambert BRDF for diffuse reflection and the Cook-Torrance BRDF for specular reflection.
Cook-Torrance BRDFs are parametrized by the surface roughness: the rougher the surface, the "blurrier" the reflection.

From a conceptual point of view, should we consider that for an extremely rough surface, the specular reflection must look like a diffuse reflection? Can there be a continuity between both components of the reflection?

No, because the underlying physics is not the same, nor the lobe shape - not to speak of their parameters such as color and Fresnel term.

Specular is really true surface interaction with the interface material/air, so it has Fresnel modulation and the internal medium has no influence on colors. But the surface condition strongly influence the reflectance, of course. Think of it as the reflections on ocean surface.

Diffuse is due to the subsurface scattering, light entering the medium, and thus gains the color characteristic and loose directionality. Think of it as the green color of the water turbidity. If depth of penetration is less than pixel size CG people don't call it "subsurface" but the physics is the same.

Of course diffuse input is what specular let pass, which is angle (and polarization) dependant.

Beside, there exist transparent medium with internal interfaces (e.g. thin shells), so case exists were light goes inside but still acts specularely (or even wavely) and not diffusely. Oppositely, most plastics and paints/varnishs and biological medium are intrinsically transparent, but contain pigments (often based on a metal atom) that cause the opacity (diffusion or specular on these "objects in the object"). Think of them as the fishes in clear water :-) .


I dont understand how a GPU computes and branches, could you please help me fix this shader preformance I have been working on how the GPU does parellel processing, and branching. However I am not yet to the point where I know how to make this shader more efficent. Essentially I dont know enough about how it works.

Anyway I have the following shader:

precision highp float;

varying vec3 o_color; //Color it should be
varying vec2 texture_coord; //Interpolated value 0-1
varying float ratio_out; //This is the height / width of the capsule on screen
varying float rdistOut; //This number is used to fill in all pixels when particle is slow enough for the geometry to be a square but still has some distance from last position.

void main()
{

    vec2 yCoord = vec2(0.5);
    float yScale = ratio_out;

    vec2 my = texture_coord * vec2(1.0, yScale);

    vec2 del = yCoord - my;

    float len = del.x * del.x + del.y * del.y;
    if ((!(my.y > 1.0) && (len < 0.25)) || (my.y > yCoord.y && my.y < rdistOut))
    {
        gl_FragColor = vec4(o_color, 1.0);
    }
    else
    {
        gl_FragColor.a = 0.0;
    }
}

What it does is draw capsules on the screen. Basically these capsules connect where the particle was to where it is.

On the cpu I compute the vertices of this with some linear algebra, but in screen space terms the rectangle we will be shading has a maximum width of W and a minimum height of W. If the capsule is moving faster then W/2 then the capsules geometry will start to get longer to accommodate.

Here are some diagrams of what is going on:

This is an image of a single capsule. Because the distance between the current and last position is not large enough so the screen-space geometry is a square. You can see that to compensate that lighter area is shaded in as well to connect the two points.

Here are some more examples of what could happen

The thing is these capsules are all individual, sometimes every capsule will be elongated, sometimes it will only be a small percent. This shader needs to be as steady fps as possible.

Now here are some tips that are useful and might help make things more efficent:

The dot product is NEVER needed past the point where the circle part is done.
If the scaleFactor is greater then 1 then the dot product is never needed past the center of the circle.

This is shown in this diagram.  In this case none of the grey area needs to preform the dot product.

Explanation of variables:

o_color: is the color that that particle is

texture_coord: these are the exact same as if you were going to texture this, they go from 0-1

ratio_out: height / width. Will never be less then 1

Inside the main function to get the coordinates in screen scale we simply multiply the y texture coordinate by the scale factor. That makes the center of the circle part a 0.5, 0.5.

rdistOut is the tiny area that the particle sometimes has when it is a square but still moving. Things should be shaded from 0.5-rdistOut.

Could you please help me remove un-needed dot product computations and comparisons as well as get rid of branching?

Please include an explanation of how you got the GPU to skip dot product calculations or why you did what you did. Next time I hope to be able to do this myself.

Rather than trying to speed up your shader by fiddling with the code, can I suggest you make use of the GPU's inherent ability to rapidly fill polygons? This will allow it to eliminate the vast bulk of the cases where there is no need to even make a decision.

What I suggest you try is something like the following: Create, say, a heptagon (i.e. the blue dashed outline (note: it shares some of the boundary with the green but I just noticed this is not particularly obvious in the diagram)) that starts from the original location and extends to the new position but lies inside the extremes of the new location's circle.
The shader for this is then utterly trivial.

Then there can be one or two cases depending on how much you want to optimise. If the tail extends outside of the end position circle, you draw the green-dashed nonagon with your decision-based shader. This contains vastly fewer pixels.
If the tail doesn't extend outside, then you'll need to mirror this polygon to get the other side of the circle. Alternatively, rather than make the decision, you could always mirror it and let the Z-test eliminate unneeded fills.

Your examples also have the have the capsules shown as opaque. If this is the case, you can also make gains with, say, TBDR-systems (a blatant PowerVR plug :-) ) or those with early-Z, provided you supply all your opaque "tail" portions first and then follow with all parts requiring decisions.

The shapes of course were just examples. You may find it's better to use more segments in your outlines in order to reduce the regions requiring the complex shader.


CUDA/OpenCL vs OpenGL compute shader for Path tracer So I want to implement a path tracer and I am confused between GPGPU computing or using OpenGL's compute shaders. I've already implemented a raytracer using GL's compute shaders.

What are the main differences between CUDA/OpenCL and CS? And is it possible to see a significant speed up if I prefer one over the other? Some main differences I already know are that using CS will make it vendor independent so I could run it on PCs having Nvidia/Amd GPUs alike. CUDA supports recursion while openCL/opengl CS don't.

What are other significant functionalities that would make a person prefer opencl/cuda over CS?

To first clear your confusion around the terms:

GPGPU stands for General Purpose computing on GPUs

CUDA is the specific NVIDIA API to perform GPGPU only on their hardware

OpenGL is a graphics specific API and is vendor independent
OpenCL is a parallel programming compute API and is vendor independent

Compute Shaders are a way to perform general purpose computations within
a rendering system but without being part of a fixed rendering architecture.
They exist for OpenGL (https://www.khronos.org/opengl/wiki/Compute_Shader)
but also for other graphics APIs like DirectX (DirectCompute) etc.

OpenCL is not only built for GPUs and can also be used to write code
for example to run massive parallel computations on supercomputers.

NVIDIA develops their own ray tracer based on CUDA and GPU computing.
The project is called Optix (https://developer.nvidia.com/optix)

To your performance question. It depends, as always on your setup and use case.
Some people did already the effort to measure differences.
One paper for example is this:
https://arxiv.org/ftp/arxiv/papers/1005/1005.2581.pdf

Another one is this:
https://dl.acm.org/citation.cfm?id=2066955

Or one thesis about it:
https://liu.diva-portal.org/smash/get/diva2:909410/FULLTEXT01.pdf

In their comparisons did CUDA often show good results. But when they for example used image processing algorithms that can be parallized very efficiently from the GPU then the comparison is nearly worthless compared what you might do in your path tracing code.

If you do not use any "classic" rendering, which you wanna combine your path traced results, I see not why you should use compute shaders for a pure path tracer. If platform independence is important OpenCL might be interesting for you since you then can take the code and run it on a complete other hardware for comparison. If you will stick to NVIDIA GPUs anyways you will likely get better performance using directly CUDA.


Compression of user-drawn "whiteboard" image I need to store user-drawn images from an online whiteboard. The amount of data gets huge quickly which is obviously not good. What I need is to store something drawn through a tablet for example in as little space as possible without too much of accuracy loss as for example letters should be distiniguishable. Bonus points for a solution with the ability to determine when what was drawn.

Don't store the rasterized image instead store the drawn strokes as vector graphics. Those should compress pretty well.

Adding timestamps to each stroke should then be very simple.

You can caching the rasterized image during execution to avoid needing to rerender every stroke every frame.


glutSolid* source code I am trying to build a dedocohedron in OpenGL, and I know that there is a built-in function in GLUT for the purpose. I was wondering if there is a way to either see the sourcecode for those functions, or to extract the generated data for vertices, colors, and indeces and print them?

The open source implementation FreeGLUT has these functions.
The geometry code can be found in freeglut/freeglut/src/fg_geometry.c.
For the dodecahedron, these are the values:
#define DODECAHEDRON_NUM_VERT           20
#define DODECAHEDRON_NUM_FACES          12
#define DODECAHEDRON_NUM_EDGE_PER_FACE  5
#define DODECAHEDRON_VERT_PER_OBJ       (DODECAHEDRON_NUM_FACES*DODECAHEDRON_NUM_EDGE_PER_FACE)
#define DODECAHEDRON_VERT_ELEM_PER_OBJ  (DODECAHEDRON_VERT_PER_OBJ*3)
#define DODECAHEDRON_VERT_PER_OBJ_TRI   (DODECAHEDRON_VERT_PER_OBJ+DODECAHEDRON_NUM_FACES*4)    /* 4 extra edges per face when drawing pentagons as triangles */
/* Vertex Coordinates */
static GLfloat dodecahedron_v[DODECAHEDRON_NUM_VERT*3] =
{
               0.0f,  1.61803398875f,  0.61803398875f,
    -          1.0f,            1.0f,            1.0f,
    -0.61803398875f,            0.0f,  1.61803398875f,
     0.61803398875f,            0.0f,  1.61803398875f,
               1.0f,            1.0f,            1.0f,
               0.0f,  1.61803398875f, -0.61803398875f,
               1.0f,            1.0f, -          1.0f,
     0.61803398875f,            0.0f, -1.61803398875f,
    -0.61803398875f,            0.0f, -1.61803398875f,
    -          1.0f,            1.0f, -          1.0f,
               0.0f, -1.61803398875f,  0.61803398875f,
               1.0f, -          1.0f,            1.0f,
    -          1.0f, -          1.0f,            1.0f,
               0.0f, -1.61803398875f, -0.61803398875f,
    -          1.0f, -          1.0f, -          1.0f,
               1.0f, -          1.0f, -          1.0f,
     1.61803398875f, -0.61803398875f,            0.0f,
     1.61803398875f,  0.61803398875f,            0.0f,
    -1.61803398875f,  0.61803398875f,            0.0f,
    -1.61803398875f, -0.61803398875f,            0.0f
};
/* Normal Vectors */
static GLfloat dodecahedron_n[DODECAHEDRON_NUM_FACES*3] =
{
                0.0f,  0.525731112119f,  0.850650808354f,
                0.0f,  0.525731112119f, -0.850650808354f,
                0.0f, -0.525731112119f,  0.850650808354f,
                0.0f, -0.525731112119f, -0.850650808354f,

     0.850650808354f,             0.0f,  0.525731112119f,
    -0.850650808354f,             0.0f,  0.525731112119f,
     0.850650808354f,             0.0f, -0.525731112119f,
    -0.850650808354f,             0.0f, -0.525731112119f,

     0.525731112119f,  0.850650808354f,             0.0f,
     0.525731112119f, -0.850650808354f,             0.0f,
    -0.525731112119f,  0.850650808354f,             0.0f,
    -0.525731112119f, -0.850650808354f,             0.0f,
};

/* Vertex indices */
static GLubyte dodecahedron_vi[DODECAHEDRON_VERT_PER_OBJ] =
{
     0,  1,  2,  3,  4,
     5,  6,  7,  8,  9,
    10, 11,  3,  2, 12,
    13, 14,  8,  7, 15,

     3, 11, 16, 17,  4,
     2,  1, 18, 19, 12,
     7,  6, 17, 16, 15,
     8, 14, 19, 18,  9,

    17,  6,  5,  0,  4,
    16, 11, 10, 13, 15,
    18,  1,  0,  5,  9,
    19, 14, 13, 10, 12
};

As for the second part of your question, tools such as renderdoc allow you to export meshes which are used in a render. To do this, you must first take a capture, find the corresponding draw call in event browser, open the Mesh Viewer and click on the export icon. You can do this with almost any mesh input to a vertex shader drawn in a supported API.


Is there a way to interpolate color across the line with only integer calculation ?` Here is my function that does interpolation

static public PixelData InterpolateColour(Point start, Point end ,float curLength, float totalLength){

    float endPercent = curLength / totalLength;             // Calculates the percent of colour at the ending point of the line to be used in current pixel
    float startPercent = 1 - (curLength / totalLength);     // Calculates the percent of colour at the starting point of the line to be used in current pixel

    float startHueInRadians[] = PixelData.PixelDataToHsbInRadians(start.getColour());   // Converting the ending point colour into a hsb colour model
    float endHueInRadians[] = PixelData.PixelDataToHsbInRadians(end.getColour());       // Converting the starting point colour into a hsb colour model

    startHueInRadians[0] = (startHueInRadians[0] == 0) ?                                            // if the start colour is red then then choosing the value to be zero or 2*pi
                           ((endHueInRadians[0] >= (Math.PI * 4)/3) ?  (float)(Math.PI * 2) : 0)    // if the end colour is red then then choosing the value to be zero or 2*pi
                           : startHueInRadians[0];

    endHueInRadians[0] = (endHueInRadians[0] == 0) ?
                         ((startHueInRadians[0] >= (Math.PI * 4)/3) ?  (float)(Math.PI * 2) : 0)
                         : endHueInRadians[0];

    float a = start.getColour().getAInInt() * endPercent + end.getColour().getAInInt() * startPercent;  //Interpolating alpha, which is not working :(
    float h = endHueInRadians[0] * endPercent + startHueInRadians[0] * startPercent;                    //Interpolating hue.
    float s = endHueInRadians[1] * endPercent + startHueInRadians[1] * startPercent;                    //Interpolating staturation, which is useless currently. :(
    float b = endHueInRadians[2] * endPercent + startHueInRadians[2] * startPercent;                    //Interpolating value, which is useless currently. :(

    float f[] = {h,s,b};

    PixelData p = PixelData.HsbinRadiansToPixelData(f);     // Converting interpolated hsb back to rgb

    p.setAInInt((int)a);    //setting the alpha value

    return p;               // returing the intepolated value
}

It is working properly but it is giant bulky code that runs very slowly because it has all kind of Ternary operator, floating point arithmetic and also it runs once per pixel.

I am currently refactoring my code to be small and less hacky.

What are some interpolation algorithm that are used in computer graphics ? because i came up with this code and it is just bad.

Yes, it is possible to use only integer calculations. I will describe how, but bear in mind that the difference in speed between integer arithmetic and floating point arithmetic is not as great as it was historically. If you want your code to run faster, it is best to profile and identify which parts of the code are taking up most time, before considering whether to convert a particular aspect to use integers.

A straight line through colour space

Looking at the code, it appears you first convert from RGB to HSB then create a linear gradient, and convert it back to RGB.

A linear gradient is a straight line through the colour space, so you can use an integer based straight line algorithm to generate it without the need for floating point arithmetic. For example, the integer based version of Bresenham's algorithm.

H, S, and B are being treated independently, so you can calculate each one separately. For example, the H component can be calculated by treating the line as a straight line from (x0, H0) to (x1, H1), instead of the usual (x0, y0) to (x1, y1).

If the range of y values along the line is greater than the range of x values along the line then Bresenham's algorithm will step along integer y values instead of x values. The algorithm can still be applied in exactly the same way, just using (y0, H0) to (y1, H1).

You can also combine all 4 calculations into a single loop, keeping track of a parameter each for determining y value, H value, S value and B value.

The only difference from plotting a line on the screen (using x and y) is that not every value of H, S, and B needs to be taken. If the x range is greater than the y range, the algorithm steps through each of the x values checking whether to increment y. If the y range is greater then it instead steps through the values of y checking whether to increment x. However, if the range of H is greater than the range of x, it makes no sense to step through H values, as many of them will correspond to the same x value, wasting calculations. If the range of H is smaller than the range of x then the algorithm can proceed as usual. If the range of H is often going to be larger than the range of x then it is worth considering a different approach. Similarly for S and B.

Fitting the colour space to your line

One way to eliminate this problem is to tailor your colour space to the x (or y if greater) range required for this specific line. Since your input colours and final output colours are in RGB, you are not constrained to a particular range for the HSB colours. You are free to convert from 256 levels of R, G, and B to 1000 levels of H, S, and B (or 1000000, or whatever you choose). If you choose to use a colour space where the range of H values used is the same integer as the range of x values used, then you won't even need to use a line drawing algorithm for the colours. Simply store the required values for converting back to RGB, and each of the H, S, and B values will be x plus a constant.

This will require modifying your conversion function, but this will probably be necessary anyway, as it currently uses radians, which are not going to help with keeping things integer.

More accurate colour spaces

Note that using a colour space like HSB will not give gradients which look as even as if using a more perceptually uniform color space such as Lab. However, since such spaces require calculating roots and you are looking to keep to integer arithmetic as much as possible, I'm assuming you are happy with HSB. If you did want to use something like Lab, you could still potentially keep to integers by using look up tables.

What inputs does this need to work for?

You also mention having to make decisions in the code which also takes up time. I would question how much difference this makes since the decisions are outside the loops and therefore only performed once. If you really want to eliminate them, ask yourself which inputs the code needs to work for.

Currently the code checks the start and end values of the hue and decides whether to change the hue clockwise or anticlockwise. If you eliminate this check the code will still work, but it may not always give the colour direction the user intended. Deciding this automatically also may not give the direction the user intended (since different users have different expectations, and the requirement may change from one use to the next). You could simply take the direction as an additional parameter, which would avoid having to make the decision.

Code review

Meaningful variable names

The use of "hue" in the naming of variables that store all three of hue, saturation and brightness makes the code less intuitive to read, potentially causing the reader to wonder whether saturation and brightness have been overlooked. It also makes the code difficult to reason about, since startHueInRadians[0] is a hue, while startHueInRadians[1] is a saturation, which is not only not a hue, but also presumably not in radians.

In addition to making the code more accessible to others, you may also find it easier to work with it yourself if the naming is consistent, especially if you look back at the code after a few months.

A potential bug

If one end of the colour gradient is at hue zero (red), the code checks whether the other end is greater than or equal to two thirds of the way around the hue circle. This corresponds to blue. If it is less than blue, then the gradient proceeds from zero upwards (anticlockwise), otherwise it proceeds from 2 pi downwards (clockwise). This means that a gradient from red to a colour with hue just above blue will go from red to magenta to blue. A gradient from red to a colour with hue just below blue will go from red to yellow to green to cyan to blue, which may not be what is expected. Intuitively, I would expect the cut off to be half way around the circle rather than two thirds, so that the gradient always includes the fewest colours (taking the shorter, more direct route).

If you did intend to have this asymmetry, note that choosing a similar red with hue just above or just below zero will override this behaviour, forcing the long route or the short route respectively.

If you want the gradient to consistently take the shorter route, then the code will need to check which route to take for all values, not just zero. For example, currently choosing the start hue just above zero and the finish hue just below zero will give start and finish colours that both look red, but the gradient will pass through red, yellow, green, cyan, blue, magenta and back to red, rather than just fading subtly from one hue of red to another.


Why is it twice as expensive to make a noise function that can be tiled? I've seen in several places that making Perlin noise loop seamlessly requires calculating it twice in slightly different ways, and summing the two results.

This Perlin noise math FAQ gives a formula:

$$F_{loop}(x, y, z) = \frac{ (t - z) \cdot F(x, y, z) + z \cdot F(x, y, z - t) }{ t}$$

to make a noise function $F$ loop in the $z$ direction. It also mentions that extending this, to loop in 2 dimensions would take 4 evaluations of $F$ and to loop in 3 dimensions would take 8 evaluations of $F$.

I understand that this gives a seamless join between tiles that is not only continuous but continuously differentiable, but I intuitively expect that to be the case if the noise function is simply evaluated once with grid points reduced modulo the required tile size. If the noise function is only ever based on the immediately surrounding grid points (4 for 2D noise, 8 for 3D noise) then surely just using the leftmost grid points when the point to calculate gets past the right hand edge of the tile will give the same quality of noise as between any other grid points?

Since I've seen this multiple calculation approach in several places I assume it must have some advantage, but I'm struggling to see the disadvantage with simply wrapping the grid points back to the start when they get too big. What am I missing?

It's unfortunate that people commonly recommend this.  Blending between two (or four, etc.) translated copies of a noise function in that way is a pretty bad idea. Not only is it expensive, it doesn't even produce correct-looking results!

On the left is some Perlin noise. On the right is two instances of Perlin noise, stacked and blended left-to-right.

The difference is kind of subtle, but you can see that the second image has lower contrast in a vertical column running down the middle.  That's where it's a 50% blend between two different instances of the noise function. Such a blend doesn't look like the original noise function: it just looks like a muddy mess.

OK, so it's not quite that bad just looking at the raw noise...but if you then do any nonlinear transformations on the image, the nonuniform contrast can cause issues. For instance, here are those images thresholded at 60%. (Think of generating islands in an ocean, for instance.)

Now you can plainly see how the image on the right has fewer, smaller white areas in the middle.

Like you mentioned, for grid-based noise like Perlin, a better way is to tile the pseudorandom gradients at the grid points. That's easy and cheap to do, and then you can apply the interpolation algorithm to the gradients as usual (much like bilinear interpolation of a tiling texture). This produces tiling noise without any weird artifacts, because it works with the underlying noise algorithm rather than over the top of it. You can use a similar strategy with Worley noise (cellular noise) by tiling the random feature points it uses as a basis.

With multiple octaves of noise it's not always so easy, though. If the relative scale between the octaves (aka "lacunarity") isn't an integer or simple rational number, then you may not be able to find a convenient tiling point where all the octaves' grids match up. You could tile each octave independently, but the overall noise would still not be tilable in that case.


Downscaling by skipping rows and columns – any example images? I'm about to implement downscaling on a microcontroller. Source resolution is e.g. 720x576 (the 576i PAL resolution, other one possible: 480i, NTSC resolution), destination is 32x16. The goal is to have RGB LED matrix (of size 32x16) flowing with various colors, shapes, shadows, etc. – a half-abstract vs. half-meaningful effect, driven by a prepared PAL or NTSC video.

I will only be able to select every 22th pixel (720 / 36) and every 36th line (576 / 16) – the microcontroller will not be able to compute averages of groups of pixels etc. This way I will obtain 32x16 downscaled image to output to the LED matrix.

I cannot find any examples of how such downscaling will behave. Tried searching for "decimation" but no luck. Which program can I use to test this downscaling? Can I quickly code this somehow – if I would to code this, I would use libSDL1, generate set of files, but this is difficult (e.g. image loading). Maybe some snippet-solution is possible? Should I care about Nyquist frequency?

A basic nearest neighbour scale function would work like so. Here I also use 24:8 fixed point arithmetic to make it more micro friendly.

const int originalWidth = 320, originalHeight = 240;
int targetWidth = 50, targetHeight = 50;

char *pSource = sourceBuffer;
char *pDest = destBuffer;    // you can use RGB instead of char

int yStep = (originalWidth << 8) / targetWidth;
int xStep = (originalHeight << 8) / targetHeight;

for(int y = 0, sampleY = 0; y < targetHeight; y++, sampleY += yStep) {
    // Point to start of scaled Y source line
    char *pSourceLine = pSource + ((sampleY >> 8) * originalWidth);

    // Output the line, scaling only in X (we already know scaled-Y)
    for(int x = 0, sampleX = 0; x < targetWidth; x++, sampleX += xStep)
        *pDest++ = pSourceLine[sampleX >> 8];
}

This will also work to scale image up as well as down, it should work on any input vs output size.


Efficiently drawing multiple instances of 3D objects I was able to write the following working code to draw any number of "sprites" (defined as 2 tringles, 6 vertices)

    Matrix.setIdentityM(mModelMatrix, 0);

    Matrix.multiplyMM(mMVPMatrix, 0, getCamera().mViewMatrix, 0, mModelMatrix, 0);
    GLES20.glUniformMatrix4fv(mShader.getMVMatrixHandle(), 1, false, mMVPMatrix, 0);

    Matrix.multiplyMM(mMVPMatrix, 0, getCamera().projMatrix, 0, mMVPMatrix, 0);
    GLES20.glUniformMatrix4fv(mShader.getMVPMatrixHandle(), 1, false, mMVPMatrix, 0);

    mPositions.position(0);
    mColors.position(0);
    mNormals.position(0);
    mTextureCoordinates.position(0);

    GLES20.glVertexAttribPointer(mShader.getPositionHandle(), mPositionDataSize, GLES20.GL_FLOAT, false, 0, mPositions);
    GLES20.glVertexAttribPointer(mShader.getColorHandle(), mColorDataSize, GLES20.GL_FLOAT, false, 0, mColors);
    GLES20.glVertexAttribPointer(mShader.getNormalHandle(), mNormalDataSize, GLES20.GL_FLOAT, false, 0, mNormals);
    GLES20.glVertexAttribPointer(mShader.getTextureCoordinateHandle(), mTextureCoordinateDataSize, GLES20.GL_FLOAT, false, 0, mTextureCoordinates);

    GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, 6*numSprites);

Using a sprite atlas this code displays many different sprite textures in one fell swoop, which is very fast.

However, I would also like to draw multiple copies of a 3D .obj file. Now I've loaded in the necessary FloatBuffers of positions, normals, texture coordinates, etc., so I can easily add these buffers to the mPositions, mTextureCoordinates buffers above, etc., no problem.

But these instances of the .obj model I'm trying to display need to be in different x/y locations.

Is there an efficient way this can be achieved in OpenGL? Maybe one of the following:

I could displace the .obj files _positions buffer before adding to the  global mPositions buffer to draw, but this seems rather inefficient.
One solution is to run a separate instance of the above code for each .obj 3D object, including an OpenGL translation for each instance. But I've heard this is costly if you have many 3D objects. (This is the reason I use a sprite atlas for 2D sprites in the above code)
"Another" approach that seems to work is to add the following example code, where _positions etc are the buffers for the `.obj. file, and I draw 5 of the same object file in a row using a translation. It's very similar to (2) but without having to repeat the OpenGL initialisation code every time.

for (int c1 = 0; c1 < 5; c1++) {

    _positions.position(0);
    _normals.position(0);
    _textureCoordinates.position(0);

    Matrix.setIdentityM(mModelMatrix, 0);
    Matrix.translateM(mModelMatrix, 0, cx + dx * c1 * 4, cy, 0);
    Matrix.scaleM(mModelMatrix, 0, 0.1f, 0.1f, 0.03f);
    Matrix.rotateM(mModelMatrix, 0, -90, 1, 0, 0);

    Matrix.multiplyMM(mMVPMatrix, 0, getCamera().mViewMatrix, 0, mModelMatrix, 0);

    GLES20.glUniformMatrix4fv(mShader.getMVMatrixHandle(), 1, false, mMVPMatrix, 0);
    Matrix.multiplyMM(mMVPMatrix, 0, getCamera().projMatrix, 0, mMVPMatrix, 0);
    GLES20.glUniformMatrix4fv(mShader.getMVPMatrixHandle(), 1, false, mMVPMatrix, 0);
    GLES20.glDrawArrays(GLES20.GL_TRIANGLES, 0, numFaces);
}

But I'm not sure if this isn't just as inefficient as the (2) ?

Maybe there's no way round it? Perhaps (2)/(3) is the only way and I'm just being over cautious because of my "bad" experience without using a sprite atlas for 2D sprites? After all, I was originally drawing 1000's of 2D sprites using method (1) originally, but I will only be drawing around 10-20 3D objects. In the 2D case, switching to a sprite atlas greatly speeded things up !

(I'm interested in OpenGL ES 2.0 primarily)

What you are looking for here is instanced rendering. Instanced rendering coupled with attribute divisors is a very powerful tool to render multiple copies of the same object using only one drawing call.

Using instanced rendering you only have to specify the geometry of an object once on the GPU. Then, you could pass along an additional buffer of matrices, one matrix for each instance of the object you are rendering. Using attribute divisors you can tell the GPU that you will only need one single matrix per a number of vertices. Attribute divisors tell the GPU when and how to advance an index in a attribute buffer. This allows you to only define a single matrix per instance instead of passing along a single matrix for every vertex in a single model.

Here, is an excellent tutorial on using instanced rendering and attribute divisors.


glOrtho() in fixed function pipeline - Unable to see change in z-axis void display(void) {
    glClear(GL_COLOR_BUFFER_BIT);
    glColor3f(1.0, 1.0, 1.0);
    glBegin(GL_LINES);
    glVertex3d(0.75, 0.25, 0.0);
    glVertex3d(0.75, 0.25, -0.75);
    glEnd();
    glFlush();
}

void init(void) {
    glClearColor(0.0, 0.0, 0.0, 0.0);
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    glOrtho(0.0, 1.0, 0.0, 1.0, -1.0, 1.0);
}

I should be able to see line from (0.75, 0.25, 0.0)  to (0.75, 0.25, -0.75), but I can't see anything on the screen.

However, I'm able to see the line using snippet below

glVertex3d(0.35, 0.25, 0.0);
glVertex3d(0.75, 0.25, -0.75);

Can anyone explain this behavior?

This projection is a 2d projection, with the Z axis pointing into your screen. When both vertices with the same X&Y are projected to screen space they will occupy the same screen coordinate so your line will either be 1 pixel or invisible.

If you are expecting a difference in screen position from those 2 coordinates then you need a perspective projection.


What is better to use for real-time computing Mesh - Plane intersection points, GPU or CPU? I want to write plugin (library) for Unity3d (it doesn't matter which framework I will choose for this, question is ), for cutting arbitrary mesh with plane (for simplicity it will be plane for beginning).

There are following steps:

1) check every triangle whether it lies above, under plane or intersected by plane, assign all vertices to VER_ABOVE or VER_UNDER lists, recalculate triangles, put them in TRI_ABOVE or TRI_UNDER lists

2) Split intersected triangles into triangle and quadrilateral, triangulate last one and put them in corresponding lists (TRI_ABOVE or TRI_UNDER)

3) Triangulate slice surface, assign it to TRI_ABOVE and TRI_UNDER

And now the question:

Where it is better to calculate intersections, triangulation (for example I will use Constrained Delaunay Triangulation or Sweep Line Non-Convex Polygonal Triangulation cutting into monotone polygonals) and other stuff: on CPU or on GPU (using Shaders), if someone could explain me general pipeline from loading cashing mesh to memory and so on on GPU and on CPU, so I can better understand the most time consuming actions and can optimize some stuff.

I will also be thankful for sharing link on code examples calculating similar stuff.

doing the calculation to decide whether a point is on one side of a plane or the other is very simple (a single dot product). Doing that 3 times and having a special case when they don't match to split the triangle is pretty fast. It's also a parallel problem. The hardest part is reserving the space for the output.

Preparing the data to start computing this on the gpu is already enough overhead that it's not worth doing it on the gpu unless the mesh is very big or you are splitting it multiple times (due to the plane moving for example). Or the data is already in good enough condition to be sent over to the gpu directly through DMA without the cpu needing to touch it at all.


How does the GPU know whether to magnify/minify? Let me first state what I think I know and please correct me if I'm too inaccurate or plainly wrong.
When you use a function like texture, you're requesting a texel value given a pair (u,v) of texture parameters. However, depending on the pixel size / texel size ratio, there's a filtering and resampling prior to returning the texel value, which you can actually configure in the corresponding API (OpenGL, Vulkan, etc).
As I understand it, this filtering and resampling occurs because either:
a) Many texels affect a pixel. This happens because the texture resolution is too high for the area on screen to be covered (i.e. the pixel resolution). So in order to avoid artifacts, the texture needs to be downsampled, which consists of applying a minification filter and resampling at a lower rate.
b) The same texel can affect many pixels. The texture resolution is lower than the pixel resolution. In this case, a magnification filter is used and the texture is upsampled.
Assuming the above explanations are correct, my question is: how does the GPU know which of these two scenarios are happening.
I reckon that it does by projecting the pixel area in texture space, as it is done for selecting the mipmap level. But even if that's the answer, how is this projection achieved? I'm failing to visualize it.

Minification vs magnification is an extension of mipmap selection. When you calculate the desired mip level, it can come out to be less than 0 (or in general, less than the sampler's configured minimum LOD). In that case you're doing magnification of the lowest mip level. If the desired level comes out to a value greater than 0 then you're doing minification.
Mipmap selection works by calculating differences of the UV coordinates between adjacent pixels on screen (see shader derivative functions). Since the GPU is always shading pixels in 2×2 quads, it can look at the set of UVs input to the sampling operation across all those pixels. Those derivative vectors establish what the pixel footprint in texel space is, and their lengths are used to calculate which mip level to sample.
The details of how these calculations work exactly can be found in the OpenGL spec, section 8.14 "Texture Minification"


Does angular diameter to FoV ratio equal object's image to total image ratio? If we have an object at certain distance D in front of a camera. The object has a horizontal diameter d, giving us an angular diameter theta (2*arctan(d/2D)) of 50 degrees (converted from radians). And a camera of 100 FoV, horizontally.
Given those figures, does that mean that the horizontal image size of the object will be exactly 50 % of the image width?

This is an interesting question, as intuitively we would think that the image size of the object will be 50% of the image width. However, upon some playing around with triangles, I found this to not be the case.
For example, take a look at the diagram below. Here we have two overlapping triangles, one with angular diameter of 50 degrees (representing the object's angular diameter), and one with angular diameter 100 degrees (representing the camera's FoV). As we can see, the object has a width of 1 unit as seen from the camera, however the image width is around 3 units - therefore in this case the horizontal image size of the object will be around 33% of the image width.

If you were to play around with different values for D and d, you would find that the angular diameter ratios would vary. There would be a way to calculate this ratio directly with some trigonometry, however I do not know of how much use that would be to you.
Hope this helps in some way :)


How to visualize density data as volume using OpenGL? I have a CT scan of a portion of a human body, with each point describing density. The data file provides X, Y, Z, and density values.  I want to know how to visualize this data, using OpenGL or a library based on OpenGL.

What you're looking for is "volumetric rendering", here is one resource with code that takes data in the form of images of the layers:
Getting started with Volume Rendering using OpenGL
A quick hack would be to use a grid of particles/"billboards", and make them transparent based on the density.


Computing sRGB color from SPDs I'm trying to write my own spectral path tracer and a bit stuck with converting computed SPDs to LDR RGB values. What I'm doing right now is convert sampled SPDs (in W/m^-1) to XYZ using CIE 1931 color matching functions:

Vec3 xyz(0, 0, 0);
float norm = 0;

for (int i = 0; i < SPD_SAMPLE_COUNT; ++i) {
    const float wavelength =
        lerp(MIN_WAVELENGTH, MAX_WAVELENGTH, i / SPD_SAMPLE_COUNT);
    const Vec3 cieMatchingFunctionValue = cie1931(wavelength);
    xyz += cieMatchingFunctionValue * spd[i];
    norm += cieMatchingFunctionValue.y;
}

xyz /= norm;

Here implementation of cie1931 is an anylitic approximation from Wyman et al.

Then XYZ to xyY (i.e. chromaticity and luminance), normalise luminance to be in [0, 1] range across the whole image (for now by dividing my max luminance just to keep it simple):

float maxLuminance = 0;

for (auto &xyz : image) {
    maxLuminance = std::max(maxLuminance, xyz.y);
}

for (auto &xyz : image) {
    Vec2 chromaticity = Vec2(xyz.x, xyz.y) / (xyz.x + xyz.y + xyz.z);
    float luminance = xyz.y / maxLuminance;

    // then to "scaled" XYZ and RGB, see below
}

Also I've tried Reinhard tone mapping operator:

float logSum = 0;

for (auto &xyz : image) {
    logSum += std::log(xyz.y);
}

float logAvgLuminance = std::exp(logSum / image.size());

for (auto &xyz : image) {
    Vec2 chromaticity = Vec2(xyz.x, xyz.y) / (xyz.x + xyz.y + xyz.z);
    const float a = 0.18; // constant from Reinhard's paper.
    float luminance = a * xyz.y / logAvgLuminance;
    luminance /= luminance + 1;

    // then to "scaled" XYZ and RGB, see below
}

Then back to XYZ, than to linear RGB (wia this transform):

for (auto &xyz : image) {
    // XYZ to chromaticity and "scaled luminance", see above

    Vec3 scaledXyz(
        chromaticity.x * luminance / chromaticity.y,
        luminance,
        (1 - chromaticity.x - chromaticity.y) * luminance / chromaticity.y
    );

    float r = dot(Vec3( 3.2406, -1.5372, -0.4986), scaledXyz);
    float g = dot(Vec3(-0.9689,  1.8758,  0.0415), scaledXyz);
    float b = dot(Vec3( 0.0557, -0.2040,  1.0570), scaledXyz);

    // Output RGB, i.e. std::cout << Vec3(r, g, b);
}

After that I still get RGB values outside of [0..1]. What am I doing wrong? It seems that "max" luminance should depend on chromaticities of the color. Is there an algorithm to account for that?

Getting sRGB values outside [0, 1] is expected and normal when using spectral rendering.

The sRGB gamut only covers a triangle in the middle of the CIE chromaticity space:

(diagram from Wikipedia)

The big "horseshoe" shape is the set of all physically possible chromaticities (CIE xy coordinates) from any possible spectrum. So your xy coordinates should be expected to land somewhere in the horseshoe...but only the ones that land inside the triangle are representable as sRGB! The others are "too saturated" for sRGB, and will have to be clamped or remapped somehow. (Tone mapping per se doesn't fix this, as it's concerned with luminance rather than chromaticity.)

There are a variety of approaches to deal with out-of-gamut colors depending on how sophisticated you want to be:

You can of course simply clamp the sRGB values, but this may change the hue and/or the luminance of the color in the process.
To preserve both hue and luminance, you can edit the xy coordinates before calculating scaledXyz. If the initial xy coordinates are outside the sRGB triangle, find the line segment from the xy coordinates to the white point (e.g. D65); then find the intersection of that line segement with the sRGB triangle. That's the closest sRGB point that has the same luminance and hue as the original point, but lower saturation. (The vertices of the sRGB triangle can be obtained by converting RGB(1, 0, 0), RGB(0, 1, 0), and RGB(0, 0, 1) to CIE xy space.)
Alternatively, rather than clamping (which throws away any detail in the saturation outside the sRGB gamut), you could try doing a linear or non-linear remapping of saturation values in the image, similar to how you do for luminance. By "saturation" here I mean distance from the white point. The idea would be to squish the whole image toward the white point so as to bring everything within the gamut. But that may affect the apparent saturation of in-gamut colors in visually undesirable ways.


Creating high quality background images for a game which have small areas of moving detail I am in the process of making a 'Game' using the Unity game engine which features pre-rendered 3d background images. Currently, we are using JPegs but I thought it would be nice to get some movement in them. I remembered Final Fantasy VII having really neat pre-rendered images with movement in them and researched it and they still look great. the images on this site are gifs but I'm not sure what compression would have been used back then, very possibly gif. I seem to recollect that animated gifs only update the area of the image which actually changes which helps with compression. My question is: Is there a modern equivalent to animated gifs that would allow me to create high-quality background images with small areas of movement in them?

The "modern equivalent" of animated GIFs is... MPEG. Or really, any decent movie compression format. These techniques are designed to spend fewer bits on static areas and more bits on moving ones. And better still, they're designed to be dynamic, allowing motion in any are at any time, rather than only in fixed areas.
So just make a short, looping movie of the "background image" you're trying to build. You may want to get into the details of the compression system you're working with, so that you can give lots of bits to the initial frame and fewer bits to moving areas.


What are affine covariant regions I am trying to understand this seminal paper. I read this post on what affine transformations and my understanding is that affine transformations are transformations that preserve parallelism of lines such as rotation, translation, etc, but NOT distortions of partials of the image, and other related transformations that cause previously parallel lines to no longer be parallel.

I also understand that covariant regions are the same region being detected from two different view points.

However, in the paper, they use the term "affine covariant regions" and state that they can be "exploited to cope with geometric and photometric deformations" (5th paragraph if introduction). This sentence throws off my understanding because I don't quite understand what it means when both terms are combined (affine - covariant). It's probably a subtle difference. Can someone please clarify?

Actually what it meant by Affine Covariant regions is that covariant regions in two images which are related by some affine transformation. So the regions found in one image and those are exactly same regions in other image which have been transformed through affine transformation.


How does vsync affect fps exactly when not at full vsync fps? I know that if you turn of vsync, it synchronizes rendering with the vertical redraw cycle to prevent tearing, and that doing so caps your rendering rate (FPS) at the monitor refresh rate, which is commonly 60hz / 60 fps, although other rates exist as well.

However, when you are not running at a full 60fps how does vsync affect your frame rate?  I've heard people say that you will be limited to a multiple of 60fps (well, ~16ms to be precise), but from observation, fps can fluctuate wildly.

It depends on how a missed frame is handled by the driver.

One option is to just wait until the next vsync, causing a hitch of 32 ms and if the application is just at the limit of 16 ms can cause fluctuations.

The next option is to queue the frame for display next frame but don't wait on it. This will still cause a visual hitch but the application can then immediately start on the next frame instead of being forced to wait 16 ms. If the next frame is done faster then the late frame may even never be shown.

The final option is to not block and push the late frame to the display immediately which can cause tearing.


Does a gaussian blur based bloom/glow shader require gamma correction? Any other tips? Hello I have a simple application with some bright 2d mono-color shapes I am hoping to make appear emissive with a nice glow around them. To do that I am producing multiple gaussian blurs of varying kernels and resolutions and adding them all together to create a composite glow.

I have seen many guides mention this and even mention what sizes and kernels they are using to produce their bloom however they never mention the math they used to put them all together. Right now I am assuming it was just regular additive blending.

I was wondering what tips you might have to put this glow effect together effectively as far as if I should do anything other than just adding pixels together to combine the various maps. Should I do something with the weight on certain textures used in the composite?

I am just trying to composite everything in an optimal manner.

My main concern is whether or not I should gamma correct my glow map or if that is accounted with the weights from the gaussian sampling or source material. Should I gamma correct my blurred composite before adding it to the source image?

In short:

You should not gamma correct your glow map. In fact, you should do everything in a linear color space. At the very end, when doing any color grading (which is the very last step), you convert the final image to the right color space. The color space include that gamma correction.

In long:

Gamma correction is a step in the encoding of images. It is usually done so that we use more bits to store the shadows and less bits to store the highlights, since we see less detail in the highlights anyway. It is also part of some color spaces. These color spaces make sure the image looks the way it is supposed to. It is the final step, often part of color grading. Just before it is send out to the display, or saved to a file. This means that everything before should be in a linear color space.

Usually things would look like this:

Load textures and convert them to a linear space, so you do not double gamma correct..
Render the image.
Do the post effects.
Convert the image to the final color space with color grading. Probably sRGB Gamma. Converting to the color space includes the gamma correcting.

Color grading is not necessary, but optional.

When having everything at linear and doing the gamma correction at the very end, you will need a high precision for the color. Using floating point framebuffers is recommended. When using the default 8-bit framebuffer the whole way, you end up getting banding effects when gamma correcting.

This setup is not always done for games, to my knowledge, since most people will barely notice it or even care about it. In the film industry, VFX and animation, using the proper color space is more important.

When not doing it the 'right' way, the textures are not converted to linear and at the end, there is no gamma correction. Lights are supposed to have a quadratic falloff to be physically correct, but when gamma correcting it with a gamma of 2.0, it becomes a linear falloff. Lights can therefore get a linear falloff and look normal. As Noah Witherspoon pointed out, when blending and blurring you should use a linear color space, however this is not always done either.

The reasons for not doing it the 'right' way, is probably that it still looks fine. You do not need to do it this way, to end up with good graphics. It is always a matter of what the artist likes the most.


Drawing a plane super-imposing over another plane I wish to draw a plane (A building's floor in this case) over a canvas plane (assume the plane is smooth). The planes are drawn as QUADS.

The problem is that both plane are located at the same height. This causes a flickering between the images as I zoom in/out.

I wish for the floor texture / plane to have a higher priority over the base canvas plane. Is it possible? If so, how may I achieve it?

I've tried putting the floor plane just slightly above the canvas plane, but it doesn't sit right with me considering my use case allows the user to set any building height and he/she may enter a very low value to height, cause a lot of problems.

The problem that you encounter is called "z-fighting" and it occurs when two objects with the same depth (z value) are rendered.

In this example, we take two objects called ObjA and ObjB. They are the same, but have different colors. Due to the very small error that you get with fixed precision floating point numbers, ObjA may have a lower fragment depth than ObjB for a given pixel and thus ObjA's color is saved to that pixel, but ObjB could have a lower fragment depth for another given pixel and ObjB's color is then saved to that pixel. Which object will have a lower depth value is quite random and causes that flickering.

There are two ways to fix this. Which one works best for you, depends on the program.

Option #1

Disabling depth testing on the second surface that is being drawn. The first surface obviously draws well, and the problem comes when drawing that second surface. By disabling the depth test, OpenGL just draws every fragment to the framebuffer and you do not have that problem with "which fragment has the lowest depth".

If you only disable the depth test for the second surface and enable it again for the other surfaces (and render the second surface after the first surface), then it should work as the first surface still writes the depth values.

This approach starts to break, when the second surface either does not fully align with the first surface or the second surface is shaped in a way so that it overlaps itself. (When rendering a sphere, for each pixel you always render two fragments, the front side and the back side of the sphere. The back side would then not be culled away.)

Option #2

Adding a bias to the depth values of the first surface. The problem is that both surfaces have the same depth value for a given pixel, but if you offset the depth values for one of them, then you do not have that problem.

Just like gl_FragColor which allows you to set the color in the framebuffer, gl_FragDepth allows you to set the depth in the depth buffer. You could get the normal depth value from gl_FragCoord (as the documentation says) and add a small amount to it. Doing this, does not actually change anything of the mesh. The mesh does not move around and it does not go further into z-space. The shader simply writes a different depth value to the depth buffer. This depth buffer is only used for depth testing and can thus easily be changed to get the effect we want. This only needs to be done for the first surface and not the second surface or any other surface.

When the bias is too large, artifacts can visibly show up around intersections with the surface, but when the bias is small it should not be noticeable.


What are the type constants that the documentation for glGetProgramResourceiv() refers to? Quoting the Khronos OpenGL documentation pages for the glGetProgramResourceiv() function:

  For the property GL_TYPE, a single integer identifying the type of an
  active variable is written to params. The integer returned is one of
  the values found in table 2.16.

But, there is no "table 2.16" in that page, there is no "table 2.16" in the specification document (where the language also appears), and several search engines failed to return any other references to such a table.

So, where is it?   What are the full list of type codes that can be returned by this introspection function (which can presumably represent all GLSL types)?

(I'm using GL core 4.3)

For whatever reason, the text for that function documentation was copied from the GL_ARB_program_interface_query extension. Extensions in OpenGL are always defined relative to a specific version of the OpenGL specification. So if an extension references something in a specification, it's talking about that version of the spec.

However, the OpenGL specification was massively restructured between 4.2 and 4.3, with whole chapters moved around. This means that table references in 4.2 become completely wrong relative to 4.3 and above.

Program interface query was written against the 4.2 specification. Oddly enough, there is no table 2.16 in that spec; the correct number for that table is 2.13 (so it's likely a typo, since 3 is right below 6 on a number-pad). But even if the extension did mention the right table number, that table moved to a whole different chapter in 4.3+ versions. It's been renumbered and moved since then, so you'll have to look at a particular specification's definition of glGetProgramResourceiv to find it.

In general, if you find something incongruous in the documentation pages, you should look at the actual OpenGL specification to see what it says about the function's behavior. In this particular case, the docs in the OpenGL wiki for this function actually includes the table.


Passing parameters in Metal Compute Kernel using Swift 4 On the CPU side, I have a structure I want to pass to the compute kernel:

  private struct BoundingBoxParameters {
    var x: Float = 0
    var y: Float = 0
    var width: Float = 0
    var height: Float = 0
    var levelOfDetail: Float = 1.0
    var dummy: Float = 1.0  // Needed for success
  }

Before running the kernel, I pass in the data to the MTLComputeCommandEncoder:

Option 1 (directly):

commandEncoder!.setBytes(&params, length: MemoryLayout<BoundingBoxParameters>.size, index: 0)

Option 2 (indirectly via MTLBuffer):

boundingBoxBuffer.contents().copyBytes(from: &params, count: MemoryLayout<BoundingBoxParameters>.size)
commandEncoder!.setBuffer(boundingBoxBuffer, offset: 0, index: 0)

Either option works fine if the 'dummy' variable exists in the structure, but fails if the 'dummy' variable does not exists. The code fails in the call to:

commandEncoder!.dispatchThreadgroups(threadGroups, threadsPerThreadgroup: threadGroupCount)

With the error:

validateComputeFunctionArguments:820: failed assertion `Compute Function(resizeImage): argument params[0] from buffer(0) with offset(0) and length(20) has space for 20 bytes, but argument has a length(24).'

On the Metal Kernel Side, here's the relevant code snippets:

struct BoundingBoxParameters {
  float2 topLeft;
  float2 size;
  float levelOfDetail;
};

kernel void resizeImage(constant BoundingBoxParameters *params [[buffer(0)]],
                        texture2d<half, access::sample> sourceTexture [[texture(0)]],
                        texture2d<half, access::write> destTexture [[texture(1)]],
                        sampler samp [[sampler(0)]],
                        uint2 gridPosition [[thread_position_in_grid]]) {
  float2 destSize = float2(destTexture.get_width(0), destTexture.get_height(0));
  float2 sourceCoords = float2(gridPosition) / destSize;
  sourceCoords *= params->size;
  sourceCoords += params->topLeft;
  float lod = params->levelOfDetail;
  half4 color = sourceTexture.sample(samp, sourceCoords, level(lod));
  destTexture.write(color, gridPosition);
}

I also get a similar issue when trying to pass in a 3x3 matrix to the compute kernel. It complains that 36 bytes are provided but expecting 48.

Anyone got any ideas on the issue?

The float2 type in the Metal structure requires an alignment of 8 bytes. The stride of the struct then becomes 24 so every value is aligned properly if you use it in an array.  Your Swift structure only uses float which requires an alignment of 4 bytes, and an overall stride of 20.

This presentation by Apple goes into some detail (starting at 30:20 of the video). The Metal Shading Language
Specification  lists the alignment of vector and matrix data types in sections 2.2 and 2.3.

Similarly, float3x3 has a size of 48 and an alignment of 16 in Metal. But an array of 9 floats would only have a size of 36 and an alignment of 4. Using the simd module can provide you data types in Swift with the same size and alignment requirements as those used in Metal.

As a side note, you should be using MemoryLayout<>.stride instead of MemoryLayout<>.size when you call setBytes.  The following Swift structure has a stride of 24 but a size of only 20.

struct BoundingBoxParameters {
    var topLeft: float2 = float2(0)   // float2 is from "import simd"
    var size: float2 = float2(0)
    var levelOfDetail: Float = 1.0
}


What is "tiled" raytracing? I've seen mention of tiled raytracing, but haven't been able to find much on google about it.

Can anyone explain what it is and how it works?

Tiled raytracing is a way of speeding up rendering.  Instead of rendering pixels on a row-by-row basis down the entire image, you render them in 2D groups (say, 32x32 tiles).  Since adjacent rays tend to intersect the same objects, this improves the cache hit rate at the CPU level.

At a smaller scale, tiling at the 2x2 level lets you use SSE/AVX/etc to compute four rays with a single CPU instruction, further speeding things up.


Can I accelerate rendering an image from a ray cloud using a GPU? Start with a precalculated "ray cloud" - the starting point and direction of a large number of rays, most of which will not contribute to the image. The image plane's position and orientation are then specified, and the task is to search the ray cloud for rays that intersect the image plane without occlusion by other objects, and output the resulting pixel values. No ray reflections would be required, as all of the work has already been done in generating the ray cloud. The only step left is to convert the rays to an image. This means that a single ray cloud can be used to generate images from arbitrary view angles.

How could I go about accelerating this process using a GPU? The decision for each ray of whether it intersects the image plane is parallelisable as these decisions are independent, but each requires access to the objects in the scene as some rays will reach some potential image planes but not others. The final step of sampling the relevant rays across the pixels of the image requires each ray to contribute to a number of the surrounding pixels. Is this also parallelisable using a GPU?

To clarify, RichieSams' understanding is correct:

Start with a set of rays pointing in arbitrary directions.
Find which of these rays intersect with the image plane, taking into account occlusion.
Calculate the colour of each pixel based on the rays that hit the image plane.

Ideally I would like to be able to accelerate this process sufficiently to move around a scene in real time, given a precomputed ray cloud over the whole scene.

I'm not quite sure I fully understand the problem, this is what I think you're asking:

You start with a set of rays pointing a all sorts of directions
You want to know which of these rays intersect with the image plane, taking into account occlusion.
I'm going to assume these rays carry some kind of color information along with them. IE. when they hit the image plane, they will accumulate color.

With that goal in mind, I would do the following:

Create 2 image buffers

For accumulated color
To keep track of the number of hits

Clear the accumulation buffer to black, and the hitCount buffer to all zeros
Create a kernel per ray

Use classic ray tracing math to find the nearest intersection

Here you can do anything from a simple 'iterate through all the object in the scene' to something like BVHs, or kd-trees.

If the closest intersection is the image plane, use the WorldView matrix for the image plane to transform the intersection point from world coordinates to view space coordinates.
Use the x, y coordinates of the image space intersection point to get the pixel coordinates of the image buffer
Add the color of the ray to the corresponding pixel in the accumulation buffer
Increment the value of the corresponding pixel in the hitCount buffer

For 4 and 5 you could also use a filter in order to apply the color to multiple pixels, rather than just one. In that case, you would add a value less than 1 to the hitCount buffer

On the CPU, or in another pass per pixel:

Divide the accumulated color buffer by the hitCount buffer

I don't see any problem with doing these steps on the GPU. Any of the compute languages should handle it fine. You might get some divergence in the actual ray tracing code, but it should be ok. The biggest potential problem is that you have to store the entire scene in VRAM.


Are non manifold meshes problematic for Virtual Reality? A simple question. I know that non-manifold meshes are problematic for physics simulations, boolean operations, and 3D printing. I was wondering if they can be problematic for a virtual reality environment as well.

No, VR is no different to flat graphics applications in this respect. If your VR experience has physics, CSG (Boolean geometry), or any of the things that would have problems with non-manifolds in a flat application, then it'll have the same problems. If your application wouldn't have problems with non-manifolds as a flat application, it won't have problems as a VR application.

The only real difference is that it's harder to stop the camera being inside objects in VR, since it's attached to the user's head. In a flat application, you might override the camera position or adjust the camera controls to avoid the camera moving through objects in the first place. In VR, those techniques don't really apply, so you might want to have an inside-outside test, so that you can react if the user puts their head inside an object (e.g. by blanking the view completely and/or asking them to move). The inside-outside test would obviously have problems with non-manifolds. It's the same problem as in a flat application, but in a flat application you might not be doing the test at all.


Beginner CG Research Project This summer I will be working with a professor on a research topic of my choice, in the field of graphics & simulation. While my professor is knowledgeable in the area he is being very hands off as he does not want to make the student research his research. The intent is to use OpenGL to work on the topic.

We previously had a talk about the idea of taking a scene, specifying a water level, flow direction, and then generating a heat map at that height level of water disturbance to then allow us to render a scene at low cost with "semi-realistic" water flow.

While this idea interests me, I was looking to try to look into more ideas regarding fluid's or possible other area's of the field to attempt to find other possible projects.

In short, what would be a good place to look for entry level research projects regarding computer graphics? Or just idea's in general to help steer a beginner.

If you are going for physics. You can do physically based rendering. Physically Based Rendering(PBR for short) is a filed of computer graphics where we take consider of physics(energy conservation, spectrum, etc) in to account and try to not do approximations.

The are two main sub fields of PBR.
(quick intro, layman's term below)

Production Rendering.
This is where accuracy is the focus. We do as little or no approximations as possible. There are algorithms in this field where if the wave-like property of light is ignored. Then the result generated by the algorithm is physically accurate. But each frame might take more than a few minutes or even hours to render. There are a lot of different topics actively being researched in this field(fast ray-scene intersection, better Monticello integration, fast/robust acceleration struct building)
Realtime Rendering.
In this field, speed is the key concern. Where accuracy is achieved by doing good approximations. But will never be physically accurate. Most research on this field is focusing on doing good and fast approximations.

These are my random ideas for fluids.

Fast Ray-Liquid intersection testing
Fast triangle mesh generation from particles.
Fast acceleration structure building from particles.
Simplification of liquid surface to generate simpler mesh

And some random ideas for general GC.

Improve sampling efficiency for sky maps in ray based rendering algorithm
Robust method of dealing with floating point accuracy problem in shadow maps

I don't know if the above topics are possible or not. Those are just ideas.


How can I get a spherical visibility mask map for a point on a concave surface? I encountered this problem doing my project. Suppose there is a 3D mesh model, e.g. a human face, I need the 3D visibility map at each of the vertices of this model. By the "3D visibility map", I mean the binary image covering the spherical surface indicating whether the point is visible from each spherical angle(different from ambient occlusion). So, it is like a mask image. For example, the visibility map for any point on a convex surface is all ones (occlusion by other objects is ignored, only self-occlusion). See the following figure for illustration

I want to know is there some efficient ways solving this problem? I need to apply it to all vertices. All I have in mind is doing brute force verification for all sampling directions. What's more, is there any existing software or tool (e.g. pbrt, Unity, Blender) that is capable of this functionality? Any suggestions and links are welcome. Thanks!

This can be done analytically at least for polygonal meshes. You can convert points into polar coordinates and project on a sphere. Edges form planes that pass through projection circles center forming circles on the sphere. These lines are all great circles, because they must pass sphere center. Great circles form linear interpolations in polar coordinates.

Image 1: Projecting a geometry on a sphere.

Converting these great circles into either Cartesian coordinates is quite well known math. Projecting them onto 2d representation by Mercator projection is well known math on account fo this being central to map making. Since overlap is not a  problem you can just slap these triangles on top of each other and merge the vector results. Or use the polar coordinates for pixel graphics and even use z buffered overlap just like a normal camera.

The only problematic point is the one where your sphere is generated (if its on a open face edge). You can simply project the point on center into its inverse normal direction on the projection sphere. Or if for example Mercator projection is desired, you can align the sphere with surface normal. then just project them to a point on the south pole.

You can even interpolate the results of individual polygons positions to get any point on the triangle. And a analytic surface coverage for the triangle.

I haven't implemented this, but Ive done it manually a few times* so i know its possible. Should be pretty easy to do as the maths involved aren't all that complicated only problem is solving which side of the problem to keep.

Image 2: Horizon map of a simple all local geometry.

Comparison with raytracing

Both sampling triangles analytically is O(N) algorithm, if vector data is sufficient. Conditioning this data may push you over to O(N^2). Raytracing is O(N log(N)). Raytracing algorithms are easier to find of the shelf so implementing analytic polygon to sphere rendering is harder. In terms of speed analytical rendering is faster as long as theres a limited number of triangles, much like how scanline rendering is still sometimes faster than tracing despite being algorithmically more complex. This similarity is not just a coincidence it can be shown that this method can be turned into a scanline renderer.

Resources for drawing great circles in Mercator projection:

Wolfram demonstration on subject.
The Projection of Circles on a Mercator Map

* About 20-25 times. I also did it for this image (yes in 3d), its as accurate as a single span Bezier curve can represent a circle. Ive also done this in map making software a few times.


Luminance and Radiance in computer screen I am currently building a graphics engine. I have a slight confusion on the imaging pipeline of computer screen. My question is how does computer transform the colour in the final frame buffer to radiance on the screen. My assumption is that the final colour in the render target should be proportional to luminance instead of radiance. Then, there is some function in the pipeline that will apply transform inverse of the photometric curve of this value to get the radiance of each RGB value. Am I correct?

1, 1, 1, the output value is white. If this represents radiance than the output should be green, because when the signal radiance is the same, the pixel should be perceived as green. Am I correct?

This is not correct. Your frame buffer most likely contains sRGB values. Of which 1,1,1 is used to represent white. Standard displays do not use spectral radiance values in the frame buffer. sRGB does not map directly or make any guarantees about the spectral power distribution (SPD) of the display. So "G" of 1 does not guarantee some wave with amplitude using W⋅sr−1⋅m−2 centered on the 550nm. Part of the problem is that different type of displays have different SPDs that map to the same perceived color.

[Evaluation of an organic light-emitting diode display for precise visual stimulation, Ito et. al]
(https://jov.arvojournals.org/article.aspx?articleid=2121657)

Now if you want to physically accurate rendering, you could store all of your lighting values as Radiance with SPDs. As L.C. mentioned, Physically Based Rendering describes that this is done with piece-wise linear functions with like 16+ channels per color. However even if you can get this to be performant, good luck converting all your RGB data into SPDs (there's an infinite # of SPDs for a given RGB value).

Instead what most renderers try to do is to treat the RGB as pseudo-radiance values for the purposes of lighting calculation. This isn't accurate, hence your lighting result changes if you are using Rec.709 or Rec.2020 primaries for instance (if you notice they have different values for mapping to luminance), but it's good enough for real time and even offline in most cases. At this point you can convert RGB Rec709 and Rec2020 values to a single scalar luminance using the Y values of it's primaries.

The Y values of the RGB primaries for Rec709 are:

    R       G       B
Y   0.2126  0.7152  0.0722

https://en.wikipedia.org/wiki/SRGB

Thus Rec709 Linear has the following nits conversion:

nits = dot(linear709Rgb, float3(0.2126, 0.7152,0.0722));

Probably one of the results of doing our calculations in RGB is that G and B add at equal weights, so it's likely to get some color shifting. There was a presentation at SIGGRAPH 2015 from Weta digital that had some examples, but it doesn't seem like the slides are available. As far as I know they are the only ones using a spectral renderer.


What is the fundamental reasoning for anti aliasing using multiple random samples within a pixel? In graphics, it's common to take multiple samples within the bounds of a pixel and combine them together (most commonly just doing an average) for a final sample pixel color.  This has the effect of anti aliasing an image.

On one hand this makes sense to me because what you are effectively doing is integrating the color of the pixel over the area that the pixel represents.  In this line of thinking, averaging "random" samples seems to be the ideal setup, for doing monte carlo integration.  ("random" could be stratified, blue noise based, low discrepency sequences etc)

On the other hand, this feels wrong (or at least not as correct as it could be) from a digital signal processing point of view.  From that point of view, it feels like we are taking a lot of samples and then downsampling using a box filter (box blur) to get the final pixel value.  In that light, it seems like the ideal thing to do would be to use sinc filtering instead of averaging the samples.  I could see that the box filter is a cheaper aproximation to sinc thinking along these lines.

This leaves me a bit confused.  Is the core idea that we are integrating the pixel area and averaging is correct?  Or is it that we are downsampling and should be using sinc, but are using a box filter because it's fast?

Or is it something else entirely?

A little bit related: Anti-aliasing / Filtering in Ray Tracing

From a signal processing point of view, you're sampling a continuous-domain signal, and you need to filter it to get rid of frequencies beyond the Nyquist limit. It's that filtering that leads to integrating over the pixel area—or more generally, integrating over the support of your antialiasing kernel (which need not be a box).

Consider your rendering function that takes a sample point $x, y$ in screen space and returns back the color found at that point. (Let's ignore any issues of random sampling for the moment, and assume it returns a "perfect" rendered color for that specific point.) This function effectively defines a 2D continuous-domain signal. Or to put it another way, it defines an infinite-resolution image, since nothing prevents this function from having features at arbitrarily small scales. In terms of frequency domain: the function is not band-limited; it can include components of arbitrarily high spatial frequencies.

Now you want to convert it down to a finite number of pixels. Much like digitizing an audio signal, when you sample it, you'll get aliasing unless you first eliminate frequencies beyond the Nyquist limit imposed by the sampling rate. In other words, you have to get rid of features smaller than the pixel grid. To do this, you apply a low-pass filter. The ideal low-pass filter is the sinc function, but for various reasons of practicality we use other filters (that don't perfectly eliminate frequencies beyond the Nyquist limit, but they at least attenuate them).

Low-pass filtering is done by convolution. If $f(x, y)$ is the rendering function and $k(x, y)$ is a filter kernel, then we can define the low-pass filtered image as:

$$f_\text{filtered}(x, y) = \iint f(x', y') \, k(x' - x, y'- y) \, dx' \, dy'$$

Then the image is safe to sample, so the final pixel values are gotten by just evaluating $f_\text{filtered}$ at the pixel coordinates.

If $k$ is a box filter, which looks like $k = 1$ within the pixel box and $k = 0$ elsewhere, then this simplifies to just integrating $f$ over the pixel box. But as noted, the box filter isn't so great and there are better choices such as tent, bicubic, and Gaussian filters.

Anyway, now that we have an integral, we can use Monte Carlo for it and combine it with any other integrals we might like to do—for lighting, motion blur, and so on. We can even apply importance sampling to the $k$ factor in the integral, by generating samples for each pixel that are distributed around the pixel center according to $k$.


Special directional light type I am currently trying to implement a specific directional light type.

This light type has been used in the game INSIDE and is called orthogonal spotlight (aka local directional light). I assume that this is a directional light which behaves like a spot light and have a squared or rectangular attenuation but I have some difficulties to integrate it in my deferred pipeline and get the general concept of this light type.

Classical directional light : dot(worldNormal, lightDir)

I hope that you'll be able to give me some clues.
Thanks a lot !

Alan Wolfe is pretty spot on, but I will sum up any way :)

render the back-faces of your "unit"-sized-box, [-1;1]
sample the zbuffer and transform into light-local coordinates (see our slides for a fast way to do that).
early-out of the pixelshader if outside of the box, i.e. if any coordinate in the local-position is outside of -1;1.
then do the dot-product between the normal and the lightdir, in whatever space is convenient (you can extract the worldspace-lightdir from the object-to-world matrix).
projecting a texture is easy, the UV-coordinates are simply the localspace xy (moved to [0;1])
using a falloff texture is eays, the coordinate is simply the localspace z (moved to [0;1])

In an actual game, you of course want to use some sort of acceleration to discard early; we used old-school stencil culling (and should have used depthbounds), but likely tiled culling, clustered culling or similar is better.

You could use zgreater testing to discard pixels where background is entirely behind the light, but iirc on most modern GPUs this will not actually be an advantage, as there will be no early-discard due to the hierarchical zbuffer being primed in the direction of zless.


Help me find out what this texture mapping technique is called I'm in the process of making a tool that requires rendered texture to follow the contours of a piece of clothing. An example would be this website https://knyttan.com/editor/jumper-editor/. The effect here is achieved by using a colour map:

I looked at the shaders that are used for this and it seems that the texture offset is calculated based on the colour channels from this map.

Now I was wondering if this is a complete bespoke way of doing this, or if this is a known technique and if it is what is it called ?

What you see in the image called a UV map. That is, it is simply texture coordinates to be looked up encoded in a image. Same thing happens in all texture lookup in 3D there is a underlying sampler that picks where to pick texture color from.

Image 1: Image showing UV map of two overlapped triangles and sampled texture with same UV coords

Here are the sources for those images, please do not overwrite the sources.

Demo showing the UV map
Demo showing the texture


High variance in path tracing specular objects I'm testing my path tracer for specular surfaces. Images rendered with my own code have very noisy appearance, and I think they look like variance.

I use point light and the bunny is of mirror material, walls are matte. If it is the problem with sampling, how should I modify the program. To my understanding, point lights are specially handled and thus the sampling results should be a constant value. So I don't think the noise is caused by light sampling. So could there be any bugs?

When only matte material exists in the scene, there are also variance, but not that bad, and global illumination effects are visible. So what could the problem?

  specular bunny:

  matte bunny:

For the specular bunny image, spp is 4. Since it is a point light, wi is directly set to the vector between point light and intersection point when calculating direct illumination, so there is actually no "sampling". When computing scattering directions, BRDF is matte when intersecting the wall, and BRDF is specular reflection when the ray hit the bunny.

Matte BRDF is cosine weighted sampled, while specular reflection just use the perfect reflection direction.

If your bunny is purely specular, then sampling the light directly at the shade point would give no contribution since the specular BSDF is a delta BSDF. It generally evaluates to zero for any direction other than the mirror direction. If it was a glossy BSDF, then it might be possible that the pdf value could be very small so that the monte-carlo estimator $\frac{f(X)}{P(X)}$ just explodes (which is generally the source of fireflies i.e having a very small pdf and the function is large).

If it was a mirror direction, then you should just see a black bunny for direct illumination and just the reflection of the surrounding scene for multiple bounces. Multiple bounces from the specular bunny should not be cause fireflies, because if you're sampling based on cosine term at the matte surface, the estimator becomes $\frac{L_i(X)f_r(X)\cos(\theta_i)}{\text{pdf}(\theta_i)}$, where $\theta_i$ is the sampled direction using BSDF sampling at the matte surface. The pdf and the $\cos(\theta_i)$ should cancel out each other perfectly. However since you're using a point light, you can never sample from the BSDF since the light is a delta light and needs an explicit connection. Hence the pdf for the estimator would be 1 all the time. The only way for large values to creep up are in the numerator. So I would suggest you to check the values by increasing bounce one by one to see when the problems start creeping in.


Need help with VBO/VAO It's been a long time since I last used OpenGL and I can't figure out VAOs anymore... I want to make a simple voxel engine using geometry shaders. This is what I want to do:

vbo1: array of GLfloat for the positions (3 floats per vertex)

vbo2: array of GLfloat for the size of the cube (1 float per vertex)

vbo3: array of GLbyte for the type of voxel (1 byte per vertex)

-> vao: GLfloat x, GLfloat y, GLfloat z, GLfloat size, GLbyte type

Here's my code:

    float *positions = new GLfloat[size * 3];

    positions[0] = 0.5;
    positions[1] = 0.5;
    positions[2] = 0;

    glGenBuffers(1, &positionsId);
    glBindBuffer(GL_ARRAY_BUFFER, positionsId);
    glBufferData(GL_ARRAY_BUFFER, size * 3 * sizeof(GLfloat), positions, GL_STATIC_DRAW);

    byte * voxelValues = new GLubyte[size];

    voxelValues[0] = 0;

    glGenBuffers(1, &voxelValuesId);
    glBindBuffer(GL_ARRAY_BUFFER, voxelValuesId);
    glBufferData(GL_ARRAY_BUFFER, size * sizeof(GLubyte), voxelValues, GL_STATIC_DRAW);

    float * sizes = new GLfloat[size];

    sizes[0] = 1;

    glGenBuffers(1, &sizesId);
    glBindBuffer(GL_ARRAY_BUFFER, sizesId);
    glBufferData(GL_ARRAY_BUFFER, size * sizeof(GLfloat), sizes, GL_STATIC_DRAW);

    glGenVertexArrays(1, &vao);

    glBindVertexArray(vao);
    glBindBuffer(GL_ARRAY_BUFFER, positionsId);
    glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 0, NULL);
    glBindBuffer(GL_ARRAY_BUFFER, sizesId);
    glVertexAttribPointer(1, 1, GL_FLOAT, GL_FALSE, 0, NULL);
    glBindBuffer(GL_ARRAY_BUFFER, voxelValuesId);
    glVertexAttribPointer(2, 1, GL_UNSIGNED_BYTE, GL_FALSE, 0, NULL);

    glEnableVertexAttribArray(0);
    glEnableVertexAttribArray(1);
    glEnableVertexAttribArray(2);

Is the code even doing what I want it to do? And is it even possible to use different datatypes in the vao?

Look good to me.

And yes you can use different datatypes for different attributes in a vao, that's the entire point of having the type parameter there in the first place.


How can deal with batch rendering when the elements in the batch changed every frame? Since I am using batch rendering, so I should pack every render units can be batched into one big VBO. But, what if I am in a case where render units are dynamically changed, some new units are added in, and some units are subtracted out. So I should reconstruct my VBO again and again in very frame. So here is the question.

Should I allocate a big enough storage in the first time and update the data in it, and only reallocate when run out of the storage, or allocate storage according to the size on demand in each frame?

Allocate a VBO with a reasonable amount of storage during startup, and update the data in it each frame. This will help the driver manage the memory efficiently. Don't destroy and recreate the buffer unless absolutely necessary; too much resource creation/destruction churn will put more pressure on the driver and can potentially lead to stalls.

To be more specific: for data that will be updated every frame, you probably want to initialize it using glBufferStorage with GL_DYNAMIC_STORAGE_BIT (or, if you're on an older OpenGL version, glBufferData with GL_DYNAMIC_DRAW). Then, to update it each frame, use glMapBufferRange with GL_MAP_INVALIDATE_BUFFER_BIT. This should be an efficient, fast path in the graphics driver.


How can I chase down VRAM & GTT memory leaks? Problem. I have inherited a large C++ 3D graphics codebase (with some shader code). The program runs for about 1-5 minutes before crashing.

Problem Data. Using system tools, one can see a sharp rise in VRAM memory usage after the program launches, which eclipses into a crash. The same is true for GTT memory usage also (although the spike isn't quite as dramatic as with the VRAM).

Questions.

Does this (probably) mean that there is a memory leak in OpenGL shader code somewhere (as opposed to a leak in regular C++ code)?
Are there well-known tools for tracking down vram/gtt memory leaks (similar to the way that valgrind can be used to track down system memory usage)?

Probably not. Most OpenGL resources (framebuffers, vertex buffer objects, textures &c.) are controlled by host code (i.e. your C++ code), but take up GPU memory. OpenGL doesn't make it very easy to manage these resources, so it's most likely that they're being created but not deleted from your C++ code. For example, it might create a new VBO per frame but never delete the old VBOs.
I've never used it for this purpose, but if you're on Windows I'd suggest RenderDoc as your first stop. It's not specifically designed for debugging source - it intercepts GL and DX calls so you can see how the application is using the library even if you don't have the source - so it won't point you to the offending line of code, but it does show you what resources are being created and when, so it should at least tell you what is being leaked and how often. It might even be that this is enough information is enough for you to guess the rest, or that the easiest way to continue after this is to do something by hand (e.g. printing __FILE__ and __LINE__ every time such a resource is created).

Sorry this is a bit of a half-answer but it's too big for a comment. I hope it gives you a good starting point at least.


Data fitting SVBRDF with L-M I am reading two shot svbrdf capture . I want to understand the data fitting.

For example, I have my input image 3000x2000 pixels, and I divide it to create my source tiles of 192x192 pixels. The way I understand this paper is, each pixel of each tile will be the svbrdf. Now we have to find the BRDF parameters for each position with the L-M method.

Then, we get svbrdf parameters for 192x192 pixels.
After the data fitting what did they do to obtain the normal map, specular albedo and diffuse albedo.

I read that paper as well last year. There's a strong assumption made by authors which is the "stationary". What this means is that in absence of light in given 3000x2000 pixels image the material is more or less the same in each tile of 192x192. The reason for this is because we the surface is lit by white light is like they have a sample of BRDF for a set of angles.

The first shot is to get the stationarity of the material. The second shot is to get how each tile reacts to the interaction with the light from a different angle.

They collect the set of all tiles with the light and they later use their BRDF model to perform the fitting.

The fitting, as you say, is done by using the L-M algorithm. There're few inconsistenties with the formula they use to perform the fitting, if I remember correctly the use non differentiable functions and they still perform the fitting.

If I remember correctly the actual fitting algorithm is constructed by one hand differentiating part of their cost function, while from the other hand they use a numerical approximation of the derivative/gradient/Jacobian.

The BRDF they use is parametrized using the several maps you mention in your question, therefore when they minimize the cost function they get optimal maps.


How to do CSG on 3D triangulated objects, originally stored as a .obj file, using openGL or other libraries? What I would like to do is to create a crater on a field. To do so, I thought that creating a universal crater object (something cone-shaped) and then subtracting it from the field on a specified point might be a good idea.

So I would specify a "middle point" on the field, take the lowest point of the crater and move the crater by its lowest point onto the middle point of the field and then push it by its size of the crater into the field. Then apply said subtracting operation.

My question is: Are there any techniques/libraries which help me accomplish this? They should be either openGL-compatible or work on .obj file format.

Describing a full CSG algorithm in a forum post is something really hard, and probably you will not find a complete answer to what you are asking here, but I can give you some suggestions.

First of all, CSG on polygon meshes is not usually employed on real-time graphics, because the algorithms are still considered to be too much resource-intensive. If you are looking for these kinds of algorithms, I would suggest reading books on computational geometry, of which Mark De Berg's is commonly considered a good introduction to the topic. If you are planning to pre-compute everything, then classic CSG algorithms are the way to go.

However, for just adding craters on a field, CSG seems to be too much overkill to me, and I would suggest some completely different approaches. If you want just a visual result (i.e., your player won't be able to go into the crater), you could use techniques such as z-buffer carving or parallax occlusion mapping, combined with some sort of decal system (preferably, using some deferred-like rendering system). Otherwise, if you need proper collision detection, you will actually need to change your mesh. You can easily apply some vertex modification operators around the center of each crater, such as pushing them down according to a radial-basis function (i.e., spheres, gaussians, etc). To accomplish this in realtime graphics, I would suggest using compute shaders or vertex feedback transforms, instead of computing everything in the CPU. These techniques will even allow you to add craters dynamically. Think of some really nice meteor strikes taking place.

I'm sorry for not being too descriptive, but I hope that some of the keywords that I have put here may help you out on finding a solution to your problem.