Alpha blending in linear colour space I've been making a simple game in OpenGL, and implemented a screen fade-out using the old 'draw a black fullscreen quad and ramp up the alpha' trick. I'm doing all my shaders in linear space and using GL_FRAMEBUFFER_SRGB to automatically apply gamma. When fading out the screen I noticed that the alpha blending also becomes nonlinear - to fade to ~25% brightness I need to use an alpha of ~0.95.

I fired up Unity for comparison, and its alpha blending also seems to work this way - when rendering in linear mode most of the opacity is bunched towards the top of the alpha range.

Is this expected behaviour? Is it correct from a PBR point of view? It seems very unintuitive. As a workaround, to get more intuitive blending, would it be reasonable to apply gamma correction of 0.45 to the alpha channel before blending?

EDIT: this is the result I get after blending a .95 alpha black quad over a pure white button. It could be bad gamma on my display as Simon F suggested in the comments, but it looks a lot brighter than 5% to me.

Just turning the comments into a fully-fledged answer as it may prove useful to others.

IMHO, the alpha channel should always be linear and treated as such.

You described blending a black quad over a white background using alpha=0.95, and mapping the result back to non-linear space. We'd expect a resulting, physical intensity value of 0.05.

If the non-linear space is sRGB, then a linear value of "0.05" would correspond to an sRGB value of "0.248" which, when mapped to 8-bit, is around 63 (or hex, 3D)

If your display is reasonably calibrated, you can verify this by creating, say, a 5x4 black image, setting one pixel to white, and then tiling that to a suitable size, e.g 200x200 pixels. Then draw a, say, 50x50 square in the middle and fill it with 0x3D3D3D. Here's an example:

If you take a few steps back from your monitor and squint, the centre grey region and the dithered outer should be almost indistinguishable, demonstrating a correct physical model.

The problem you are describing appears to be related to the fact that our visual system is non-linear, much as our hearing is too (though probably not with the same curve). For example, the analogue volume control on a hifi amplifier will use a logarithmic potentiometer rather than a linear one .

You thus 'simply' need to change your per-frame alpha non-linearly to get a perceptually even fade out.


how to put a pixel somewhere in a window(directx) I'm interested in direct3d game programming and I'm just a beginner.
I'm familiar with the Win32 API and i was wondering...
how to put a certain pixel somewhere in the window
is there like a function or anythting?
thank you...

there is no direct way to place a pixel on a window I am afraid.
An indirect way would be to draw a small quad equal to the size of a pixel in relation to your window
and essential y d3d 11 and 12 are most useful to draw geometry
So I recommend you start from there and take your creativity ahead


How Sunlight is added in Bidirectional Path tracing? How is distant directional light source(like sun) added in BDPT?
If this is an indoor scene that gets sunlight through windows, how is this handled in BDPT(other than to consider Sun as a big disc far away and to sample a position on it's area, as this probably wouldn't be efficient for points within the room)

Would it be valid instead to consider window polylines as area sources that emits light in one direction(directional source)? The light samplePosition would be same as sampling a point on the window area (with pdf 1/A ?) and sampleDirection with pdf=1? How about connecting s=1 and t and MIS weights? are they need to be modified or same codes can be used?

Any help/hint is appreciated, Thanks.

This can be done in two ways - either use a sky with a sun in it as an cubemap-type light source outside your building, or use the windows as light sources. With BDPT, you could sort of do both, and consider the windows area light sources where the light differs depending upon the angle, using a sun and sky image/model, and then you could implement importance sampling so that you usually get light rays going from the sun instead of the darker sky. This way you get all the accuracy of a full sun and sky model, and it'll still look good at low sample rates.


How to subdivide faces with a half edge DS? Assume we have a mesh represented as a half edge datastructure.
We want to subdivide faces without introducing unnecessary vertices (i.e the new number of vertices must be exactly V + E where V is the old number of vertices and E is the old number of edges).
This is what a single subdivided triangle looks like:

If I naively try to iterate over every edge and reconnect faces I get the following sequence

Clearly this is wrong.
If instead I iterate over the faces I get the following pattern:

This is happening because at every step I am trying to to:

Focus on only one simplex at a time,
Maintain the connectivity information of the mesh in a valid state (every edge has <2 faces, vertices only exist at the end of edges, faces only have 3 Half edges...), like for example, I am avoiding getting into this state:

Any suggestions?

No one answered so I will. The solution is to do the subdivision in 2 passes, first generating new triangles as I showed in the question.

Then on the second pass you flip them.

As described there:
https://github.com/cmu462/Scotty3D/wiki/Loop-Subdivision


2D Line of Sight + Visibility Calculations I am currently working on an algorithm to calculate visibility in a 2D top-down level.

The level contains

actors, with position and "sight" information
walls, with start & end positions
light sources, with position and range

I have written a fragment shader that renders my scene based on a texture I compose of the informations above.

Red channel = line of sight for actors (in the example just the selected token is an actor)
Green channel = light sources with their range and their respective line of sight
Blue channel (not present in example) = night-vision of actors (means areas are shaded despite no light present)

I am using Babylonjs and am naively rendering all those calculations one after the other into a DynamicTexture using the canvas 2d context.

Can I use the GPU for some of these calculations (or all?) I am no expert for shader programming and don't know where to start / look.

The code for my line-of-sight calculation can be found here: https://gitlab.com/aiacta/visibility

In this SIGGRAPH paper is described a similar operation for the visibility calculation from every pixel in the domain (you only need calculate it from actors position), which has been implemented in GPU.

Although it is oriented to fluid dynamics, it may be a starting point to look.

You can see other related shader examples like this one with implicit geometry.


How to count the photons when Photon Mapping I wrote a rather elegant and compact Photon Mapper in CUDA. But I treat all my photons they same, and ignore their colour.
This simplicity has some nice properties: e.g. I can just count them with one integer value for each little subdivision patch on the surface.
However, to properly model the global illumination, I should take into account the colour, after a photon bounces from a surface. (A red carpet, reflects red light onto a nearby wall, not white light.)
An obvious approach would be to replace the single integer count with three floating point counters, for R, G and B.
But I feel there must be a more elegant solution to this? Russian roulette on the R/G/B channels perhaps? Although this would create more noise? But it would mean fewer data per photon, and thus better use of the cache?

I'm not sure if I understand your question correctly and don't know what you mean by "counting" and how you want to replace this by "color". A simple photon mapper looks like this: 1) photon tracing: emit, scatter, store photons 2) path tracing: gather photons in a given radius and approximate radiance using this density.
To be more precise, in pseudocode (with russian roulette) this would look like:
xi is the position, wo the outgoing direction, W the photon power. All three are vectors of length 3. p is the probability of the Russian roulette. A photon is a struct that stores the power, direction, and position.
xi, wo, W = emitter.sample()
while(true)
   xi = traceRay(xi, wo)
   if(isDiffuse(xi))
      storePhoton(xi, wo, W)

   if(rand < p)
      break
    W /= p;

    wo = bsdf.sample(xi)
    W *= bsdf(wo)/pdf(wo)*cos

You repeatedly call this function for a random emitter until you stored a predefined amount of photons (e.g. 500'000). After this, you build the photonmap, such as a k-d tree to make querying easy. As you can see, the "color" is stored in the power of the photon.
The next step is the gathering the photons, and it's normal path tracing but when you hit a diffuse surface, you gather all the photons in a fixed radius (other kernels can also be used). Here, t is the throughput, wi the incoming direction, the photon_map is the datastructure that we build before and populated with photons, and total_photons is the total amount of photons that we emitted.
...

if(isDiffuse(xi))
   for photon: photon_map.get(xi, radius)
      radiance *= t*bsdf(xi, wi, photon.wo) * photon.power / (pi*radius^2 * total_photons)
   break
...


How to change a picture to fail comparison algorithms? Is there a way to change a picture so it would fail to match against existing copy of the same picture on the big social networks such as Facebook, Twitter, Instagram, Tinder and others? Desired result should be as close as possible to the original.

It's dependent on the algorithm used for comparison, but if you have access to that algorithm, you can add small amounts of noise to pixels randomly until it misclassifies the image.

If you had deeper knowledge of the algorithm used, and it was differentiable, you could use gradient descent (ascent) to modify the image to increase the error of what was detected. This is the reverse of what you do when training image comparisons/classifiers which modify the weights to minimize the error.

But in short, if you dont have much knowledge of how the algorithm used works, I personally would try adding noise to pixels randomly, such that the image was still easily recognizable, but a decent amount of noise was added in.


What is wrong with my metallic PBR shader? I feel like my light conservation math isn't correct, but I am not sure which part is wrong.

Is reflected light supposed to influence the diffuse light?

Does metalness only affect the reflected light or does it also affect ambient and diffuse light?

vec4 falbedo; // surface color
vec3 fnormal; // surface normal

vec3 ambientLight;

vec3 directionToCamera; // Camera normal
vec3 directionToLight; // Directional light normal
vec3 lightColor; // Directional light color

float metalness; // 0-1
float shininess; // 1-256
float subsurface; // 0-1

// Diffuse light
float lightNormalDot = dot(fnormal, directionToLight);
float lightIntensity = max(.0, lightNormalDot);

// Blinn–Phong reflection
vec3 halfway = normalize(directionToCamera + directionToLight);
float reflectionFactor = max(.0, dot(halfway, fnormal));

float reflection = pow(reflectionFactor, shininess);

vec3 res = falbedo.rgb * ambientLight;
res += subsurface * lightIntensity * lightColor * falbedo.rgb;
res += (1. - subsurface) * reflection * mix(lightColor, lightColor * falbedo.rgb, metalness);

gl_FragColor = vec4(res, falbedo.a);

What is wrong ? Almost everything. It is not a PBR material at all. PBR metallic materials do not use ambient term and do use generally microfacet BRDFs and ComplexFresnel.


Tone Mapping vs Gamma Correction There is something about these concepts I'm not understanding.  I thought the point of gamma correcting images is to make sure that the values in linear space map correctly when displayed on screen:

It seems from this picture that the camera stops are all getting the same representation (code words) when mapped to the output (purple line).  If this is the case, we shouldn't need tone mapping, but obviously we do.  There is something not right in my thinking and am hoping someone in the internets help me fill in the gap.  Thanks!

I think it would be a good idea to get back to the basics. It's a large post, so there's a recap at the end.
Colour can be represented in a couple of ways. We can have Red, Green and Blue (RGB). We could have CMYK (used in printing). There's also YCbCr/YUV (one for luminance and two for colour) and similar flavours of that. We have HSV/HSL/HSB (hue, saturation, value/luminance/brightness). And even more. These are called colour models (or color models, depending on where you live). RGB is the most popular model in terms of computer graphics. It's how colour film stock, digital sensors, monitors and roughly our eyes work. So, from now I'll be using the RGB colour model. The things in the rest of this post will have equivalents for the other models, but for simplicity it's just RGB for now.
We have three values, R G and B, but that raises the question. If we have 100% red, 0% green and 0% blue, what kind of red is it? The answer is, we don't know. So, we have to define what kind of red red is, what kind of green green is, what kind of blue blue is and what kind of white white is. White is also important to define. This definition is called the colour gamut. The colour gamut defines the chromaticities for red, green, blue and white. Chromaticity is colour but without the luminance (basically just the hue and saturation). By defining the colour gamut, we will know exactly what colour we have for any given RGB value.
But there's another thing. If we want to send an image to our monitor or save it to a file on our drive, we don't want to have to send over or save a lot of data. We basically want to use the least amount of bits we can. We found out that having 8 bits for the red, green and blue values is enough. However! Our eyes are more sensitive to changes in brightness in dark areas and less sensitive in bright areas of an image. So, while 8 bits is enough for the bright areas, in the darker areas we can really see that we only have 256 possible values for red, green and blue. We fix this by moving around the values in order to use more of those 256 values for the darker areas and less on the brighter areas. Right now I am using 8 bits as an example, but this is the same for any amount of bits (10 and 12 bits is also popular but has the same problem). We do this using a transfer function. In short, a transfer function redistributes the values in order to more efficiently use a limited amount of bits.
Gamma correction (or just gamma) is a form of a transfer function. As you can see in your graph (the blue line) it nudges the values upwards which means that the range [0,0.5] becomes [0,0.73]. This means that we have almost 50% more bits for the darker areas of our image! That's what we want. Gamma isn't the only kind of transfer function. Another popular one is log, which is an umbrella term for transfer functions (or colour spaces using transfer functions) that make use of logarithms. With logarithms, each stop of light gets roughly the same amount of bits, which is really useful for images that have a very high dynamic range (like what comes out of a cinema camera). Perceptual quantizer (SMPTE ST 2084) is another transfer function that is used a lot in HDR video (like HDR10 and Dolby Vision) and it's made to take advantage of how our eyes perceive brightness (which falls somewhere between a logarithmic curve and cube-root curve). Values that haven't been transformed by a transfer function are called linear.
When you combine a colour model, colour gamut and transfer function, you get a colour space. A colour space specifies how the colours are represented (what colour is what). There is one very important thing however, you do not need a transfer function for a colour space. sRGB is a very popular colour space which is used by practically every computer monitor. It uses the RGB colour model with the sRGB colour gamut (Rec.709 uses the same gamut) and the transfer function is generally a gamma of 2.2 (although it actually isn't, but most use the gamma for simplicity, go look at its Wikipedia page). But you also have linear-sRGB which is also a colour space that just like sRGB, uses the RGB model and sRGB gamut, but doesn't have a transfer function. The values are in linear (often referred to as "scene linear").
To try and make things a bit more clear, here's an example showing how a transfer function works when displaying an image that is either CGI or a video game being rendered. The first thing that is important is that both the computer/program and the display need to agree on the colour space to use. In this example, we use sRGB. Basically, the program sends the sRGB image to the display and the display takes that sRGB image and converts it to the actual electronic signals to send to the pixels.

And now we finally get to tone mapping. A tone mapper is a function that takes in a large, generally infinite, range of values and compresses it to a smaller, often limited, range of values. Tone mapping is generally used to apply a "look". A very popular tone mapping is the filmic s-curve, which is called a "film characteristic curve" or "Hurter–Driffield curve". It adds some contrast in the shadows and instead of the highlights clipping once they reach 100% brightness, it adds a nice roll-off that gradually brings it to 100%. This tone mapper is used in a lot of places. It's how film stock behaves, it's what digital cinema cameras often do, in video games they use this curve. Basically, because it looks good and with it we can see values above 100% (because the tone mapper maps it to something below 100%). If you want to tone map your image, it should be done before the transfer function/colour space conversion (for example converting from linear-sRGB to sRGB), because tone mapping isn't part of a colour space and is basically a look (just like applying a filter over an image).
However, you don't need a tone mapper. If you take a cheap camcorder, DSLR, your phone (depending on the settings) or a broadcast camera they use to film the news and such (especially before the 2010s), then all of them don't do any kind of tone mapping. They just apply the sRGB or Rec.709 colour space (and thus also the transfer function) and that's it. What you often do see is that parts of the image go to 100% white very quickly and then clips Here's a picture that I took with my phone. It's just normal average sRGB and no tone mapping.
You can see that the cloud is completely white and the trees are quite dark. Using a tone mapper, I could bring those values closer together and not have the clouds be 100% white without any detail, while still being able to see detail in the trees.
To recap

Gamma correction is a type of transfer function, which is part of a colour space.
A colour space tells us how to represent and store colours.
When we want to send images to our display, we need to apply the transfer function.
The display will undo that transfer function (and will do it no matter what, so you have to apply the transfer function on your end to get the correct values when displayed).
A tone mapper is both to apply a look and to compress the range of RGB values to a smaller range so that we can see a high dynamic range image on a standard dynamic range monitor.
While you need to apply a transfer function when sending the image to the display, you don't need to apply a tone mapper. Only if you want to.


Why are RGB tertiary colors so similar to RGB primary colors? I was suprised to learn that RGB tertiary colors are much more similar to RGB primary then secondary colors.
I was expecting they will be halfway between them and look more like this:

Rather then this:

Is there a reason/purpose for this disproportion?
(I know how to calculate values- this is not my question. I'd like to know why correctly calculated tertiary RGB colors seems to be much closer to primary then secondary RGB colors)

Likely a gamma correction issue. Tertiary colours are some permutation of (1, 0.5, 0) in RGB, right? So the issue arises because without applying gamma, 0.5 appears less than half as bright as 1, so the blended colour skews towards the primary (1,0,0) rather than the secondary (1,1,0). The image below has linear colour on top, gamma corrected on the bottom, you can see the gamma corrected colours appear much closer to a halfway tone.


Programmatically generating vertex normals I'm working with Kinect face api, it provides an array of vertices and indices for triangles which is to be rendered to make the face image.

The no of vertices and their order in array as well as the indices given by kinect is always constant.

However the api gives no information on UV data and vertex normals.

The application requires me to keep the vertex order as given out by the kinect as their positions in 3d space change as per facial movement so generating uv and normals in a 3d editing software is out of question.

I managed to generate UV by projecting the vertex positions onto a 2D plane  since there were very few vertices on same plane.

However, I don't know how to generate vertex normals for the mesh, without vertex normal the face mesh draws with no depth of its features from prespective, although silhouette is visible as vertex positions are correct.

I understand that due to absence of vertex normals the lighting wont work correctly on it and hence the pale featureless mesh it looks right now.

so how do I generate vertex normals when all I have is just vertex position and the index of vertices to make triangles out of it?

Computing the normal from vertex positions is quite simple using the vector cross product.

The cross product of two vectors $u$ and $v$ (noted $u \times v$, or sometimes $u \wedge v$) is a vector perpendicular to $u$ and $v$, of length $||u \times v|| = ||u|| \cdot ||v|| sin(\theta)$, with $\theta$ the angle between $u$ and $v$. The direction of the vector will depend on the order of the multiplication: $u \times v$ is the opposite of $v \times u$ (the two directions perpendicular to the plane).

If you are not familiar with the cross product, I invite you to read about it and get comfortable with it. Normals will then seem simple.

Flat shading normals

If you have a triangle $ABC$, $AB \times AC$ is a vector perpendicular to the triangle and with a length proportional to its area. Since the normal is the unit vector perpendicular to the triangle's plane, you can get the normal with:

$N = \dfrac{AB \times AC}{||AB \times AC||}$

In code, this would look like n = normalize(cross(b-a, c-a)) for example. Just apply this over all your faces and you will have your normals per face.

For each triangle ABC
    n := normalize(cross(B-A, C-A))
    A.n := n
    B.n := n
    C.n := n

Note that this assumes vertices are not shared between triangles. I am not familiar with the Kinect API; it's quite possible that they are shared, in which case you would have to duplicate them, or move on to the next solution:

Smooth shading normals

After lighting with normals computed as above, you will notice that triangle edges are apparent. If this in not desirable, you can compute smooth normals instead, by taking into account all the faces that share a same vertex.

The idea is that if a same vertex is shared by three triangles $T1$, $T2$ and $T3$ for example, the normal $N$ will be the average of $N1$, $N2$ and $N3$. Moreover, if $T1$ is a big triangle and $T2$ is a tiny one, you probably want $N$ to be more influenced by $N1$ than by $N2$.

Remember how the cross product is proportional to the area? If you add up the cross products then normalize the sum, it will do exactly the weighted sum we want. So the algorithm becomes:

For each vertex
    vertex.n := (0, 0, 0)

For each triangle ABC
    // compute the cross product and add it to each vertex
    p := cross(B-A, C-A)
    A.n += p
    B.n += p
    C.n += p

For each vertex
    vertex.n := normalize(vertex.n)

This technique is explained in longer detail this article by Iñigo Quilez: clever normalization of a mesh.

For more on normals, see also:

How to Calculate Surface Normals for Generated Geomtery


Shear(?) matrix projection along Z (depth) axis I'm creating a primarily 2d game (platformer perspective) with 2d background parallax layers. The parallax effect falls out of the fact that I'm positioning a bunch of layers in 3d and using a perspective camera. I'm using Unity which has a left-handed coordinate system with camera pointing down positive Z (positive X rightwards, positive Y is upwards).
I'd like to occasionally apply an effect that "artificially raises up" the parallax layers in the background. So that the layers further into the background increase their Y position as their Z position increases. Hopefully this diagram makes it clear:

How would I construct a 4x4 matrix that I could multiply by my usual perspective matrix in order to produce this effect? (Is it calling shearing?) I thiiink (but I'm not 100% sure without seeing it) that I want the Y change of each layer in screen space to be proportional to 1/z.
(Note: I realise I could fake this by moving the background layers directly but I'd like to avoid that if possible!)

I belive what you are looking for is a scale Matrix, or actually it will end upp with as a shear matrix for you.
Usually they look like this
Sx  0   0   0
0   Sy  0   0
0   0  Sz   0
0   0  0    1

If you have no scaling, Sx, Sy, Sz represent the scaling in corresponding dimension.
So put the to 1 for no scaling.
But now you want to scale the height depending on the z value. So we want to modify it abit. If you would put Sx, Sy, Sz to the value one. And the put a higher value than one on Syz, then it would scale the height (Y), the futher away you are.
Sx  0    0   0
0   Sy  Syz  0
0   0   Sz   0
0   0   0    1


Volumetric Obscurance : line integrals I would like to compute volumetric obscurance with line integrals like described in this paper.

For a 2D sample, it is possible to know the "depth in sphere". But I don't understand how they compute the occluded (in red on the following image) part and the unnoccluded (in yellow) part of the line in the sphere.

I think that they say that given the depth read through the sample and the "depth in sphere", we should be able to deduce each part of the lines.

This is based on the formula for a sphere that expresses its surface $z$ as a function of $x, y$:
$$z(x, y) = \pm\sqrt{r^2 - x^2 - y^2}$$

So, if $x, y$ is the vector from the sphere's center to your sample point, the total length of the line segment (both occluded and unoccluded parts) is the distance between the top and bottom surfaces, which is $L = 2\sqrt{r^2 - x^2 - y^2}$. (This formula appears on slide 51 of that paper, BTW, although they've apparently scaled so that the radius is 1.)

Then, if you know the depth of the sample point relative to the sphere's center (call it $\Delta z$), you can calculate the fraction of the line that's occluded as
$$\frac{\tfrac{1}{2}L - \Delta z}{L}$$
or equivalently
$$\frac{1}{2} - \frac{\Delta z}{L}$$


Do I need to rebind uniforms or attributes when changing shader programs? Rendering the scene usually involves more than one shader program which, in my case, all use the same attributes and share at least some of the uniforms. To have them working properly, I currently play safe, meaning I rebind the attributes and get the appropriate uniform locations every time I switch between shader programs. So basically multiple times in every frame, which is probably not the best approach.

So, is it (in general) necessary to rebind attributes and uniforms after switching shader programs? And why?

If so, is there a way to do this once at start of the program and never have to touch them again (except for setting the uniform values)?

You don't need to rebind the attributes, so long as you ensure that their location stays the same in both shaders. (Usually using the layout(location = X) syntax in GLSL, but can also be done with glBindAttribLocation if former is not available.)

Uniforms, however, are part of the Shader Object state, and so will need to be set at least once for every shader. One way to minimize the cost of this is to use a Uniform Buffer Object, which will contain all your uniform values and can then be bound to a shader with a single call.


GL_SAMPLE_ALPHA_TO_COVERAGE and depth testing I am currently looking into alternatives to my current, somewhat slow but exact, OIT approach. Specifically, GL_SAMPLE_ALPHA_TO_COVERAGE to convert the alpha value into a per-sample mask.

// draw opaque objects
glEnable(GL_SAMPLE_ALPHA_TO_COVERAGE);
glEnable(GL_SAMPLE_ALPHA_TO_ONE);
glDisable(GL_CULL_FACE);
// draw translucent objects

However, if multiple alpha-blended faces are behind one another (such as the back faces which I would want to be visible) I am only seeing the front most one. Is depth testing not done per sample? Can I force it to be done per sample? According to https://www.khronos.org/opengl/wiki/Depth_Test it should already be per sample.

Multisampling is enabled and works fine for everything else (my OIT shaders in fact do sorting per sample which might be a reason for the slowness). The FBO being drawn into is provided by Qt.

(Promoting comment to answer)
Do the faces have the same alpha value? That might generate the same coverage mask for the surfaces and, hence, only the closest one would be seen.


Is there a term to refer to a cube-break-down process? The image shown below illustrate the process to break a cube down into 4 small equal cubes.

Is there a term to refer to this process, something like cube-break-down?

Why would I like to know the term?

Because I am trying to perform this process by coding, and I am wondering if there existing code base to do this.


Non-uniform rational B-spline (NURBS) Basics I'm trying to understand NURBS curves (surfaces later!) but I have some trouble understanding the very basics of its inner workings. Could someone please explain a few things to me? As I come from Bezier curves, a comparison between these two would be especially useful.

The "rational basis function" looks a tiny bit like Bernstein polynomial of (rational) Bezier curve. Does the parameter u also go from 0 to 1?
How to "add detail" to a curve? I mean, with Beziers if I needed to describe a more complicated shape I would simply "stitch" several Beziers together. Or less often, increase the degree. I understand I can increase the degree of NURBS as well and put several NURBS curves side-by-side, but is this how it should be done?
The Wikipedia article, at least to me, seems not very clear about this "knot vector". What is it anyway?

B-Splines and Beziers are parallel inventions of more or less the same thing. Where Beziers try to start from the idea of fitting tangents. B-Splines start with the idea of basis functions. NURB Splines (or the rational part in fact) are just generalizations of B-Splines so you can describe accurate conic sections*, as they are of special interest in engineering.

First let us start with a simple NURB Spline terminology. The rationale of these curves is a bit different than for Beziers. First there is the concept of a span. A span would roughly be equivalent to a whole Bezier spline except in nurbs you can have any number of spans.

Image 1: One cubic NURBS span. This is a bit atypical in formulation

Each span is formed by the curve degree + 1  control points**. Each curve can consist of any number of points. Each consecutive span reuses the points form previous span by dropping one point and taking one point more in the list. So making more complex curves is as easy as just appending more points to the curve.

NOTE: The images curves are a bit atypically parametrized, ill explain what this means in the next section. When i take the concept of knots up. This is just a easier way to explain how the curves glue together.

Image 2: 2 cubic spans after each other, each span uses 4 points. together they form one curve. They share most points with each other.

By now we have probably answered the 2 question about adding complexity. But I would like to add that this scheme ensures better continuity than a bezier curve. Additionally you can make the point array that forms the hull cyclic. Forming a closed curve.

Image 3: A closed cubic NURBS surface has as many spans as it has points. Each color is one span.

Parametrization

Up until this point one could just say that the stringing together the spans is a trick just as "sewing" the Bezier curves. But there is a difference. The curve is parametrisized along its length. So the curves are not separate they do not interpolate form 0 to 1 on each span like Beziers do. Instead the underlying curve has a cusomizable parameter range. The parameter is stored in something called a knot, and each knot can have a arbitrary increasing value in the sequence. So you can parametrize the entire curves u range to 0 - 1 or 0 to 12. The parametrization also does not have to be uniform.

This parametrization changes how the curve is shaped. Why would this be useful? Well you can adjust the tension along the curve for one. Or you could encode the length of the curve into to the U parameter. One peculiar use is to make the NURBS curve act like a Bezier curve either fully or just partially (bezier like in the ends but not in the middle for example).

Image 4: Same points different knot sequences. The green NURBS curve corresponds to a Bezier curve that has a parameter range of 0-2 instead of 0-1

Ok so what are the knots? They are simply the ranges of the basis functions. Since the cubic b-spline with 4 points has 4 interpolating functions it needs 8 knots. Only areas where 3 functions overlap and sum up to 1.0 can a line be drawn.

Image 5: 2 different basis functions, a bezier like and a uniform segment parametrisation, spread to 0-1 range.

And now we have mostly described the answer to question 1. The range is not defined you can stretch the basis functions as you see fit. And finally the knot vector simply produces the parameter ranges for the basis functions. Theres still one more thing that governs the shape of the curve and that is the weight vector. But that another story to be told elsewhere.

* This rational in this case means that a NURBS curve does not have to be a polynomial, as you cant describe a circle with polynomials.

** One can define other types of points.


Interlacing vertex buffer data with extra efficiency I am making a 2d game in opengl es 2.0

Inside are tons of rectangles defined by 4 points and one 4 component color.

I am using vertex buffer objects, and I have heard that it is efficent to interlace the data.

So like traditionally you would do

Corner.x
Corner.y
Corner.z
Corner.rgba
(repeat for each corner)

However in my situations two assumptions can be made that can probably make things faster

1. All the rectangles z values are 0
2. All corners of the rectangle have the same color.

Is it possible, and what would it look like to have a buffer object structured like this.

Corner.xy
Corner.xy
Corner.xy
Corner.xy
Color.rgba?

Is it even possible to have opengl assume that the Z is always 0? Is it possible to reuse the color like to hat?

Is it even possible to have opengl assume that the Z is always 0?

Yes it is. Just set the component count to 2 in glVertexAttribPointer and the other 2 components (z and w) will be auto filled with 0 and 1 resp.

  Is it possible to reuse the color like to hat?

No it is not. Opengl (and most other graphics apis) require that each vertex is referenced by only a single index.


Why Diffuse Light use max(N · H, 0) instead of just letting it be negative? In Cg tuts, Diffuse Section

  Surfaces that face away from the light will produce negative dot-product values, so the max(N · L, 0) in the equation ensures that these surfaces show no diffuse lighting.

My question is why not just leave it be negative? IMO, both the negative and zero will make the color be black.

If lighting with multiple lights, they add together to make the final lighting value. A negative light would darken other lights, which is incorrect.  This is also true if you add in specular reflections, emissive lighting, or other sources of lighting. The negative lighting from one source of lighting would darken some other source of non negative lighting.


Plotting points along a 3D line segment I'm trying to plot some points along a 3D line segment and could really use some help. In 2D, I've found success getting the angle of the line, the sine & cosine of the angle, and then adding the cosine & sine to the starting point of the line's x & y respectively.
In 3D I think I'm getting the angle correctly and thus x & y using cosine and sine, but I'm not sure what to do for z. Can anyone please advise on how this is derived? Thank you!
def GetAngle(line):
    fCrossX = line.p1.y * line.p2.z - line.p1.z * line.p2.y
    fCrossY = line.p1.z * line.p2.x - line.p1.x * line.p2.z
    fCrossZ = line.p1.x * line.p2.y - line.p1.y * line.p2.x
    fCross = math.sqrt(fCrossX * fCrossX +
        fCrossY * fCrossY + fCrossZ * fCrossZ)
    fDot = line.p1.x * line.p2.x + line.p1.y * line.p2.y + line.p1.z + line.p2.z
    return math.atan2(fCross, fDot)

v1 = Vector(100,0,0)
v2 = Vector(500,0,0)
line = Line(v1,v2)
angle = GetAngle(line)
sin = math.sin(angle)
cos = math.cos(angle)
points = list()
for i in range(int(line.length)):
    x = line.p1.x+(cos*i)
    y = line.p1.y+(sin*i)
    # what do I do for z???
    pt = Vector(x,y)
    points.append(pt)

You shouldn’t need to use any trigonometry here at all. If you get the vector from v1 to v2 and divide it by the number of points you want along the line, each subsequent point is v1 + (that vector) × (the index of the point). Works in any number of dimensions, faster than the trigonometric approach.
v1 = Vector(100, 200, 300)
v2 = Vector(500, 400, 300)
steps = 25 # or however many subdivisions you want: int(length(Line(v1, v2))) if you want an equivalent to the original
stepAmount = Vector((v2.x - v1.x) / steps, (v2.y - v1.y) / steps, (v2.z - v1.z) / steps)
points = list()
for i in range(steps + 1)
    x = v1.x + stepAmount.x * i
    y = v1.y + stepAmount.y * i
    z = v1.z + stepAmount.z * i
    points.append(Vector(x, y, z))


Is there a way to resize an image without creating new colors in the output image? I have been working with a file from the USGS that is a heightmap of the Moon. I resized it to 65536 x 32768 px for use as a displacement map in Blender, and turned it into tiles. Then I created tiles from the original image and noticed that they were about a fifth of the file size of the ones from the resized image, even though they have twice the pixels. Image Magick had remapped the image using many tones of gray that weren't present in the original. Presumably this is why the files are now so much bigger.
Original histogram:

Resized histogram:

Is there any way I can resize this image without the process creating new gray tones?

A nearest-neighbor resize will do what you ask but I am not certain it will give you much smaller tiles.

It is quite likely that the tiles made from the original image are smaller simply because they have less high frequency content and thus compress far better.

You can look at it the other way around: there are more terrain features in the tiles of the resized image so it takes more space to store them. That's a very gross simplification of the truth, of course. And it assumes a reasonably efficient compression algorithm.


What is the best practice for data in a VBO? To start, this is my VBO Data Struct:

//-----------------------------------------------------------------
struct Vertex
//-----------------------------------------------------------------
{
    //-----------------------------------------------------------------
    Vertex()
    : x(0), y(0), z(0),
      s(0), t(0)
    //-----------------------------------------------------------------
    {}

    //-----------------------------------------------------------------
    Vertex(Vertex& rhs)
    : x(rhs.x), y(rhs.y), z(rhs.z),
      s(rhs.s), t(rhs.t)
    //-----------------------------------------------------------------
    {}

    // Position
    float x;
    float y;
    float z;

    // Texture Coords
    float s;
    float t;
};

I am reading a lot of tutorials and all seem to submit the color of their polygons with the VBO to the GPU. However, if you ever change the color, you need resubmit the data to the GPU. Why is that so common, when you can bind the color via an uniform? Isn't that heaps and bounds faster? One could argue that you need to bind the color then for each Polygon or at least Sprite or Model, depending on what you are doing, if you use the same shader for each of them, which is common practice.

So I am confused how to design my (learning) 2D Game from the ground up. Should I put the 4 floats (r,g,b,a) into the VBO and add it to the layout or simply keep changing it via glUniform4fv? Is there a best practice for this?

If the color needs to vary per vertex, it will need to be an attribute in the vertex buffer. If it's acceptable to have the color be constant per draw call, then by all means make it a uniform.

By the way, editing a vertex buffer and resubmitting data to the GPU, while certainly slower than updating a uniform, doesn't have to be a huge inefficiency. By using GL_DYNAMIC_DRAW when you set up the buffer, combined with using glMapBufferRange and GL_MAP_INVALIDATE_BUFFER_BIT to update it, the driver can manage the buffer memory and transfers efficiently. That's how you would typically do a particle system, for instance, with the particles' positions, colors, etc. changing every frame. Furthermore, if only one or two vertex attributes need to change, you can split those off into their own buffer to decrease the total data updated.


How does a computer upscale 1024x768 resolution to 1920x1080? Without there being black bars I mean. 1080 isn't any multiple of 768 so is there some sort of data loss?

In essence a image is  a group of point samples (read A pixel is not a little square3). When you transform or scale the image you need to resample it. So what you do, theoretically, is take the point samples and convert them into a continuous function. Then you sample that continuous function and reconstruct a signal. So, there are two or three different phases here.

Converting the samples to a continuous function, (function reconstruction).
(Transforming)
Re-sampling the signal

Note that none of these steps have a fixed form. In practice, when optimized, it's impossible to tell that there are steps. The transformation does not really have to be simple it could be mapping the shape into a spiral etc.

Image 1: A 1-D signal reconstructed by different filters.

In practice, quite a bit of lore is known about signal reconstruction in the signal processing field. Designing these filters and choosing the right one is a art form on its own. But, in essence, the choice of filter is a tradeoff between blurring and ringing. Of course the algorithm also has other qualities such as how many instructions it takes to implement and how fast and how much memory it needs etc. Which can be very important in realtime or embedded applications.

Image 2: Overview of entire process.


algorithm that openGL uses to construct 3D surface from vertices a 3D model is a collection of points, and basically a 3D software has algorithm to find out how those points connect to make surfaces.
I'm searching for what those algorithms are called, that can use a set of points and properly fill in the dots to create the accurate 3D model from the set of points to create the 3D object intended.
Please mention the name of the algorithm if it's a general approach, if it's file specific then please assume for .obj files. Any links or references would also be highly appreciated.

Most 3D models are not just collections of points; they’re also the information about how those points are connected. Those are polygon meshes. What you’re describing sounds like a point cloud, which is what processes like 3D scanning often produce. That article has a “conversion to 3D surfaces” section; if you search for “point cloud meshing” elsewhere you should also find a bunch of information about ways to do this. One new-ish technique worth looking at, which is more robust than a lot of older approaches to the problem: Fast Winding Numbers for Soups and Clouds.
For OBJ files in particular, polygons are specified with a list of vertices using the f command as described here.


SDF rendering for large world There are lots of cool things you can do with SDFs to render implicit surfaces. Most of the examples I've seen have been “all in the shader”, where the entire distance function is encoded in the shader source code.

Examples:

https://www.iquilezles.org/www/articles/distfunctions2d/distfunctions2d.htm
https://www.shadertoy.com/view/3lsSzf

The problem is that all of those examples assume the shader has total knowledge of the entire world. It evaluates a single distance function, which is complex, but complete.

I want to make a large world (too large to fit in a shader, certainly), and am looking for approaches to render it with the SDF approach.

Does anyone have experience with this? It is 2D-only, if it helps.

The stumbling block seems to be: Since the rendering happens in the fragment shader, I somehow have to transfer “game world” information into that shader. But there do not seem to be good ways to send bulk data to the fragment shader.

Some ideas I have had:

Write one generic shader that can draw, say, a combination of 500 SDFs.

The input to the shader (maybe a UBO?) would contain an encoded version of a piece of the world, with commands like “put a circle at (x,y,radius), do a union with the next object, …” to build the total SDF for that piece of the world.
On the CPU side, I'd have to split the world into chunks that could be rendered by that shader, and populate the input data appropriately for each draw call.
So for 2D, I might do a tile-based render of the screen, where each tile has “small” amount of data, enough to be handled in the shader.

Generate shaders on-the-fly, depending on the part of the world I want to render.

Here, the shader code would look a lot like the “everything baked in” shaders, but I'd just be generating the code on the CPU.
This approach seems bad though, since compiling shaders is a pretty heavy process, in my experience. I did a test with an SDF that was the union of a few thousand circles, and it took 30sec+ to compile.

Any other ideas?

You tile idea seems to be a good one to cut down on local complexity. You can't reasonably evaluate every primitive everywhere. Not if you want good performance anyway. So you need a simplified version for each tile, where you remove elements which have insignificant contribution. This is an interesting problem of its own. A simple version of this would be to:

Assign each primitive "usefulness bounds" beyond which it is considered 0. What exactly those are will depend on the curves you're using to map distance to world height. To make your life easier, pick something which actually goes to 0 over a finite distance. Or alter your favorite curve so it does.
For each tile, trim out primitives for which the bounds don't intersect the tile. You can do this with a simple loop to begin with.
Further simplify the operation tree. This is a little vague because you did not provide details on the kind of operations which will be in your function. But essentially, you want to do the like of replacing $x \times 0$ by $0$ and $x + 0$ by $x$, until you can't anymore. The zeroes come from the previous step where you removed some primitives.

You may need spatial hierarchies if you want this to scale to a world with millions of primitives. This can quickly become a complex topic so I'll leave it at that for now. Ask again when you get there :-)

For your more immediate problem, I suggest using the CPU to rewrite your SDF using reverse polish notation. This will allow you to write your shader as a simple interpreter, with two large buffers as inputs:

An integer buffer with commands (union, add, multiply, point primitive, segment primitive, etc).
A float buffer with operands and transforms for the primitives.

The shader is then just a loop over commands which consumes operands as needed. You'll need different command/operand buffers for every tile, obviously. Depending on the typical nature of your SDF, you may find it more efficient to have your primitives always perform union and have an explicit push command for the rare exceptions. There are a lot of ways to go about this really.

You should take care to make sure the resulting SDF can be evaluated with a reasonably small stack as this will be fixed in your shader. This is another problem by itself, perhaps well suited for the computer science SE, but probably not too difficult to solve on your own either.

I'm sorry if I have not provided enough OpenGL specific details but GPUs are not my specialty.


Surface fairing, implementation detail I'm reading through Polygon Mesh Processing, section 4.3 (Surface fairing).

The reference I mentioned defined defines "Surface Fairing" as

  The goal of surface fairing is to compute shapes that are as smooth as possible... in general fair surface should follow the principle of simplest shapes, the surfaces should be free by any unnecessary details or oscillation".

It continues...

  This can be modeled by a suitable energy that penalizes unaesthetic behavior of the surface. A minimization of this fairness energy - subject to user defined constraints - eventually yields to the desired shape.

The energy defined is function of a surface $x : \Omega \to \mathcal{S}$ and the form is

$$
E_M(x) = \int_{\Omega} \sqrt{det(\bf{I})}dudv
$$

where $\bf{I}$ is the first fundamental form, however instead of this the dirichlet energy

$$
\tilde{E}_M(x) = \int_{\Omega} \lVert x_u \rVert^2 + \lVert x_v \rVert^2 dudv
$$

Is the one usually used. Higher order energies are also used, depending on how smooth is the shape we're looking for. Using the Euler-Lagrange equation (in a continuous setting) we end up with the simple PDE

$$
\Delta x = 0,
$$

subject to suitable boundary conditions. Discretazion to meshes is done by means of the Laplace-beltrami operator which I'll define below.

In general it is my understanding that everything boils down to solve a problem of the form

$$
\Delta^k x = 0
$$

where $x = x(u,v)$ is our surface. If we descretize to a mesh the equation above might be rewritten as

$$
L^k x = 0
$$

Where $L^k$ is the $k^{th}$ power of the Laplacian matrix, and $L$ itself is defined using the Discrete Laplace Beltrami operator, which is something like

$$
\Delta x = \frac{1}{2A_T} \sum_{j} (cotg (\alpha_{i,j}) + cotg (\beta_{i,j}))(x_j - x_i)
$$

However despite the fact it might be very simple to understand in principle at least I'm trying to figure how this would be implemented.

This is because to me the angles depends from the mesh vertices and therefore the system isn't actually linear. The other approach I was thinking is to use a gradient flow (diffusion) approach of the form

$$
\frac{\partial x}{\partial t} = -\Delta^k x
$$

the above, in principle can be discretized as follows

$$
x(t + h) = x(t) - h L^k x(t)
$$

And $L$ might be computed on the current known state of the surface $x(t)$, however the above is an explicit integrator which might lead to instability, and solving the issue would be done by solving

$$
(id + h L^k) x(t + h) = x(t)
$$

But in such case there's the problem of $L$ which would depend from the unknown state $x(t+h)$, so the problem I'm having here is actually similar to the first one I exposed...

So the actual question is:

If the mesh $x$ is unknown so is the discretization of $\Delta^k$, how is this operator constructed then?

Update

I believe I've found the original paper, and I quote the bit I find interesting for my question:

From section 5.4.:

  We can however proceed in exactly the same way, as the changes induced in a time step will be small. We simply compute the non-zero coefficients of the matrix:
  $I - \lambda dt K$
  where $K$ represents the matrix of the curvature normals...

So I guess the idea is actually what has been suggested already in one of the comments, we compute the matrix $I - \lambda dt K$ for the current iteration (say $i$) and we keep the very same matrix for a small number of iterations $n$ after such number of iterations we update again the matrix.

However in the paper the autors confirm that the Laplace-Beltrami operator isn't linear and therefore the system isn't in general linear.

Could anyone confirm if my understanding is correct?

Yes, your understanding it correct. The Laplace-Beltrami depends on the current state $x$ and you have to recompute $L$ if $x$ changes. Therefore you cannot write matix of $\Delta$ without knowing the state $x$.

To expand on solving $\frac{\partial x}{\partial t} = -\Delta x$:

Discretization in space turns $\Delta$ to $L$, discretization in time turns $\frac{\partial x}{\partial t}$ to $\frac{x_{n+1} - x_n}{h}$ and decide if we evaluate the right hand side at time $t_n$ or $t_{n+1}$

Option 1 = Forward Euler:
$$
x_{n+1} - x_n = h L_n x_n
$$
where $L_n = L(x_n)$, $L$ is considered as a function of the state $x$. As you have mentioned this method can be unstable if we take too big step $h$. Therefore, we often resort to the second option.

Option 2 = Backward Euler:
$$
x_{n+1} - x_n = h L_{n+1} x_{n+1}
$$
Because $L_{n+1}$ depends on $x_{n+1}$ the above equation is non-linear system, we cannot solve it with a simple linear solve.

The common approach is to Taylor expand $L$ around the point $x_n$, general Taylor expansion around point $x$ is
$$
L(x + \Delta x) = L(x) + \nabla L(x) \cdot \Delta x + \dots
$$
therefore for $x= x_n$ and $\Delta x = x_{n+1} - x_n$ we get
$$
L_{n+1} = L(x_{n+1}) = L(x_n + \Delta x) = L(x_n) + \nabla L(x_n) \cdot \Delta x + \dots
$$

The simplest thing to do is to take just the first term of the Taylor expansion, i.e. we replace $L_{n+1}$ with $L_n$. This way, we end up with a linear problem
$$
x_{n+1} - x_n = h L_{n} x_{n+1}
$$

You can take higher order terms or solve the Backward Euler equation with non-linear solver. However, in your case that is probably unnecessary and the headache of computing the gradient of $L$ is not worth it(I might be wrong here, I do not have hands on experience with solving this equation).


How does GLSL code get “loaded” and “compiled”? I just wanted to find out for sure how GLSL code gets loaded and compiled.

Does the g++ compiler do it?

No, g++/gcc are not involved.

In OpenGL (prior to Vulkan), GLSL code is submitted to the driver as source code strings via a glShaderSource call. The driver is fully responsible for compilation, i.e. the driver must provide a built-in GLSL compiler for that GPU architecture. This would typically happen on startup of an app, or during level loading for a game.

With Vulkan, SPIR-V was introduced, which is a device-independent bytecode for shaders (vaguely based on LLVM IR). Now, GLSL code can be pre-compiled to SPIR-V offline using the shaderc toolchain. The developers would do this ahead of time and ship the SPIR-V bytecode with their app. Then at runtime the SPIR-V gets submitted to the Vulkan driver and compiled the rest of the way to GPU machine code.

It's also possible to compile other languages to SPIR-V; for instance, Microsoft's dxc compiler for HLSL can optionally output SPIR-V, so you can use this to write Vulkan shaders in HLSL. Also, SPIR-V support has been backported to OpenGL 4.6, so you can now also load precompiled SPIR-V shaders in OpenGL.


Where is the shown display image actually stored and accessed? In a modern day desktop or mobile computer, you got the CPU, the RAM, the GPU and it's own video RAM. When a modern OS renderes an image to the screen, where is the actual image data for that screen located? (Talking about a default HDMI output port)

Is it within the memory of the OS Kernel in RAM, fetching the rendered image of an onscreen window from the GPU when preparing the next frame?

Is it within the GPUs VRAM, accessed and manipulated by low level CPU instructions?

Is it within the GPUs VRAM, mapped via DMA to the CPUs adress space?

And also, where can I read up on implementing some of these basic access patterns for a custom written science project OS? A friend of mine managed to get the SVGA mode running in assembly, but I guess one can do more.

For modern GPUs it's commonly stored in VRAM which for discrete GPUs are separate from the CPU memory. This usually means there is a penalty associated with letting the CPU access this as the data typically has to be copied across some bus. Mobile devices tend to use shared RAM for both GPU and CPU.


Blinn Phong rendering I am trying to render Blinn Phong as a BRDF model on a surface plane. I assume that every normal in every pixel is perpendicular with the plane.
I know that to do a rendering we have to use 3d programing API such as OpenGL or WebGL and so on.
My purpose is to render the surface plane without those APIs. The steps that I take are, I create 2 spaces:

image space: (0,0) is at the top left of the image.
camera space: (0,0) is at the middle of the image.

For vector computation, I convert every pixel position of image space to camera space then to world space.

double[] cam=((new double[]{0,0,1}));
double[] lum=((new double[]{0,0,1}));
double[] E=new double[3];
double[] L=new double[3];
double[] H=new double[3];
double D2;
Mat image=new Mat(rows,cols,CvType.CV_8UC3);
System.out.println("Start!!");
System.out.println("ok");
for(int i=0;i<tileHNumber;i++)
{
    for(int j=0;j<tileWNumber;j++)
    {
        for(int s=0;s<192;s++)
        {
            for(int t=0;t<192;t++)
            {
                originS=s+192*i;//ligne
                originT=t+192*j;//colonne
                double[] pos=(Image2Cam(new double[]{originT,originS,0}));
                E=normalize(XY(cam,pos));
                L=normalize(XY(lum, pos);
                double[] le=addXY(L,E);
                H=normalize(le);
                D2=dot(E,E);
                double blue=color(blueColor,pos,E,L,H,D2);
                double green=color(greenColor,pos,E,L,H,D2);
                double red=color(redColor,pos,E,L,H,D2);
                image.put(originS, originT, new byte[]{(byte)(blue),(byte)(green),(byte)(red)});
            }
        }
    }
}
public static double color(double col,double[] pos,double[] E,double[] L,double[] H,double D2)
{
    double value=0;
    double[] N=normalize((new double[]{pos[0],pos[1],1}));
    double diffuse=Math.max(0, dot(N,L));
    double specular=Math.pow(Math.max(0, dot(N,H)),60);
    value=d*diffuse*0.5+0.2*d+specular*255*0.3;
    return value;
}

The 2 images below show my results: the first one shows diffuse image only. The second one shows specular image only represented by the white color.

My question is why the lobe specular is not surrounding the center of the image.

If I'm guessing right, you are drawing a plane which is facing the camera, so your normals need to be 0,0,-1.

change

double[] N=normalize((new double[]{pos[0],pos[1],1}));

to

double[] N=normalize((new double[]{0,0,-1}));


Does PBR incur a performance penalty by design? I am wondering if the various PBR models used in modern game engines for real time are heavier in terms of computation, that more classic approaches to rendering.

Worded differently, is PBR just a different way to organize one's rendering pipeline, or does it add serious complexity (especially in regards to lighting) ?

Depending on the answer, what would be the bottleneck for a low-resource implementation ?

As mentioned in this answer, Physically-Based Rendering isn't a set number of things. It's a concept. It's akin to saying something is 'Environmentally Friendly'. There are many different techniques to be environmentally friendly and someone can implement those techniques to varying degrees. The same is for PBR.

In the end, Physically Based Rendering is just that, based on Physics in some way, shape, or form. One of the most basic 'physical' rendering techniques I can think of is using Lambert shading. While you may say, "Well that doesn't take into account specular light, or etc, etc. etc.", I agree. Lambert shading is a crude representation of the physics of light. But, pedantically, it is PBR, because it is based off the physical observation that light intensity decreases with viewing angle.

A renderer can implement as many, or as few PBR techniques as they choose. Some can be very computationally expensive, others, quite cheap. They can also choose to enable or disable certain techniques for different targets. For example, UE4 has a different set of techniques for mobile games, than for high end PC games.

Now, that all said, when today's modern renderers talk about being "Physically-based", they are generally referring to using a modern BRDF, generally some form of Cook-Torrance, and physically plausible inputs. Inputs being: materials with physically-plausible color values and lights with physically based intensity values.

Using a modern BRDF can be very computationally expensive, but, there are many approximations that can make it more friendly.

Using physically-plausible inputs is generally not that difficult to implement on the renderer side, rather it's more difficult for artists. IE. teaching them what the different material values mean, and what are physically plausible values. This can be a huge task, depending on your artists, as seen in the presentations here, here, and here, for example. In my opinion, this is the hardest task in switching a studio.

Edit:

One last issue, as brought up by @NathanReed in the comments:

Implementing a modern BRDF in the shader doesn't necessarily immediately make the rendering better. In many cases, it can make other parts of the pipeline a glaring issue. Below are some additional things that are important in getting the most out of a modern BRDF:

HDR lights and doing all rendering calculations in HDR
Gamma correct rendering
Indirect lighting of some kind (skybox lighting, static environment probes, screen space reflections, etc)

This is huge for metals. Metals have no diffuse reflectance, so most of the surface of the metal will be black if you don't have indirect reflections
This is also one of the more difficult things to get real-time, especially for dynamic scenes.


Relationship between roughness and BRDF In a BRDF $f_r$, assume Cook-Torrance model, we have a microfacets distribution $D$. This distribution essentially models the rougheness of the material as far as I've understood. Given a direction $v$ it models the amount of microfacets having that orientation.

Now say we have a material, for each point $p$ in theory we have a 3D function that tell us the orientation in a given direction.

This explantion (not detailed) makes sense to me, however when I read about the roughness map what I see is that it's a kind of texture where each pixel has values between 0 and 1 (from black to white) I do struggle to understand the relationship between this map and the microfacets distribution.

Could anyone clarify?

Most normal distribution functions (NDFs) are parametrized by some variable (tipically $m$ or $\alpha$) that determines the "roughness" or "spikiness" of the NDF (this is often meant to be the rms slope of the surface).

Here is an example of how a roughness parameter effects an NDF: https://www.desmos.com/calculator/kcevxp3wm0

we can think of the NDF as a function of the direction as well as of the roughness value (as in $D(\theta,\phi,m)$. Here, $\phi$ and $\theta$ were used to represent spherical coordinates for the microsurface normal direction).

A roughness map can also be thought of as a function: one that returns a value for $m$ for every point $p$ on a surface. Hence, $m(p)$.

Since $m$ is an argument of the NDF we can write the previous expression like this: $D(\theta,\phi,m(p))$

And, because $m(p)$ may be different for every $p$ but stays fixed when we only vary $\phi$ and $\theta$, we effectively get a whole different NDF on every point on the surface.


How to obtain a parametric equation out of a part designed in Autodesk Inventor? Whenever you make a part in inventor, the software calculates de properties of the whole body given a constant density. Then, automatically,  it shows the inertial tensor. As you can recall from math, the inertia equation is $I_x= \rho\int(y^2+z^2)dV$
for the x axis. So, clearing rho, how can I get the right hand of the equation that is been solved in the underlying algorithm without doing any hand calculations?

I guess your question is how to compute the inertia tensor automatically. A parametric equation may not be always possible, but we could write a program to avoid hand calculations as much as possible.
The inertia tensor $\mathbf{I}$ is defined as an integral over the object domain $\Omega$ (see Inertia tensor):
$$
\newcommand{\V}[1]{\mathbf{#1}}
\begin{align}
\V{I} &= \int_\Omega \rho (\V{r}\cdot\V{r} \delta - \V{r}\V{r}^T)\,dV
\end{align}
$$
where $\V{r}$ is the position relative to the center of mass (COM),$\rho$ is the density, and $\delta$ is the identity matrix.
Note that $\text{tr}(\V{r}\V{r}^T) = \V{r}\cdot \V{r}$, the above definition  can be rearranged as
$$
\begin{align}
\V{T} &= \V{r}\V{r}^T \\
\V{I} &= \int_\Omega \rho (\text{tr} \V{T} \delta - \V{T})\,dV
\end{align}
$$
Analytic object
For analytic objects you can substitute their equations into the integral and do symbolic integration. This step can be automated with any symbolic computing software.
Object in tetrahedral mesh
We assume the object is simple and has a closed boundary.
An analytic solution is not possible in this case because we do not have an explicit equation for the object. But we could compute the inertia tensor numerically.
The only integral we have to compute is
$$
\begin{align}
\V{C} &=
\int_\Omega \rho \V{r}\V{r}^T\,dV
\end{align}
$$
To compute this integral, this paper could be helpful.
The general idea is to compute $\V{C}_t$ for each tetrahedron $t$, and sum $\V{C}_t$ to get $\V{C}$.
We first compute the integral for a canonical tetrahedron $t0$ whose vertices are
$(0,0,0)$, $(1,0,0)$, $(0,1,0)$, and $(0,0,1)$.
In maple, this canonical integral $\V{C}_{t0}$ is (assume $\rho$ constant)
r:=<x,y,z>;
                                                    [x]
                                                    [ ]
                                               r := [y]
                                                    [ ]
                                                    [z]

with(LinearAlgebra):
rrt:=r.Transpose(r);
                                             [ 2               ]
                                             [x      x y    x z]
                                             [                 ]
                                      rrt := [        2        ]
                                             [x y    y      y z]
                                             [                 ]
                                             [               2 ]
                                             [x z    y z    z  ]

seq(seq(int(int(int(rrt[i,j],x=0..1-y-z),y=0..1-z),z=0..1),i=1..3),j=1..3);
                      1/60, 1/120, 1/120, 1/120, 1/60, 1/120, 1/120, 1/120, 1/60

Then the integral $\V{C}_t$ can be computed by a change of variable.
Assume the coordinates of a world space tetrahedron can be represented as $\V{r}=\V{F}\V{r}_0 + \V{x}$, where $\V{F}$ and $\V{x}$ are constant.
We have
$$
\begin{align}
\V{C}_t &= \rho \int_{\Omega_t} \V{r}\V{r}^T\,dV \\
&= \rho \int_{\Omega_{t0}} (\V{F}\V{r}_0 + \V{x})(\V{F}\V{r}_0+\V{x})^T \det \V{F} \,dV_0 \\
&= \rho \int_{\Omega_{t0}} (\V{F}\V{r}_0\V{r}_0^T\V{F}^T + \V{x}\V{x}^T + \V{F}\V{r}_0\V{x}^T + \V{x}\V{r}_0^T\V{F}^T) \det \V{F} \,dV_0 \\
&= \det(\V{F}) \V{F}\V{C}_{t0}\V{F}^T + \det(\V{F}) m \V{x}\V{x}^T + \det(\V{F}) m \V{\bar{r}}\V{x}^T + \det(\V{F}) m \V{x}\V{\bar{r}}^T
\end{align}
$$
where $\V{\bar{r}}$ is the COM and $m$ is the mass.
Objects in other form
I did not use Inventor before but I suspect the part produced by it may have other formats, such as Bezier surface. For now I would suggest tetrahedralize the object and apply the above method.


Why do I need to inverse the orientation matrix of a camera to be able to translate it in the direction it is facing? I've been going over some OpenGL learning resources and one of them is this article: Modern OpenGL 04 - Cameras, Vectors & Input in which the author presents an approach to building a Camera class that can be used to derive the view and projection matrices.

In the accompanying source code there is a function that returns the orientation of the camera as a transform matrix (permalink to source code) which is implemented like this:

glm::mat4 Camera::orientation() const {
    glm::mat4 orientation;
    orientation = glm::rotate(orientation, glm::radians(_verticalAngle), glm::vec3(1,0,0));
    orientation = glm::rotate(orientation, glm::radians(_horizontalAngle), glm::vec3(0,1,0));
    return orientation;
}

If I'd like to translate the camera relative to the direction it is facing then I would assume that I could multiply the orientation matrix by the translation vector to get a new vector that does the translation in the direction the camera is facing.

However, this does not yield the result I was expecting. After looking at how the forward, up and right axis are derived (permalink to source code):

glm::vec3 Camera::forward() const {
    glm::vec4 forward = glm::inverse(orientation()) * glm::vec4(0,0,-1,1);
    return glm::vec3(forward);
}

glm::vec3 Camera::right() const {
    glm::vec4 right = glm::inverse(orientation()) * glm::vec4(1,0,0,1);
    return glm::vec3(right);
}

glm::vec3 Camera::up() const {
    glm::vec4 up = glm::inverse(orientation()) * glm::vec4(0,1,0,1);
    return glm::vec3(up);
}

I can see that I need to multiply the translation vector with the inverse of the orientation matrix to get the expected result but I don't understand why. Isn't the orientation (as used in this article) a rotation matrix? If that's the case then why do I need to inverse it?

People always forget that there is no "camera" in OpenGL. In order to simulate a camera you have to move the whole world inversely. So if you want ur camera looking 30 degrees downward, you move the whole world 30 degrees upwards. If you want your camera moved to the left, you move the whole world right. That's why you will notice the "-" sign in the translation vector where he used the variable position.

Hence the reason there is no separate "view" matrix in openGL, it's modelview matrix combined. That's the reason for taking the inverse. Since a view matrix represent's the camera's forward, up and side, you have to take the inverse since the orientation is defined for the world not the camera.


How can I generate procedural noise on a sphere? I would like to generate procedural noise on the surface of a sphere (e.g. to procedurally generate planets or marble-textured balls). Of course, I could just take a standard noise algorithm and map it onto the sphere, but this has all the issues of projecting a plane onto a sphere, like distortions on the poles or elsewhere.

I suppose I could generate volume noise and "cut out" the sphere, but this seems unnecessarily inefficient - and if the noise has some grid-based artefacts, these would still not appear uniformly on the sphere. Furthermore, at least in the case of Simplex noise, cutting 2D slices out of 3D noise generally looks different than generating 2D noise right away.

Is there any way to avoid these issues, e.g. by generating noise natively on the sphere? The noise should have at least the quality of Perlin noise, ideally that of Simplex noise.

I'd consider just going with 3D noise and evaluating it on the surface of the sphere.

For gradient noise which is naturally in the domain of the surface of the sphere, you need a regular pattern of sample points on the surface that have natural connectivity information, with roughly equal area in each cell, so you can interpolate or sum adjacent values. I wonder if something like a Fibonacci grid might work:

I haven't chewed through the math to determine how much work it would be to figure out the indices of and distance to your four neighbors (I don't even know if you end up having four well-defined neighbors in all cases), and I suspect it may be less efficient than simply using 3D noise.

Edit: Someone else has chewed through the math! See this new paper on Spherical Fibonacci Mapping. It seems that it would be straightforward to adapt it to sphere noise.

If you are rendering a sphere, not just evaluating noise on the surface of a sphere, and are fine with tessellating your sphere to the resolution of your noise lattice, you can create a geodesic grid on the surface of the sphere (a subdivided icosahedron, usually):

Each vertex of the sphere can have a randomly generated gradient for gradient noise. To get this information to the pixel shader (unless you want straightforward interpolation like value noise), you may need a technique like this article's wireframe rendering with barycentric coordinates: do unindexed rendering, with each vertex containing the barycentric coordinates of that vertex in the triangle. You can then read from SV_PrimitiveID (or the OpenGL equivalent) in the pixel shader, read the three noise gradients from the vertices based on what triangle you are on, and use whatever noise calculation you like using the interpolated barycentric coordinates.

I think the most difficult part of this method is coming up with a scheme to map your triangle ID to three samples in order to look up the noise values at each vertex.

If you need multiple octaves of noise or noise at a finer resolution than your sphere model, you may be able to do a coarse geodesic grid with vertices and do a few levels of subdivision in the pixel shader. i.e. from the barycentric coordinates, figure out which subdivided triangle you would be in if the mesh was further tessellated, and then figure out what the primitive ID and barycentric coordinates would be for that triangle.


What is the purpose of the projected solid angle $dA^\perp$ term in the definition of Radiance? In the definition of Radiance, which according to PBRT is defined as

  flux per unit solid angle $d\omega$ per unit projected area $dA^\perp$

$$L_i=\frac{d\Phi}{d\omega\ dA^\perp}$$

My original assumption was that the $dA^\perp$ term is used to weaken the irradiance contribution due to the incident angle (which I assume is just Lambert's cosine law). However, in the rendering equation, we've already explicitly apply Lambert's law with the $\cos\theta$ term:
$$\int_\Omega L_i(\omega_{i}) \, f_r(\omega_{i}\rightarrow \omega_{o}) \, \cos\theta_{i} \, d\omega_{i}$$

I assume we arn't applying Lambert's law twice, so what exactly is this $dA^\perp$ term?

Your main idea is more or less correct. The cosine hidden in the projected area measure $dA^\perp = dA\cos(θ)$ compensates the weakening of irradiance due to incident angle (the Lambert's cosine law). This makes radiance independent from the incident angle. My guess is that the main motivation was to make it more practical to work with.

The cosine in the rendering equation comes from the definition of BRDF:

$$
f_{r}\left(\omega_{i}\rightarrow\omega_{o}\right) =\frac{\mathrm{d}L_{o}\left(\omega_{o}\right)}{\mathrm{d}E\left(\omega_{i}\right)} =\frac{\mathrm{d}L_{o}\left(\omega_{o}\right)}{L_{i}\left(\omega_{i}\right)\mathrm{d}\sigma^{\bot}\left(\omega_{i}\right)} =\frac{\mathrm{d}L_{o}\left(\omega_{o}\right)}{L_{i}\left(\omega_{i}\right)\cos\theta_{i}\mathrm{d}\omega_{i}}
$$

Which can be rewritten as

$$
\mathrm{d}L_{o}\left(\omega_{o}\right)
=f_{r}\left(\omega_{i}\rightarrow\omega_{o}\right) L_{i}\left(\omega_{i}\right) \cos\theta_{i} \mathrm{d}\omega_{i}
$$

Which can be integrated over the hemisphere the get the radiance reflected in a given direction $\omega_o$

$$
\int_\Omega f_{r}\left(\omega_{i}\rightarrow\omega_{o}\right) L_{i}\left(\omega_{i}\right) \cos\theta_{i} \mathrm{d}\omega_{i}
$$

Voila, the rendering equation!

Conclusion: Althought both cosines come from the same fundamental priciple, they serve different purposes.


What is 2D BRDF Slice/Lobe? I was recently trying to read some rendering papers and textbooks. The word 2D BRDF slice/lobe is often used while no official definition is provided. I know that BRDF is a 4D function of input direction and output direction. What is the 2D slice/lobe based on that? Are they the same concept? If not, what's the definition of each of them?

A BRDF is a 4D function, but as we cannot really visualize a 4D function in a comprehensible way, if we want to understand its behavior it can be instructive to fix 2 or 3 of the parameters and graph the others. This kind of graph can be called a "slice". (The mental image I have is like: the BRDF is some oddly shaped fruit, and we're taking a knife and making a slice through it, to examine the shape of the cut area.)
In particular, a common way to do this is to fix the incoming light direction; then the BRDF can be graphed as a function of the outgoing light direction, like this:

(image from http://brdflab.sourceforge.net/)
The orange line on the left represents the fixed incoming light direction. This is a polar graph, so the magnitude of outgoing light in each direction is graphed as the radius of the shape.
In this case we can see that most of the light is going out along the specular reflection direction, to the upper right; but some is heading back toward the source as well (retroreflection). These are the "lobes". Lobes are not precisely defined, but intuitively they are the visible features in the shape of the function. Here, we would say there is a "specular lobe" and a "retroreflection lobe". In fact, this BRDF looks like it has two specular lobes, one long skinny one (which represents a tightly concentrated, bright highlight) and another short and fat one (a more spread-out, soft/rough highlight). There appears be a diffuse lobe as well, which shows up as a hemisphere around the origin in this type of graph.
It's also common to simplify the graph even more by constraining the outgoing light directions to be coplanar with the normal and the incoming light, so now you're only graphing the (N, V) angle. That gets you a graph like this:

(image from Physically Based Rendering in Filament)
This is the one that's most likely to be called a "slice". This diagram also shows two "lobes", a specular one and a diffuse-ish one.
Synthetic BRDFs (what we mostly use in graphics) are usually constructed by taking a weighted average of various lobes generated by different models, e.g. a Lambert diffuse lobe with one or more microfacet specular lobes, etc. When looking at measured BRDFs, we could try to identify the "lobes" by fitting some model lobes to the measured data.
To sum up, "slice" is a type of graph that allows us to visualize the features of a BRDF, and "lobe" is the name for the typical kind of features that we see in the BRDF shape, or that we use to construct the BRDF in the first place.


Can I parallelise the scene for a raytraced image, rather than just the pixels? Raytracing an image can be performed in parallel by calculating the colour of different pixels on different machines. However, this still requires each machine to have access to the entire scene to be rendered. Is there a way of also subdividing the scene so that different machines have access to different parts of it but can still calculate their assigned pixels correctly?

I cannot imagine a way in which this could be possible in general. Are there any particular circumstances in which this can be achieved?

Yes, such a thing is possible. There are a few different ways of doing it, but the basic idea is to split up the scene into chunks and assign these chunks to different machines. Each machine then traces rays locally within their assigned chunk.

The tricky bit is of course when rays leave a chunk, in which case machines need to synchronize and exchange rays that cross the local chunk boundaries. A recent paper from this year's EGSR, Distributed Out-of-Core
Stochastic Progressive Photon Mapping, describes how to do this efficiently for Progressive Photon Mapping, although other light transport methods might be feasible.

Although such distributed ray tracing is possible, it is certainly not the most efficient form of parallelization, and non-uniform lighting directly translates into non-uniform distribution of the workload. It is really only beneficial when the size of the scene far exceeds the memory of a single machine.


Issue with size() function in Processing language? I´m new to the Processing language. Trying to do my first sketch I´m having trouble with the dimension of the display window width and height because even though I´ve tried putting a variety of values in the size() function, the size of the window doesn´t change at any time, it always has the same size (100px x 100px). The function fullScreen() has the same results. How could I resolve this problem?

It´s a simple visualization of a "star field". First block, class Star, simply create a star and fill it with a color. Second block is where I´m trying to adjust the size of the display window but it doesn´t work.

class Star {
  float x;
  float y;
  float z;

  Star(){
    // Places the star in random X and Y coordinates
    x = random(0, width);
    y = random(0, height);
  }

  /* Colors the star */
  void update(){
    fill(255);
    noStroke();
    ellipse(x, y, 8, 8);
  }

  void show(){}
  }

// Array that contains all the amount of stars created (100 in this case)
Star[] stars = new Star[100];

void Setup(){
  size(800, 800); // TROUBLE HERE. DOESN´T WORK
  for(int i = 0; i < stars.length; i++){
    stars[i] = new Star();
  }
}

void Draw(){
  background(0); // Colors the background
  for(int i = 0; i < stars.length; i++){
    stars[i].update();
    stars[i].show();
  }
}

The window is 100px, 100px because it is the default size and you are not changing it, as your
size() is never being called because Setup() is not a predefined function.
setup() is the function, lower case s not upper case S. Same with draw().

Below is the working code.

class Star {
  float x;
  float y;
  float z;

  Star(){
    // Places the star in random X and Y coordinates
    x = random(0, width);
    y = random(0, height);
  }

  /* Colors the star */
  void update(){
    fill(255);
    noStroke();
    ellipse(x, y, 8, 8);
  }

  void show(){}
  }

// Array that contains all the amount of stars created (100 in this case)
Star[] stars = new Star[100];

void setup(){
  size(800, 800); // TROUBLE HERE. DOESN´T WORK
  for(int i = 0; i < stars.length; i++){
    stars[i] = new Star();
  }
}

void draw(){
  background(0); // Colors the background
  for(int i = 0; i < stars.length; i++){
    stars[i].update();
    stars[i].show();
  }
}


Iridescence/Thin-film interference integration as a layer I'm trying to integrate the thin-film interference effect described in the paper "Belcour, L., & Barla, P. (2017). A practical extension to microfacet theory for the modeling of varying iridescence." (https://hal.archives-ouvertes.fr/hal-01518344/document) into my material model, which is based on google/Filament's material model (https://google.github.io/filament/Filament.md.html).

However I'm having trouble trying to understand the way to add them together. Let's say I'm shading a pixel, direct lighting goes like this:

vec3 Fr = specularLobe();   // D * F * V
vec3 Fd = diffuseLobe();    // 1 / PI * diffuseColor

return (Fd + Fr) * light.color * light.attenuation * NoL;

And on the other side, indirect lighting goes like this...

vec3 Fd = diffuseIrradiance() * diffuseColor;
vec3 Fr = specularIrradiance() * DFG();

return (Fd + Fr);

Now, if I want to add the thin film/iridescence layer on top of the base material, in what way does it take into account to bottom layer for all the lighting?

I've been looking at this implementation in UE4 (https://polycount.com/discussion/comment/2604578#Comment_2604578), but they seem to input the result of the BRDF calculation directly into the 'Base Color' of the material, which I really don't think it is what I'm looking for, correct me if I'm wrong...

In this other implementation in Unity (https://github.com/Xerxes1138/Iridescence/blob/master/Iridescence.shader) they seem to account for iridescence in both IBL and direct lighting separately, but the layer base is not taken much into account as far as I know.

Finally, in the paper itself, the following statement is made:

  Our extension consists in replacing the classic Fresnel reflectance
  term, F , by a more complex term, R accounting for all inter-reflections
  inside the thin-film layer, including constructive and destructive
  interference effects.

From this I understand that simply by replacing the Fresnel Reflectance with the Airy Reflectance function should do the trick, but again, I'm not sure which would be the appropriate approach to being as physically correct as possible.

(The original supp-code from the paper can be obtained from: https://hal.inria.fr/hal-01518344v2/file/supplemental-code%20%282%29.zip)

Thanks for the help!

Unfortunately, the iridescence model is not made to be applied to a diffuse term. Pascal and I made it for microfacet models only (that is the specular term).

One way to understand how to include it to a game engine might be to look at Unity's HDRP implementation.

In the Lit.hlsl to see how to incorporate the iridescence Fresnel into a specular + diffuse BRDF:

https://github.com/Unity-Technologies/ScriptableRenderPipeline/blob/master/com.unity.render-pipelines.high-definition/Runtime/Material/Lit/Lit.hlsl

PreLightData GetPreLightData(float3 V, PositionInputs posInput, inout BSDFData bsdfData) {

    [...]

    if (bsdfData.iridescenceMask > 0.0)
    {
        bsdfData.fresnel0 = lerp(bsdfData.fresnel0, EvalIridescence(topIor, viewAngle, bsdfData.iridescenceThickness, bsdfData.fresnel0), bsdfData.iridescenceMask);
    }
}

See the implementation of the EvalIridescence in BSDF.hlsl:

https://github.com/Unity-Technologies/ScriptableRenderPipeline/blob/30f1b16c8094ca133c165e56380ebb7ab9e7c0d9/com.unity.render-pipelines.core/ShaderLibrary/BSDF.hlsl

What they are doing is to blend the standard Fresnel term with the iridescence one based on a iridescence mask value.

I hope this is helpfull.


Organic 2d tree generation I'm looking for a code to generate organic 2d trees like thiskind of shapes

Gerard Ferrandez has an example to generate some tree but not organic like the picture shape https://codepen.io/ge1doot/pen/BGWzgW?editors=0010

Any recomendation will be much appreciated.

Many thanks!

It could be generated as a Lindenmayer System (L-system) with some rules for branch widths (plus some fractal noise) and branch bifurcation. They are parameters you can play with.

You can find lots of resources in the website of the Biological Modeling and Visualization research group in the Department of Computer Science at the University of Calgary. Professor Przemyslaw Prusinkiewicz is the researcher who popularized these systems, and he has made his main book The algorithmic beauty of plants available for free.


Moires patterns despite using mipmaps I am working on Silverlight app. I am trying to render floor covered with tiles. Although I am using mip maps I am still getting awful Moire patterns, when trying to render big area.

What I am doing is creating cuboid and then cover it with single tile:

In my pixel shader I am multiplying texture coordinates in order to create tiled floor (otherwise I would get one tile stretched over my cuboid). Pixel shader code:

float Width                 : register(c3);                  //Width of cuboid
float Height                : register(c4);                  //Height of cuboid

texture texTexture;
sampler textureSampler  : register(s0) = sampler_state {
    Texture = (texTexture);
};

struct VsOutput
{
    float4 position : POSITION;
    float3 dirLightPosition : COLOR;
    float2 texCoord : TEXCOORD0;
    float3 normal   : TEXCOORD1;
    float3 view : TEXCOORD2;
};

float4 main(VsOutput IN) : COLOR
{
    float2 texCoord = float2(IN.texCoord.x * Width / 4.0f,
                             IN.texCoord.y * Height / 4.0f); //multiplying texture coordinates

    float4 texColor = tex2D(textureSampler, texCoord);

    return float4(color.r, color.g, color.b, 1.0f);
}

My output is:

What else can I do to prevent from creating Moires patterns on texture?

Two things come to mind:

When generating your smaller mip map levels try to avoid using a simple 2x2 box filter because, though cheap and cheerful, they do a really poor job of removing high frequency information (that exceeds the Nyquist limit) as well as over filtering some of the lower frequency information you need to keep. (Also, as an aside, you need to perform the MIP map generation in linear space. If your source data is sRGB, you thus need to map to and from linear).  FWIW, In Williams' original paper on MIP mapping, he said he used a "box (Fourier) window" to generate the prefiltered levels, which would be a sinc function in image space.
Standard Trilinear filtering will also over- and under-filter parts of the texture. If you don't like the aliasing you may have to a) adjust the bias to increase the filtering (which will increase blurring) and/or b) try using anisotropic filtering.


Determining slopes of lines in perspective projection Suppose I have parallel lines running along the ground (into the horizon). If I have a camera and I know its tilt angle, is there a way to calculate the slopes of the lines as projected to the 2D image plane? That is, the green angles:

However, the distance from the lines are not known. What parameters of the camera are used in the calculation?

You definitely need to know distances and angle of view of the camera.  For example, if you add lines that are slightly further apart, they will be at different angles.  Like a multi-lane highway.


How to convert from glBegin() and glEnd() to VBOs? How to convert from glBegin() and glEnd() to VBOs (Vertex Buffer Objects)?

I found a simple example here:

GLfloat vertices[] = {...}; // 36 of vertex coords
...
// activate and specify pointer to vertex array
glEnableClientState(GL_VERTEX_ARRAY);
glVertexPointer(3, GL_FLOAT, 0, vertices);

// draw a cube
glDrawArrays(GL_TRIANGLES, 0, 36);

// deactivate vertex arrays after drawing
glDisableClientState(GL_VERTEX_ARRAY);

http://www.songho.ca/opengl/gl_vertexarray.html

So if I have code in very "standard" glBegin() and glEnd() fashion like:

glBegin(GL_TRIANGLES);

    for(int j = 0; j < vecf.size(); j++){
        glNormal3f(vecn[vecf.at(j)[2]-1][0], vecn[vecf.at(j)[2]-1][1], vecn[vecf.at(j)[2]-1][2]);
        glVertex3f(vecv[vecf.at(j)[0]-1][0], vecv[vecf.at(j)[0]-1][1], vecv[vecf.at(j)[0]-1][2]);
        glNormal3f(vecn[vecf.at(j)[5]-1][0], vecn[vecf.at(j)[5]-1][1], vecn[vecf.at(j)[5]-1][2]);
        glVertex3f(vecv[vecf.at(j)[3]-1][0], vecv[vecf.at(j)[3]-1][1], vecv[vecf.at(j)[3]-1][2]);
        glNormal3f(vecn[vecf.at(j)[8]-1][0], vecn[vecf.at(j)[8]-1][1], vecn[vecf.at(j)[8]-1][2]);
        glVertex3f(vecv[vecf.at(j)[6]-1][0], vecv[vecf.at(j)[6]-1][1], vecv[vecf.at(j)[6]-1][2]);
    }

glEnd();

Then how do I convert this to VBOs?

Particularly, I already know that I need to supply vertices array and normals array, but I seem to confuse the orders of some vecs, because what's drawn becomes out as distorted.

Back when I did the transition from immediate mode to vertex buffer object (VBO) in my own code base, I have found it easier to proceed with these steps:

From naive immediate mode code...

The original code was something like this:

glBegin(GL_TRIANGLES);
glNormal3f(nx, ny, nz);
glVertex3f(px, py, pz);
// [...] arbitrary long sequence of such function calls.
glEnd();

1. To array based immediate mode code...

First, keep using the immediate mode, but make sure that all values passed to the vertex functions come from an array. This seems to be already the state of the code described in the question.

float vertexData[] = { ... }; // Can be all together, or split by attribute type, as you prefer.
glBegin(GL_TRIANGLES);
for (int i = 0; i < numberOfVertices; ++i)
{
    // The indices are going to depend on the data layout.
    glNormal3f(vertexData[0], vertexData[1], vertexData[2]);
    glVertex3f(vertexData[3], vertexData[4], vertexData[5]);
}
glEnd();

2. To vertex array version...

Second, you can replace the loop by a handful of calls with vertex arrays.

glEnableClientstate(GL_Vertex_Array);
glVertexPointer(3, GL_FLOAT, 6 * sizeof(float), vertexData);
glNormalPointer(GL_FLOAT, 6 * sizeof(float), vertexData + 3);
glDrawArrays(GL_TRIANGLES, 0, numberOfVertices);
glDisableClientState(GL_VERTEX_ARRAY);

3. To vertex buffer version

Finally from the vertex array based version, the transition to VBO is minimal:

// An initialization part will create and populate the vertex buffer:
GLuint vboId = 0;
glGenBuffers(1, &VBOid);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, VBOid);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, 6 * sizeof(float), vertexData, GL_STATIC_DRAW);

// A draw part will bind the vertex buffer and issue the drawcall.
// It's almost the same code as before, except:
// - We bind a VBO
// - We pass (0+offset) as the last parameter for the gl*pointer().
glBindBufferARB(GL_ARRAY_BUFFER_ARB, VBOId);

glEnableClientstate(GL_Vertex_Array);
glVertexPointer(3, GL_FLOAT, 6 * sizeof(float), (void*)(0));
glNormalPointer(GL_FLOAT, 6 * sizeof(float), (void*)(0 + 3));
glDrawArrays(GL_TRIANGLES, 0, numberOfVertices);
glDisableClientState(GL_VERTEX_ARRAY);

And that's it. Both vertex array and VBO allow to work directly with vertices or with vertex index, so be careful about possible confusion there.

P.S.: I have written this answer without double checking the code, so it's probably ridden with typos and not meant for copy-pasting. I invite you to read the documentation, understand the API and refer to other examples when you implement it. At first, VBO can be really annoying to get to work properly.

I have used the same distinction when answering this other question about performance of VBO.


Handling GPU resource management I recently finished the base implementation of a DX12 rederer and I started to think in ways of improving the performance. I think that one problem that my implementation has, is memory fragmentation, as I mostly perform "single" allocations for my resources. During run-time some resources are deleted and recreated which I guess will cause even more memory frag. There are quite a few articles online about writing system memory managers but I've never encountered something like that for video memory and GPU resources.

Could anyone explain the basics of a system like this?
Thanks!

EDIT:
I had this link laying around: Memory Management in Direct3D 12 | Windows Dev Center. That gives some good general information about the topic.

Whenever there's too much churn of things being created and destroyed, regardless of the specifics of the situation, a common way to dealing with this is to make free object pools.

How these work is when you are done using something, instead of actually freeing or destroying it, you put it back into a list or array of "free objects".

When you need a new object of a specific type, you check the object pool to see if there's a free one lying around.  If there isn't one there, you create a new one the "expensive" way and use it.  When you are done with it, you put it in the free pool for use in the future.

A possible issue with free object pools is that you may have a spike in objects for some reason and then you are left with a ton of unused objects sitting in a free pool that will never be needed again.  In situations like this, you can keep track of how long (in number of frames for instance) since an object was last used, and if they get over a certain unused age, free them from the object pool.

PS - make sure and profile your code to find what the ACTUAL performance bottlenecks are before attempting to solve problems.  Optimizing without profiling before (to find the problem) and after (to verify you improved things, and knowing by how much) won't get you very far (: