FXAA vs MSAA on modern hardware looking for suggestion whether it is worth implementing a post process FXAA pass on modern mobile GPU's (mainly looking at apple A Series chip), or should i simply use the MSAA.
Did a basic test with implementing basic FXAA which took eight neighbouring samples and averaged them for result. The performance of this FXAA was even a bit slower than turning on MSAA.
I am using Metal API, if that helps.

Most mobile GPUs use tiled rendering, which can make MSAA surprisingly cheap relative to desktop GPUs. If the app is doing simple forward rendering without filtering/blurring passes, the GPU may be able to do MSAA rasterization and resolve directly on chip (per tile) without ever storing the MSAA samples out to memory. This may make MSAA both faster and less power-hungry than a technique like FXAA that relies on filtering the framebuffer (which necessitates storing it out to memory and reading it back in when sampling).

As for image quality, it's difficult to compare (and is subjective anyway). FXAA might be better at reducing jaggies depending on the MSAA level, but at the cost of over-blurring edges. FXAA can also address shader aliasing, for example aliasing due to specular on a rough normal map and such things, which MSAA won't help with. Ultimately you'll have to make a choice whether the aesthetic benefits (if any) outweigh the increased cost.


Best technique to draw overlapping colored line segments that follow the same route I'm not too versed in computer graphics so my question may be vague.

I'm given a sequence of GPS coordinates which I draw on an iOS map, and they define bus routes. Some of the bus routes happen to share sections of certain streets however, and when multiple routes are being drawn on the map the colored line segments are just being overlapped currently.

I'm currently looking into computationally separating the lines from each other before drawing, but I'm not sure that will get any success. So my backup is: are there any good techniques to draw the lines in a way that somehow does not hide the lower line segments, and retains at least partially their color?

Again as a disclosure, I'm not very knowledgable in this area. I know from Photoshop there are different color modes on the layers that have different effects when overlapped. Would something of that sort help in this situation? Or maybe there's a way to dash the lines such that it alternates each color, but I imagine that would be fairly difficult.

I'm just probing around for different techniques for this problem.

This question should most probably be asked on GD.SE or UX.SE. These sites specialize in how to design the graphics and how to choose the graphics for your purpose. But since you are here basic options are:

Routes dont overlap but are parallel like electronic layouts (image 1, A), Works well unless you have many overlapping things.
Space interleaved lines, differently colored line regions (image 1, B). Such as dashed lines, however many other interleaved shapes are possible.
Spatial solutions, lines on top of each other but laid over in 3d (image 1, C), Works well in isometric drawing.
Different line widths (image 1, D)
User suggested, overlay (image 1, E).
Same color but labels on segments (image 1, F), note not all segments need to be labeled as humans can interpolate values somewhat. This has benefit of being able to be paired with any other effect and works well with selection highlights with info intact.

Image 1: Different overlay methods.

Note: None, except the labeling works well when there are very many different lines as humans capability to see different colors in this kind of context as differing data is not very good (ask UX.SE).

As for what is best depends. None of these are particularity hard to do in code or vector drawing application (but that is out of scope ask GD.SE). Whichever is best for the user or the design is also not within the scope of this software (ask  GD.SE or UX.SE).


live music visualisation Does anyone out there know of any software for developing real time visualisations and analysis of real time sound, say using a mike? I've found a few but none allow the user to build up their own visualisations.

Thanks for any ideas, Jonathan

One tool I've seen people use for this sort of thing is Processing, a Java-based language for real-time artsy stuff that supports graphics and audio. I haven't ever worked with it myself, but apparently it's pretty easy to use it for programming visualizations. Here's an example of a simple visualization someone made; look in the video description for a link to their code—it's quite short. Their example loads an mp3 file, but Processing also supports real-time input from a mic.


Stecklov operator vs Laplacian operator There's one paper published at siggraph that captured my attention : Steklov Spectral Geometry.

I'm not an expert in geometry processing, but I'm trying to learn as much as I can.
You can easily read in the introduction:

  In this paper we provide a practical and mathematically justified spectral approach to extrinsics geometry for geometry processing via an extrinsic alternative to the intrinsic Laplace-Beltrami operator.

I read through the paper and, for who like me enjoy signal processing, is a very nice read. It my understanding the operator they propose an alternative of the classic laplace beltrami operator:

$$
\Delta_{\mathcal{S}} \psi(x) = 0
$$

The operator proposed is

$$
\left\{
\begin{array}{ll}
\Delta \psi(x) = 0 & x \in \Omega \\
\nabla_n \psi(x) = \lambda \psi(x) & x \in \Gamma
\end{array}
\right.
$$

Which is the "Steklov eigenproblem". The first line of the system above is the laplace equation again (but volumetric) the second line is a condition on the normal derivative.

What I don't understand is why does the former operator represent an "intrinsics" operator while the latter represent an extrinsic operator.

I'm not entirely sure, but I guess it might have to do with the notions of intrinsic vs extrinsic curvature. The Laplace–Beltrami operator on a manifold is, I think, intrinsic in the sense that it can be constructed from the "internal" geometry within the manifold itself, and doesn't depend on any embedding in an ambient space. The Steklov operator, on the other hand, looks like it's written explicitly in terms of an embedding, so would be extrinsic in that sense.


Interpolate vertex attributes with $z$ AFTER homogeneous divide Usually, if we want to interpolate some vertex attributes, say $A_1$, $A_2$ and $A_3$, we have to first divide them by their corresponding $z$ in view space, then we interpolate them using barycentric weights and finally we multiply them by the interpolated $z$ at the current pixel. To interpolate this $z$, we have to inverse the 3 $z$ of each vertex, then interpolate them and finally inverse the result to have the interpolated $z$.
In other words, the interpolated attribute $A$ is equal to:
$$
A = z \left(w_0 \times \frac{A_1}{Z_1} + w_1 \times \frac{A_2}{Z_2} + w_2 \times \frac{A_3}{Z_3}\right)
$$
where $w_i$ are the barycentric weights, $Z_i$ are the depths of each vertex and $z$ is the interpolated depth:
$$
z = \frac{1}{w_0 \times \frac{1}{Z_1} + w_1 \times \frac{1}{Z_2} + w_2 \times \frac{1}{Z_3}}
$$
Now my question is: instead of going back and forth between attributes and their reciprocals, can we just use the vertices'$z$ in normalized device coordinates, that is, the $z$s that have been divided by $w$, after the perspective projection?
If I understand correctly, we interpolate the reciprocal of vertex attributes, because it's linear in screen space, but what about the $z$ in NDC space, after the homogeneous divide? If we can manage to use it, we already have a $z$ that we can linearly interpolate, because it has been homogenized?
In the book "Game Engine Architecture", page 667, they say:

with w-buffering, we cannot linearly interpolate depths directly. Depths must be inverted prior to interpolation and the re-inverted prior to being stored the w-buffer.

In other words, we don't need to invert $z$ back and forth to find the interpolated depth: we just have to use the $z$ in NDC space, that has been divided by view-space $z$. So we can directly interpolate the $z$ in NDC space to have perspective-correct depth, but how can I use this interpolated depth to interpolate other vertex attributes?

The $Z$ of NDC space is related to $1/Z_\text{view}$ but not the same. With a typical projection matrix,  they're related by an affine remapping,
$$
Z_\text{NDC} = a \, \frac{1}{Z_\text{view}} + b
$$
where $a, b$ are some constants related to the near and far planes. So, in general you wouldn't be able to substitute $Z_\text{NDC}$ for $1/Z_\text{view}$ in the interpolation expressions. The $a$ factor isn't a problem, as it would factor out and cancel, but if $b \neq 0$ then it will produce some extra terms and throw off the interpolation.
If I'm hearing you right, it sounds like you're concerned about the idea of needing to prepare every single vertex attribute value (multiplying by $1/Z_\text{view}$) before interpolation and then reverse that afterward. Well, there is an easier way: we can create perspective-corrected barycentric coordinates $w'_0, w'_1, w'_2$, defined by
$$
w'_i \equiv \frac{w_i / Z_i}{w_0/Z_0 + w_1/Z_1 + w_2/Z_2}
$$
Then we can interpolate any attributes by $w'_0 A_0 + w'_1 A_1 + w'_2 A_2$, and it will be perspective-correct interpolation. You can verify this is equivalent to your original interpolation formula in your question. If we have more than three scalar attributes to interpolate (which we usually do), this would be the cheaper way to do it.


Should you measure mesh complexity in triangles or vertices? I have seen people using either vertices or triangles as a quick heuristic to estimate mesh memory usage and mesh rendering time.

I always thought vertices were better, because the number of vertices processed in the vertex shader is known, whereas the number of fragments rendered varies depending on depth complexity, distance to the screen, area of the triangles....

However it seems triangles are used just as often as vertices, if you need one to do a naive estimation, which one is better?

Since you're talking about "vertex shaders" and "fragments", I assume that your question is to be interpreted in the context of real-time rendering using the graphics pipeline on a modern GPU through some modern graphics API. Even if we narrow down the question like that, I don't think a general answer can be given to what you're asking. It depends a lot on what kind of meshes/scenes/application you're dealing with. In general, it's not even clear what "number of vertices" or "number of triangles" should mean exactly. There isn't really the number of vertices and the number of triangles. If you're rendering a non-indexed triangle strip, then there's a direct relation between the number of vertices and the number of triangles. If you're rendering an indexed triangle list, then do we count the number of vertices that were referenced, the number of vertices that were shaded, or do we count the number of indices that were used to form triangles? If you're rendering patches with tessellation, then is the number of vertices the number of control points or is it the number of vertices produced by the tessellation? What if I have a geometry shader that eats a few points, and turns them into loads of overlapping triangles? What if my geometry shading streams the odd triangle to a stream output buffer but doesn't send anything to the rasterizer at all!? The point is, there's a great many different places in a graphics pipeline where something that one might call a "vertex" or a "triangle" could pop up…

In a typical triangle mesh that is supposed to represent the boundary surface of some physical object, there will be a direct relationship between the numbers of vertices and triangles. In such cases, either the number of vertices or the number of triangles is arguably as good a measure as the other when it comes to giving an estimate of mesh complexity. But then, what "complexity" are we talking about exactly? Complexity in terms of storage can probably be estimated from either number of vertices or triangles as explained above. Complexity in terms of rendering time is a whole different story. Depending on your application, it might be that neither the number of triangles nor the number of vertices is a good predictor for performance at all. It's actually trivial to write a program that makes even the latest and greatest graphics card sweat by rendering only a single triangle using no vertex buffer. Because your bottleneck is rasterization, fragment processing, or the framebuffer, for example. In such a case, you'd probably be more interested in the number of fragments produced, or the depth complexity…


What does "region to be shaded" mean in the PCF paper? I am reading the percentage closer filtering (PCF) paper by Reeves, W. et al. (https://graphics.pixar.com/library/ShadowMaps/paper.pdf).

At at section 3.2, it mentioned that "each shading calculation represents some region on a surface". And "each region to be shaded is mapped to a region in the shadow map".

For my understanding, using Opengl fragment shader, for each shading calculation, we just operate on each pixel. We do the shadow calculation on each pixel in the camera image.

Can anyone explain what does that "region to be shaded" mean? Thanks!

Texture mapping is the process of providing a mapping from the surface to a location on the texture. That is, given a particular point on a surface, you can find a specific point in a texture that corresponds to it. Giving positions a UV texture coordinate is a common means of defining a texture mapping, but you can use a function of arbitrary complexity.

Texture mapping is at least theoretically reversible. That is, given a particular texel, you can use the texture mapping to determine where on an object that texel appears. Of course, not every texture mapping is reversible. Some mapping cause a texel to be used in multiple locations on a surface, and not every mapping uses every texel in the texture.

But the important part of the reverse mapping operation is this: a texel maps to some area of the surface, not a single point.

Shadow mapping is a form of texture mapping. It defines a mapping from the scene to a texture by performing perspective projection of the points in the scene onto the surface of a texture, based on a light position.

If you reverse this mapping, a texel in the shadow map maps to an area in the scene. This is the "region in the scene".


Help me understand the voronoi shader I am trying to understand the voronoi shader implemented in the Book of Shaders in the Tiling and iteration section: https://thebookofshaders.com/12/

Specifically how do these lines calculate the distance from a point to its neighboring tile's point:

// Random position from current + neighbor place in the grid
vec2 point = random2(i_st + neighbor);
vec2 diff = neighbor + point - f_st;

// Distance to the point
float dist = length(diff);

// Keep the closer distance
m_dist = min(m_dist, dist);

Specifically how is the diff calculation working? I would have expected i_st to reappear there.

1) Scale the texture coordinates from [0,1]x[0,1] -> [0,3]x[0,3]. It works with higher integer scales as well. This creates an imaginary 3x3 tiles grid.

st *= 3.;

2) Split the upscaled texture coordinates into the integer part and its float reminder.

vec2 i_st = floor(st); //the tile coords of the fragment - an integer pair {0,1,2}x{0,1,2}
vec2 f_st = fract(st); //fragment coords within the tile - a float vec2 [0,1]x[0,1]

//e.g. let's take
//  st == vec2(1.1,1.7)
//  i_st == vec2(1.,1.)
//  f_st == vec2(.1,.7)

Now st == i_st + f_st holds.

3) In order to avoid terms confusion I will call the points that build the Voronoi cells Voronoi points. Each of the 9 tiles has one Voronoi point inside to produce a Voronoi cell. So there will be 9 Voronoi cells visible in the resulting texture.

The main hack here is to have this regular 3x3 tiles grid with the same amount of Voronoi points and randomize their positions only within the respective tile. So the distribution will be pseudo-even. You give up irregular distributions by this assumption, e.g. all points grouped in the upper-left corner, but such arrangements are highly unlikely to result from biologic processes that the cellular noise tries to mimic.

Since random2 is deterministic, it will return the same vec2 each time you enter the same i_st, so for a fixed i_st it will stay constant across all fragments.

vec2 point = random2(i_st); //pick the Voronoi point, e.g. vec2(.6,.4)

The global position of the picked Voronoi point is in fact i_st + point.

4) Take the distance to the Voronoi point inside of the current tile as the initial minimum distance.

vec2 diff = point - f_st //e.g. vec2(.6,.4) - vec2(.1,.7) == vec2(.5,-.3)
float dist = length(diff); //e.g.0.583

5) So you know that there is exactly one Voronoi point in each cell, thus by simple geometric means you can prove that you need to check only the direct neighbors. You need to test if any of the neighboring tiles has a Voronoi point closer to the current fragment than the Voronoi point from 4). Two loops iterate over all 3x3 neighborhood of the current tile. It is much easier to do that relatively {-1, 0, 1}x{-1, 0, 1} instead of using absolute numbers {i_st.x-1, i_st.x, i_st.x+1}x{i_st.y-1, i_st.y, i_st.y+1}. Moreover, we know that each tile is 1x1 in size.

for (int y= -1; y <= 1; y++) {
    for (int x= -1; x <= 1; x++) {
        // relative neighbor coordinates will be sufficient
        // this is the integer part indicating the neighboring tile coordinates
        vec2 neighbor = vec2(float(x),float(y)); //e.g. vec2(-1., 1)

        //to be continued

6) Now you will need to compute the absolute coordinates of neighboring tile to obtain its Voronoi point consistently across all fragments.

        //the global neighbor tile coordinates are neighbor + i_st
        vec2 point = random2(i_st + neighbor); //note that this is a local scope variable

        //to be continued

7) And now the most interesting step for you, computing the distance to the neighboring Voronoi point

        //neighbor: relative tile coords, e.g. vec2(-1., 1.)
        //point:    Voronoi point inside of the neighboring tile e.g. vec2(0.9, 0.2)
        //f_st:     current fragment position within its home tile,
        //          e.g. when st == vec2(1.1, 1.7) then f_st == vec2(.1,.7)

        vec2 diff = neighbor + point - f_st;

        //to be continued

The example values yield:

neighbor + point: vec2(-1.,1.) + vec2(0.9, 0.2) == vec2(-0.1, 1.2)

That is the relative position of the neighboring tile Voronoi point. Relative means **with respect to the lower-left corner of the current fragment's tile.
The same relative coordinates apply to f_st and also to point from 3).

(neighbor + point) - f_st: vec2(-0.1, 1.2) - vec2(.1,.7) == vec2(-.2,.5)

This is shorter distance (0.538 vs. previous 0.583) so for the example positions the Voronoi point from the upper left cell is closer.

8) The rest is pretty straight forward. Compute the magnitude of the diff vector and keep the minimum.

        float dist = length(diff); //distance to the neighboring Voronoi point

        m_dist = min(m_dist, dist); //remember the minimum of distances
    }
}

I hope this will help you to understand that. The main simplification is the switch to relative coordinates as described in 7).

Looking at the code I see that the middle tile (current fragment tile) will be evaluated twice. This can be reduced by omiting step 4) initializing dist to 2*sqrt(2) which is very roughly 3. That is the length of the diagonal of two cells - the farthest distance the current fragment st could be from one of the neighboring Voronoi points.
Border tiles are handled in an elegant implicit way. random2 generates a Voronoi point for them as well and involves it in the minimal distance computation. So You will see 9 complete and possibly up to 16 more incomplete Voronoi cells.

Hope that helps, feel free to ask any further details.


CrazyBump Shape Recognition I've recently been using CrazyBump, and I think I'm starting to understand how it works. But I'm confused about what it calls Shape Recognition. According to this website, that feature attempts to add directional lighting information, but what exactly does that mean?

Here's my test texture. It's simple, just so I can understand what's going on.

Here's the generated normal map, before Shape Recognition.

Here's the generated normal map, after Shape Recognition.

Based on the article, it seems like, when it adds "directional lighting information", the program tries to infer a location of a light source and base the normals on the colors with the added information of the directional lighting from the pixel to the inferred light source. I feel like I'm making no sense right now, so here's an example using their pictures:

As you can see, on the far right of the normal map, it's very pink, just like how the sample image is completely white. I would assume this is the case because the inferred light is directly on top of it, with all the normals (in sort of a curved fashion, resulting in the gradient) pointing right at it. In contrast...

This normal map isn't trying to figure out a light source, and instead just kind of uses the sample texture as a heightmap of sorts.
Late reply, but hope this helps! :)


Rigorous and simple explanation for satisfying boundary conditions in Stam and Bridson papers I read Stable Fluids and the code by Jos Stam and FLUID SIMULATION SIGGRAPH 2007 Course Notes by Robert Bridson. The boundary conditions in the two papers are different in my sight. I don't understand them. How are these two ways related to solving a linear system of PDEs with boundary conditions?

In Stam's paper, he uses a function called set_bnd to set boundaries, so is this function just setting the boundaries that I need in my matrix of linear systems of Pressure Poisson and Diffuse Equation? In Numerical Analysis, we don't modify the matrix with the boundary values, but we put the boundary condition either it's a Neumann or Dirichlet. I don't really understand what kind of boundaries is this set_bnd function. My thinking now is that it sets the boundary values which will be used to solve linear systems but what kind of boundaries?

In Bridson's paper, I don't find any implementation about boundaries except for speaking about the normal components of velocity. Actually, I can't find any relation between this and Stam's way. In the section on making fluid incompressible, he says something about boundaries but I don't understand. He just satisfies boundary conditions on the boundary cells then he solves the system or he put the boundaries in the matrix?
Actually I don't know how to satisfy u*n = u_solid * n which I think is the no stick boundaries for interacting between a fluid and solid boundaries.

First of all, I would like to state that handling the boundary conditions correctly is challenging, both from mathematical perspective and from an implementation perspective. Additionally, I think most of the papers don't explain the handling of boundary conditions sufficiently.

Furthermore, it is worth noticing that you have to apply boundary conditions to both the velocity field and the pressure field (and they are linked to each other). Additionally, you have to distinguish between is the pressure solve (or projection step) and the diffusion step, which must be treated separately.

In the very beginning you need to decide what kind of physical behavior you want to have at the boundaries. The standard example is a square surrounded by solid walls where no fluid can enter or leave. In this case you constrain the normal component of the velocity field to be zero ( a dirichlet boundary condition for the diffusion step). As we don't want changes for the normal component of the velocity in the projection step,  we also need to constrain the pressure. If you check the pressure update routine (which makes the velocity field divergency free) you can see that the derivative of the pressure field in direction of the normal must be zero, I.e. a Neumann boundary condition for the pressure field.

Let's assume you are talking about the boundary conditions in the pressure solve/ projection step. Stam states that for pressure there is a Neumann boundary condition. As written above this condition implies that the velocity won't be modified at the last stage of the projection step (the pressure update). In order to achieve the dirichlet boundaries for the velocity as bridson states it, you require to have homogenous Neumann conditions. So for pressure, both authors have equivalent boundary conditions.

With these considerations you need to set up your system of linear equations. For this it is worth to note the equations for some cells (I.e. one row of the system of linear equations) and check how the matrix and the right hand side must be modified to account for the boundary conditions. In a 2d scenario we have a 5 point Laplace stencil that results in a 4 on the diagonal and -1 for each of the adjacent cells (times some constants). In cells that are adjacent to the boundary there is at least one value that is outside of the simulation domain and care has to be taken what we do with this value. For homogenous dirichlet conditions this value is simply zero and no changes are required for the matrix and the right hand side. For Neumann boundaries the constraint is that there is no change across the boundary,  I.e. the value should be identical to the one of the cell. This reduces the value on the diagonal of the matrix by -1, I.e. 3 for a cell with one Neumann boundary. These considerations result in the linear system that you have to solve.

The set bnd method (that doesn't appear in the paper by the way) is a trick to make the computations without the necessity to modify the matrix. This trick only works with gauss seidel or Jacobi iterations (As commented on your other question). The set bnd method is part of the code that stam published. Personally I find it difficult to read and understand because of the variable names and the "b" that changes the behavior significantly and that seems to confuse you as well (see your other question).

Edit:
At first glance it looks like stam and bridson apply different boundary conditions, but they are actually the same. If we assume a solid body that is not moving and the velocity is zero before the pressure correction then the difference between them is zero. The term in parentheses in (4.23) will then be evaluated to -p_{i,j} which is exactly a  homogeneous Neumann boundary condition for the pressure. I've got the second edition of bridsons book,  where  this is explained in equation 5.3 and at the bottom of the page in more detail.

Edit2: up to know we discussed the Neumann boundary conditions for the pressure. No slip/slip boundary conditions are applied to the velocity and for both (with non-moving walls) you need homogenized Neumann boundary conditions for the pressure. With stam's approach of discretization, I.e. storing all velocity components at the cell center it is easier to realize no slip boundary conditions as the boundary for all velocity components are at cell faces. So to realize no slip boundary conditions in the diffusion step (mathematically you can't realize no slip conditions with the Euler equations ( no diffusion)) you need to use homogeneous dirichlet boundary conditions for all velocity components.

Edit3: It is not straight forward to use the set_bnd method on a staggered grid for no slip boundary conditions. The storage for the tangential (to the boundary) components is not ideal.

Edit 4: A Neumann boundary condition for the pressure implies that there are no changes for the (normal) velocity at this boundary. If you subtract the pressure gradient at such a boundary, the pressure gradient is simply zero due to the boundary condition.

Edit 5: Slip / no slip boundary conditions (for the velocity) are not(!) realized in the pressure correction step. For non-moving solid walls there is always a homogeneous Neumann boundary condition for the pressure no matter if you impose a slip or no-slip boundary condition.

Edit 6: I don't "have" facebook ;)

There are more subtleties like singular systems for Neumann- only boundary conditions, central differences in staggered or standard grids, distance to the boundary (at cell border or at cell center outside three domain) etc., that I could address if you detail your question.


Ray tracing pseudocode shadow I don't understand the if (depth > MAX) return Black part. Does it have something to do with shadows, because in other algorithms they shoot a shadow ray towards the light source to check for shadows but they don't have it here?

This is a recursive function (it calls itself). Each time it calls itself it does so with depth + 1 instead of depth. When it is called with depth > MAX, it simply returns Black rather than call itself again. This ensures that the function will eventually stop recursing and return a value. It is the colour equivalent of returning zero.

You can visualise this as a light ray reflecting from multiple surfaces successively. Each time the ray reflects from a surface the same function is used to calculate the colour of the light in that direction. If you set MAX to 0, there will be no reflections in your resulting image. If you set MAX to 2, there will be reflections of reflections which will look more realistic, but take longer to calculate.

Intuitively, MAX is the number of surfaces the ray will bounce off before it is decided what colour should be displayed.


Accurately printed CIE diagram This is more just for curiosity rather than actually wanting to get one, but is there a place where I could buy a printed version of the CIE xy colour space diagram which accurately recreates all of the visible colours outside of the regular printing gamuts?

Its not actually possible to do. Because that paper you print on needs to have properties not avalialbe. The color on the edge are very pure and can not be mixed. I dont also see any way they could be reflected either. Also the xy diagram is not the full color space just a slice of it.


Self intersection artifacts in 3D image raytracing I am developping a voxel raytracer for global illumination.
I have been successful in multiple aspects but there is a problem I am not sure how to solve.
The model works as most other Voxelization based algorithms work. We use rasterization to store the scene in a 3D volumetric structure and then we raytrace inside of said 3D structure.
In the "pure" model (only voxels) I can generate prety decent shadows (ignore the circular artifacts):

However when I try to create a combined scene (normal rendering for colors, voxels for shadows) I sometimes get self intersection based artifacts:

I can improve the image a bit by using an epsilon tolerance (considering that if we are too close to the original starting position we haven't collided with anything yet):

However doing this creates poor shadows by eliminating certain collissions (look at the archs for example, where the areas too close to the source should be darkened but are instead lightened).
I want to know if there is a better way to check for self intersection, other than checking the original position?
i.e I want to be able to get as close to the shadows in the first image as possible.

I want to know if there is a better way to check for self intersection, other than checking the original position?

Since cubes are convex solids, a self-intersection check shouldn't introduce artefacts in your case, while it would in a more general ray-tracer (see my answer about the shadow terminator problem for more about that). In general, it's a good idea for your ray object to have a reference of some kind to the object it came from, as well as the kind of ray it is (primary, shadow, reflection, refraction). Normally the reference should be to the exact polygon, not just the whole object; in your case, it should be the particular cube. You could identify it by the co-ordinates of the cube, if you're using spatial hashing to store your voxel grid.

Then, in your intersection test, if your ray is a shadow ray, you should check whether the origin cube is the same as the cube being intersected, and discard that intersection if so.


Is there some book about how to design a 3D math lib? 3D math covers vector algebra and matrix algebra, which are easily calculated on paper, but not so easy when coding them effectively. I only leant some basic algorithms on matrix in my Data Structure course, which is not enough when it comes to accurate and effective operations.

Is there some books on this topic?

Books: http://www.essentialmath.com/book.htm is the one.

Best algos on the real 3D topics (and some tricky 2D like image resize and premult-alpha ), explained and with source most of the time.

The website is very nice too, the "tutorial" full of gems

Adding http://www.geometrictools.com/index.html which is the most complete source code repository on the topic, with loads of very precise algos implemented.


Why does substracting -1 in shader fail for a float data type? I am loading a texture in a WebGL fragment shader (GLSL version 1.00).

This is my working code:

precision highp float;
uniform sampler2D u_texture;
varying float v_coord;

void main(void) {
  float inv_coord;
  gl_FragColor = texture2D(u_texture, vec2(v_coord, 0.5));
}

v_coord holds the texture coordinates for each vertex. What I'm trying now, is to invert the texture coordinates. I thought about something like abs(v_coord - 1). However, whenever I try to substract - 1 from v_coord, my shader fails to compile.

precision highp float;
uniform sampler2D u_texture;
varying float v_coord;

void main(void) {
  float inv_coord = v_coord - 1; // fails here
  inv_coord = abs(inv_coord);
  gl_FragColor = texture2D(u_texture, vec2(inv_coord, 0.5));
}

Why does substracting minus one in fragment shader fail for a float data type? How to invert the texture coordinate value correctly?

Without seeing the error message I can't be sure but I think it's failing on the 1 being int instead of a float.

float inv_coord = v_coord - 1.0;

There is a simpler method, you can do 1.0 - v_coord and do away with the abs.


Why does the G1 term have a 2 in the numerator in Walter's formula, but not in Heitz's generalized formula? In Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs, Eric Heitz derives the Separable Masking and Shadowing function as (P.83-84):

In the mentioned Microfacet Models for Refraction through Rough Surfaces by Walter et al., Smith term is computed as:

I don't see a parameterisation of Λ that could make appear the generic form (Heitz version) from the specific form derived by Walter. What am I missing?

Actually I just had to solve the following equation to find a solution, consistent with the presence of the "2" in the numerator:

$\frac{1}{1+\Lambda (m)}=\frac{2}{1+\sqrt{1+\alpha ^{2}tan_{2}\theta_{m} }}$

$1+\Lambda (m)=\frac{1+\sqrt{1+\alpha ^{2}tan_{2}\theta_{m} }}{2}$

$\Lambda (m)=\frac{-1+\sqrt{1+\alpha ^{2}tan_{2}\theta_{m} }}{2}$


Algorithms for scan converting b-spline and nurbs In all computer graphics books there are algorithms for scan converting simple primitives like lines, circles, ellipse,...

I can't find algorithms for more advanced curves like bezier curves, b-spline, nurbs.

Where can I find the references ?

I have also another doubt: are scan converting and rastering the same thing ?

Ignoring Non-uniform B-splines (rational or not), I have had some experience with rasterisation of Beziers and, since there is a trivial mapping from Uniform B-Splines to Beziers, those too.

I have used two different techniques: The first was a scan-line renderer that used Newton-Rhapson to compute the intersection of the current scanline with the curve. This requires the first derivative but, as that can be trivially derived, again as a Bezier representation, from the control points of the parent curve, it's easy to obtain. Further, the bounding box of the control points of the 1st derivative can be useful to detect if the region might have more than 1 intersection with the scan line.
The current scanline can use the previous scanline's solution as a starting point - usually that gives a highly accurate solution with one iteration.

Alternatively, a second approach is to simply apply binary subdivision of the curve and render it as series of straight line segments with your favourite polygonal renderer. If I recall correctly, at least with cubic Beziers, with each subdivision the error between the line segment joining the end points of the (sub)Bezier and the true curve decreases by a factor of 4. In most cases it therefore doesn't take many subdivisions before the difference between the polygonal approximation and the true curve is insignificant.

Finally, if you need info on derivatives of Beziers (or almost everything else related to them), this astounding web-page appeared in my twitter feed recently.


Path tracing: How to ensure we are sampling a direction vector within the visible hemisphere of a surface? Given a surface Normal vector of an intersection point, how can one generate a random direction vector that is guaranteed to be in the visible hemisphere? The way I am currently doing this is to sample a random direction vector d and dot it with the Normal and keep that vector if the dot product is greater than 0. This method is not guaranteed to generate a vector in the visible hemisphere of BRDF as ~50% of the random vectors would have a 0 or negative dot product result.
I saw somewhere that there is a way to transform the randomly generated vector d to place it in the visible hemisphere using a transformation matrix but I don't know how to do it. Can someone write a [pseudo]code for how one generate a direction vector using the transformation method?

If it’s on the wrong side of the normal, you don’t have to throw it away—negating it will give you a vector that’s in the visible hemisphere.
Answering your comment, to get a vector that’s within some angle θ of the normal, this should work (GLSL):
vec3 direction(vec3 normal, vec2 randomValues, float maxTheta) {
    // pick an orthogonal tangent vector using the method described here: http://lolengine.net/blog/2013/09/21/picking-orthogonal-vector-combing-coconuts
    vec3 tangent = (normal.x > normal.z) ? vec3(-normal.y, normal.x, 0.0) : vec3(0.0, -normal.z, normal.y);
    // and a bitangent vector orthogonal to both
    vec3 bitangent = cross(normal, tangent);

    float phi = 2.0 * PI * randomValues.x;
    float theta = randomValues.y * maxTheta;
    float sinTheta = sin(theta);
    return tangent * cos(phi) * sinTheta + bitangent * sin(phi) * sinTheta + normal * cos(theta);
}


Fading particles and transition I'm trying to render a particle simulation. I want each particle to leave a trace. To achieve this I'm using a framebuffer and two textures. On each frame the algorithm goes like this:

Render to a framebuffer previously rendered screen texture with opacity * 0.9;
Render current particles at updated positions;
Switch to main display and render current content of the framebuffer;

This works fine, and I get nice looking particle trails:

Problem

I want to allow users to pan and zoom canvas with these particles, similar to a map interface. This is achieved by applying a transformation matrix to each particle.

I want to keep particle's state, but I don't want the pan/zoom to leave a particle trace:

Bad

Here I'm dragging canvas up and down with mouse cursor. See how particles leave a trace?

Good

Here I'm also dragging the canvas up and down, but particles remain on their trajectory, without leaving a pan trace:

I'm wondering what is a proper way of achieving this? I got a basic solution working by applying a transformation to a previous frame's texture, so that it approximately matches what will be rendered in the current frame, but I there still a few artifacts:

When zooming in, the updated texture "explodes":

I assume this is because I'm using fixed point size for a particle (regardless of the zoom level), but when we scale texture, we scale rasterized image, and thus all particles rendered on it also get bigger.

When a particle enters the scene (crosses the border of visible area), it is also very likely to leave a trail:

I'm not sure why this one happens. My screen/back textures are the same size, I'm just translating back texture. I thought it might be related to clamp_to_edge setting, but I don't know how to prove or validate this. I've got a workaround by extending canvas beyond visible area (say by 2%) and in the fragment shader I check that if particle is closer than 2% to the border - I give it transparent color. This kind of "fixes" the problem, but I feel uncomfortable not knowing why it's happening.

Finally, if transformation is very-very small (think kinetic movement), I also observe artifacts:

I'd appreciate if you could explain some of these artifacts, suggest possible solutions, or general practices used to solve this problem.

I made a visualization a little like yours years ago, and used a similar technique, because it's the natural approach to the problem. After I spent a few years writing drivers for modern GPUs, I wouldn't use this approach at all, because it creates a dependency from one frame to the next, which introduces pipeline stalls and/or false sharing.

Today, I'd keep the path of the particle as part of its state, and render the whole trail each frame, instead of reusing the old FBO. It's conceptually a little more difficult, and you have to be careful that the way you draw the past path is exactly the same each frame. For example, if you represent the path with some kind of cubic spline, you can't just move the final control point forward, because it'll change the whole curve slightly. A circular buffer of (x, y) positions would be much easier to get right.

What you get in exchange for this extra complexity is (a) reduced frame time, memory use, and power consumption (which might be important if you're on mobile), and (b) you won't have your current problem at all. Maybe the frame time advantage won't be as big in WebGL, because you'll end up doing extra work in Javascript, which is slow, but memory bandwidth is more of a bottleneck on most systems. Just make sure that the co-ordinates you store are in the space of your simulation, and transform them into screen co-ordinates when you draw them, just like you would with a 3D scene.


How does hardware texture compression work? That it compresses the data compared to the pixel array is obvious.

But what makes it different from from normal compression (like png, jpeg)?

"How (hardware) texture compression works" is a large topic. Hopefully I can provide some insights without duplicating the content of Nathan's answer.

Requirements

Texture compression typically differs from 'standard' image compression techniques e.g. JPEG/PNG in four main ways, as outlined in Beers et al's Rendering from Compressed Textures:

Decoding Speed: You don't want texture compression to be slower (at least not noticeably so) than using uncompressed textures. It should also be relatively simple to decompress since that can help achieve fast decompression without excessive hardware and power costs.
Random Access: You can't easily predict which texels will be required during a given render. If some subset, M, of the accessed texels come from, say, the middle of the image, it's essential that you don't have to decode all of the 'previous' lines of the texture in order to determine M; with JPEG and PNG this is necessary as pixel decoding depends on the previously decoded data.
Note, having said this, just because you have "random" access, doesn't mean you should try to sample completely arbitrarily
Compression Rate and Visual Quality: Beers et al argue (convincingly) that losing some quality in the compressed result in order to improve compression rate is a worthwhile trade-off. In 3D rendering, the data is probably going to be manipulated (e.g. filtered & shaded etc) and so some loss of quality may well be masked.
Asymmetric encoding/decoding: Though perhaps slightly more contentious, they argue that it is acceptable to have the encoding process much slower than the decoding. Given that the decoding needs to be at HW fill rates, this is generally acceptable. (I will admit that compression of PVRTC, ETC2 and some others at maximum quality could be faster)

Early History & Techniques

It may surprise some to learn that texture compression has been around for over three decades. Flight simulators from the 70s and 80s needed access to relatively large amounts of texture data and given that 1MB of RAM in 1980 was > $6000, reducing the texture footprint was essential. As another example, in the mid 70s, even a small amount of high speed memory and logic, e.g. enough for a modest 512x512 RGB frame buffer) could set you back the price of small house.

Though, AFAIK, not explicitly referred to as texture compression, in the literature and patents you can find references to techniques including:
 a. simple forms of mathematical/procedural texture synthesis,
 b. use of a single channel texture (e.g. 4bpp) that is then multiplied by a per-texture RGB value,
 c. YUV, and
 d. palettes (the literature suggesting use of Heckbert's approach to do the compression)

Modelling Image Data

As noted above, texture compression is nearly always lossy and thus the problem becomes one of trying to represent the important data in a compact way whilst disposing of the less significant information. The various schemes that will be described below all have an implicit 'parameterised' model that approximates the typical behaviour of texture data and of the eye's response.

Further, since texture compression tends to use fixed-rate encoding, the compression process usually includes a search step to find the set of parameters which, when fed in to the model, will generate a good approximation of the original texture. That search step, however, can be time consuming.
(With the possible exception of tools such as optipng, this is another area where typical use of PNG & JPEG differ from texture compression schemes)

Before progressing further, to help with further understanding of TC it's worth taking a look at Principal Component Analysis (PCA) - a very useful mathematical tool for data compression.

Example texture

To compare the various methods, we'll use the following image:

Note that this is a fairly tough image, especially for palette and VQTC methods as it spans much of the RGB colour cube and only 15% of the texels use repeated colours.

PC and (post mid 90s) Console Texture compression

To reduce data costs, some PC games and early games consoles also made use of palette images, which is a form of Vector Quantisation (VQ).
Palette-based approaches make the assumption that a given image only uses relatively small portions of the RGB(A) colour cube.  A problem with palette textures is that the compression rates for the achieved quality is generally rather modest. The example texture compressed to "4bpp" (using GIMP) produced

Note again that this is a relatively tough image for VQ schemes.

VQ with larger vectors (e.g. 2bpp ARGB)

Inspired by Beers et al, the Dreamcast console used VQ to encode 2x2  or even 2x4 pixel blocks with single bytes. While the "vectors" in the palette textures are 3 or 4 dimensional, the 2x2 pixel blocks can be considered to be 16 dimensional. The compression scheme assumes there is sufficient, approximate repetition of these vectors.

Even though VQ can achieve satisfactory quality with ~2bpp, the problem with these schemes is that it requires dependent memory reads: An initial read from the index map to determine the code for the pixel is followed by a second to actually fetch the pixel data associated with that code. Additional caches can help alleviate some of the incurred latency, but adds complexity to the hardware.

The example image compressed with the 2bpp Dreamcast scheme is
.
The index map is:

Compression of VQ data can be done in various ways however, IIRC, the above was done using PCA to derive and then partition the 16D vectors along the principal vector into 2 sets such that two representative vectors minimised the mean squared error. The process then recursed until 256 candidate vectors were produced.
A global k-means/Lloyd's algorithm approach was then applied to improve the representatives.

Colour Space Transformations

Colour space transformations also make use of PCA noting that the global distribution of colour is often spread along a major axis with far less spread along the other axes. For YUV representations, the assumptions are that a) the major axis is often in the luma direction and that b) the eye is more sensitive to changes in this direction.

The 3dfx Voodoo system provided "YAB", an 8bpp, "Narrow Channel" compression system that split each 8 bit texel into a 322 format, and applied a user selected colour transform to that data to map it into RGB. The main axis thus had 8 levels and the smaller axes, 4 each.

The S3 Virge chip had a slightly simpler, 4bpp, scheme that allowed the user to specify, for the entire texture, two end colours, which should lie on the principal axis, along with a 4bpp monochrome texture. The per-pixel value then blended the end colours with appropriate weights to produce the RGB result.

BTC-based schemes

Rewinding some number of years, Delp and Mitchell designed a simple (monochrome) image compression scheme called Block Truncation Coding, (BTC). This paper also included a compression algorithm but, for our purposes, we are mainly interested in the resulting compressed data and of the decompression process.

In this scheme, images are broken into, typically, 4x4 pixel blocks, which can be compressed independently with, in effect, a localised VQ algorithm. Each block is represented by two "values", a and b, and a 4x4 set of index bits, which identify which of the two values to use for each pixel.

S3TC:  4bpp RGB (+1bit alpha)
Although several colour-variants of BTC for image compression were proposed, of interest to us is Iourcha et al's S3TC, some of which appears to be a rediscovery of the somewhat forgotten work of Hoffert et al that was used in Apple's Quicktime.

The original S3TC, without the DirectX variants, compresses blocks of either RGB or RGB+1bit Alpha to 4bpp. Each 4x4 block in the texture is replaced by two end colours, A and B, from which up to two other colours are derived by fixed-weight, linear blends. Further, each texel in the block has a 2-bit index that determines how to select one of these four colours.

For example the following is a 4x4 pixel section of the test image compressed with the AMD/ATI Compressenator tool. (Technically it's taken from a 512x512 version of the test image but forgive my lack of time to update the examples).

This illustrates the compression process: The average and the principal axis of the colours are calculated.  A best fit is then performed to find two end points that 'lie on' the axis which, along with the two derived 1:2 and 2:1 blends (or in some cases a 50:50 blend) of those end points, that minimises the error.  Each original pixel is then mapped to one of those colours to produce the result.

If, as in this case, the colours are reasonable approximated by the principal axis, the error will be relatively low. However if, like in the neighbouring 4x4 block shown below, the colours are more diverse, the error will be higher.

The example image, compressed with the AMD Compressonator produces:

Since the colours are determined independently per-block, there can be discontinuities at block boundaries but, as long as the resolution is kept sufficiently high, these block artefacts may go unnoticed:

ETC1:  4bpp RGB
Ericsson Texture Compression also works with 4x4 blocks of texels but makes the assumption that, much like YUV, the principal axis of a local set of texels is often very strongly correlated with "luma". The set of texels can then be represented by just an average colour and a highly quantised, scalar 'length' of the projection of the texels onto that assumed axis.

Since this reduces the data storage costs relative to say, S3TC, it allows ETC to introduce a partitioning scheme, whereby the 4x4 block is subdivided into a pair of horizontal 4x2 or vertical 2x4 sub-blocks. These each have their own average colour.
The example image produces:

The area around the beak also illustrates the horizontal and vertical partitioning of the 4x4 blocks.

Global+Local

There are some texture compression systems that are a cross between global and local schemes, such as that of the distributed palettes of Ivanov and Kuzmin or the method of PVRTC.

PVRTC:  4 & 2 bpp RGBA
PVRTC assumes that an (in practice, bilinearly) upscaled image is a good approximation to the full-resolution target and that the difference between the approximation and the target, i.e. the delta image, is locally monochromatic, i.e. has a dominant principal axis. Further, it assumes the local principal axis can be interpolated across the image.

(to do: Add images showing breakdown)

The example texture, compressed with PVRTC1 4bpp produces:

with the area around the beak:

Compared to BTC-schemes, the block artefacts typically are eliminated but there can sometimes be "overshoot" if there are strong discontinuities in the source image, for example around the silhouette of the lorikeet's head.

The 2bpp variant has, naturally, higher error than the 4bpp (note loss of precision around the blue, high frequency areas near the neck) but arguably still of reasonably quality:

A note on decompression costs

Although the compression algorithms for the schemes described above have a moderate to high evaluation cost, the decompression algorithms, especially for hardware implementations, are relatively inexpensive. ETC1, for example, requires little more than a few MUXes and low-precision adders; S3TC effectively slightly more addition units to perform the blending; and PVRTC, slightly more again.  In theory, these simple TC schemes could allow a GPU architecture to avoid decompression until just prior to the filtering stage, thus maximising the effectiveness of internal caches.

Other Schemes

Other common TC modes that should be mentioned are:

ETC2 - is a (4bpp) superset of ETC1 that improves the handling of regions with colour distributions that don't align well with 'luma'. There are also a 4bpp variant that supports 1 bit alpha, and an 8bpp format for RGBA.
ATC - Is effectively a small variation on S3TC.
FXT1 (3dfx) was a more ambitious variant of the S3TC theme.
BC6 & BC7 : An 8bpp, block-based system supporting ARGB. Apart from HDR modes, these use a more complex partitioning system than that of ETC to attempt to better model image colour distribution.
PVRTC2 : 2&4bpp ARGB. This introduces additional modes including one to overcome limitations with strong boundaries in the images.
ASTC : This is also a block based system but is somewhat more complicated in that it has a large number of possible block sizes targeting a wide range of bpp. It also includes features such as up to 4 partition regions with a pseudo-random partition generator, and variable resolution for the index data and/or colour precision and colour models.


How does the Lane Riesenfeld algorithm work? I understand de Casteljau's algorithm and I am familiar although not fully experienced with B-Splines. I am trying to understand the geometric interpretation of B-splines through the algorithm (just like how de Casteljau's algorithm shows the geometric interpretation of Bézier curves). Thank you in advance.

What you're looking for is called de Boor's algorithm. It lets you compute a point on a b-spline curve by doing a series of linear interpolation (LERP) calculations. So, it works very much like the de Casteljau algorithm for Bezier curves. In fact, the de Casteljau algorithm is a special case of de Boor's algorithm.

A link
Another one
And another


How does a path tracer with next-event estimation work? I am trying to implement a simple path tracer with next-event estimation in Java. The general idea is to trace a ray through the scene as usual (using a cosine distribution to determine the next random direction at every bounce), while also taking the direct illumination into account at every bounce. I think I have most pieces of the puzzle available, but I can't fit them together correctly.

Here is how I shade matte objects: add the direct illumination to the radiance coming from a random ray (chosen according to a cosine distribution), the latter multiplied by the BRDF.

public RGBSpectrum shadePath(Shading shading, HybridPathTracer tracer, int length) {
    cosinusSampler sampler = new cosinusSampler(1);
    Point sample = sampler.computeSample();

    Vector w = shading.getIntersectionPoint().getNormal();
    Random PRNG = new Random();
    Vector r = new Vector(PRNG.nextDouble(), PRNG.nextDouble(), PRNG.nextDouble()).normalize();
    Vector v = r.cross(w);
    Vector u = v.cross(w);
    Vector wi = u.scale(sample.x).add(v.scale(sample.y)).add(w.scale(sample.z));
    Ray ray = new Ray(shading.getIntersectionPoint().toPoint(shading.getRay()), wi);
    RGBSpectrum reflection = tracer.trace(ray, length + 1);
    RGBSpectrum brdf = shading.getColour().scale(getDiffuseReflection());
    return shade(shading).add(brdf.multiply(reflection));
}

When a ray reaches an emissive material, black is returned, unless the path length is equal to 1:

public RGBSpectrum shadePath(Shading shading, HybridPathTracer tracer, int length) {
    Ray ray = shading.getRay();
    Vector n = shading.getIntersectionPoint().getNormal();

    if (length == 1 && n.dot(ray.direction) < 0) {
        return getTexture().getColour();
    } else {
        return RGBSpectrum.BLACK;
    }
}

I read Progressive Path Tracing with Explicit Light Sampling | Computer Graphics and thought that I was using the same idea in my code, but I must be doing something wrong, because my images are getting too bright when the maximal path length increases (see the examples for maximal length 2, 4 and 6 underneath). What is the problem?

Note: I know I should also consider some edge cases for specular surfaces, but maybe it's better to ignore those first and focus on matte objects. (I think I have prepared my code for specular surfaces anyway.) My shade function for direct illumination of matte objects should be fine, but here it is for reference:

public RGBSpectrum shade(Shading shading) {
    RGBSpectrum colour = shading.getColour();
    double L = 0;
    final Point intersection = shading.getIntersectionPoint().toPoint(shading.getRay());

    for (Light light : shading.getScene().getLights()) {
        double l = 0;

        for (Point sample : light.computeSamples()) {
            Vector wi = sample.subtract(intersection);
            double lengthSquared = wi.lengthSquared();
            wi = wi.normalize();
            double cos = shading.getIntersectionPoint().getNormal().dot(wi);

            if (cos > 0) {
                Ray shadowRay = new Ray(intersection, wi);

                if (!light.inShadow(shadowRay, shading.getScene())) {
                    if (light instanceof PointLight) {
                        double denom = 4 * Math.PI * lengthSquared;
                        l += light.getPower() * cos / denom;
                    } else if (light instanceof RectangularLight) {
                        Rectangle rectangle = ((RectangularLight) light).getRectangle();
                        double cosp = -wi.dot(rectangle.getNormal());
                        l += light.getPower() * cos * cosp * rectangle.getArea() / lengthSquared;
                    }
                }
            }
        }

        L += l / light.getSamplesNumber();
    }

    return colour.scale(getAmbientReflection() + L * getDiffuseReflection());
}

I noticed three potential problems. First, this bit of code looks suspicious:

Vector r = new Vector(PRNG.nextDouble(), PRNG.nextDouble(), PRNG.nextDouble()).normalize();
Vector v = r.cross(w);
Vector u = v.cross(w);
Vector wi = u.scale(sample.x).add(v.scale(sample.y)).add(w.scale(sample.z));

The vectors v and u are not normalized, so the construction of wi will be distorted. (Also, using a random vector r to construct the basis is really more work than necessary; you might take a look at this paper for a faster way of doing that.)

Second, you have a getAmbientReflection() in your direct lighting code—that should not be there for path tracing, where "ambient" light is properly calculated by the recursive trace.

Finally, I think the result of your direct illumination path ought to be scaled by $1/π$. This is because the Lambert BRDF is equal to $c_d/π$ where $c_d$ is the diffuse color. The direct illumination path needs to evaluate the full BRDF, including that $1/π$ factor, using the irradiance sample coming in from the light source.

However, the cosine-weighted recursive trace does not need a $1/π$, as it is cancelled out by the cosine PDF. That one should just use $c_d$, like you have it.

You can check that the $1/π$ factors are correct by checking that the average brightness is equal with and without next-event estimation (keeping the path length the same).

BTW, note that the image getting somewhat brighter as you increase the path length is to be expected, as longer paths = more bounces = more illumination. (Russian-roulette path termination fixes this.) Longer paths will also give stronger global illumination / "color-bleeding" effects. That seems to be what's happening in your images—look at the color of the shadow under the sphere; it gets progressively more influenced by the color of the walls and sphere as you increase the path length.

If this is undesirable, try reducing the diffuse reflection factors of all the materials in the scene; they might be unrealistically high, and cutting them down will reduce the bounce light and color-bleeding.


Why do vulkan extensions need to be enabled? This is sort of an API design question. Why do vulkan instance/device extensions need to be specifically enabled, in order for their calls to be available?

Wouldn't it make more sense to have extensions just always “be there” if they're supported in this environment, and then the API user can query which ones are actually available after device creation?

Another way of phrasing this question might be: Is there any reason not to enable every supported extension when creating a vulkan instance/device?

The reason I'm asking is because I'm specifically worried about vulkan interop between multiple programs. The fact that each program's VkDevice is distinct means you can't just straight up share VkImages. But agreeing on a common VkDevice is tricky because they may have different extensions/features enabled, so the interop API must account for the fact that both users may provide a set of extensions that they want enabled on the resulting VkDevice. This seems needlessly complex - wouldn't it be easier to just create one VkInstance and one VkDevice with everything enabled?

Extensions can change the visible behavior of the implementation, not to mention the implementation of it.

Consider KHR_external_semaphore and its various sub-extensions. If you don't specify them, then the VkDevice's implementation knows that you're not going to need to be able to turn a VkSemaphore into an externally-accessible mutex and back. So the implementation may be able to use a more efficient VkSemaphore implementation.

If the extension is always on, the device must assume that you're going to use it. And thus, any possible efficiency is lost.

  wouldn't it be easier to just create one VkInstance and one VkDevice with everything enabled?

How? If you have two VkDevices, unless they are both attached to the same VkPhysicalDevice, there's no guarantee that "everything" is the same thing for both devices.

Cross-process Vulkan interop is not something external code is expected to define. There are extensions that provide various aspects of such interop.


How do you loop through the contents of a buffer in DX11? Okay, so I'm currently using the IAGetVertexBuffers() method and I'd like to loop through some of the returned buffers to verify the contents.
However, I'm kind of new to DirectX and C++ in general. I tried using a a simple C++ 11 loop:
for (const auto& verts : veBuffer) {
        std::cout << verts << std::endl;

But I get an error saying this statement requires a suitable begin statement. Any suggestions would be appreciated at this point.

There are a couple of issues.  First you need to make sure that veBuffer has the correct type, so that probably means you need to cast it to pointer to vertex:
const auto vertices = reinterpret_cast<const Vertex*>(reBuffer);

Second a range-based for-loop needs to know where the range ends and giving it a raw pointer does not provide that information, so you cannot use it here.  Instead you will need to know how many vertices there are and use a normal for-loop:
const size_t numVerts = 32;
for (size_t i = 0; i < numVerts; ++i) {
    const Vertex& vert = vertices[i];
    ...
}


Too soft shadows from multiple suns I want to render the transient shadows due to the sunlight over a day in one image. So I placed all the suns, as directional light, in the scene. However the shadows in the rendered image(a raytraced image) are too bright and so they look very much faded(below image). This is understandable as there is just too much light from all those suns.

Basically I am trying to achieve an image similar to the below one, where all shadows are distinguishable and not that soft. However I cannot think of any way to achieve this. I appreciate anyone's help.

The problem is that if you have 8 light sources (as your test image appears to have), then anything that is under only one shadow will be 7/8ths of full brightness, which is a fairly small change. Anything other than that is unphysical, so it'll be hard to convince your raytracer to produce a different result by changing the lighting.
Instead, after rendering the image, you can apply some kind of nonlinear processing of the image colors so that that 1/8th difference stands out more. In your second image, this seems to have been done by directly replacing the light gray fully-lit areas with white, but a more proportionate, yet still very visible, result could be achieved with a simple "levels" color edit applied to the whole image to increase contrast, as I've done for this version of your first image:

By pushing light gray to white, and darker grays more to black, the contrast in the image is greatly increased. The cost is that the building shapes are dark and hard to see.
To make the buildings look better and stand out, you could render your buildings a second time without any ground (so the resulting image has a transparent background), then layer that on top of the color-edited image, so that they are unaffected by the contrast adjustment.

It might be a good idea to use color to help distinguish the buildings from the shadows. In particular, I'd suggest avoiding making buildings gray (or, more generally, colored the same as the sunlight). Here I've used a "color temperature" edit on the above to make the buildings layer bluer and the shadowed ground layer yellower (probably too much):

Of course, this is all rough examples based on quickly editing your sample image; I think you can produce very legible and elegant results by taking these ideas and doing them better (by being able to adjust your raytracer lighting as well as postprocessing the image, and rendering a separate buildings-without-ground image as I suggested).


Generation of lower resolution textures So I have a game and I am trying to fake glow inside of it by using the hardwares capability to downsize textures causing them to blur.

The texture I wish to do this with is in a render buffer that is the size of the screen (not power of two)

What would be the quickest way to get these lower resolutions created?
Tricking the software to make minimaps?
Another fbo with the texture rendered on it in smaller size?

I would think rendering to a downsampled FBO would be much faster than generating mipmaps.  glGenerateMipmap will generate the complete mipmap pyramid, which seems wasteful:

https://www.khronos.org/opengles/sdk/docs/man/xhtml/glGenerateMipmap.xml

Besides, only OpenGL ES 3.0 lets you manually select mip level when texturing:

https://www.khronos.org/opengles/sdk/docs/man3/html/textureLod.xhtml

Artistically, this might not give you the look you want.  In my experience, simply downsampling gives a grainy look.  I would recommend also doing a pair of one-dimensional blurs, as described in this article from GPU Gems:

http://developer.download.nvidia.com/books/HTML/gpugems/gpugems_ch21.html


Renderbuffer needs to be bound before glFramebufferRenderbuffer? The documentation of glFramebufferRenderbuffer only says that renderbuffertarget needs to be GL_RENDERBUFFER, though it does not specify that the attached renderbuffer, that is also given to the function, needs to be bound to GL_RENDERBUFFER beforehand.

I deem it not having to be so, because why would I have to specify the renderbuffer again if it should be bound beforehand?

Especially since the glNamedFramebufferRenderbuffer was introduced, it should not be nesseccary to bind anything.

You must fully create a renderbuffer before you can attach it to an FBO.

The pre-DSA OpenGL Object creation functions do not create objects; they create object names. It's like creating an uninitialized pointer variable; it doesn't point to anything yet.

The actual creation of the object happens the first time the name is bound to the context. At that moment, the name is given a real object.

Under normal circumstances, it was pretty much impossible to use an object without binding it to the context, so this distinction couldn't be noticed. However, once the ability to attach objects to one another came to be, this distinction became noticeable.

Pre-DSA, the only way to fully create a renderbuffer object was to bind it to the context.

However, with ARB_DSA/GL 4.5, the glCreate* functions actually create objects (imagine that!). So if you use glCreateRenderbuffers, you can attach the renderbuffer to an FBO without binding it first.


Explanation of a the transparency of an .EXR file Simply put, what does the transparency data of an EXR file represent?

In an EXR, the convention is to store associated alpha, also known as premultiplied. The latter term however doesn't do justice to the nuances of the alpha format as compared to its evil distant cousin known as unassociated alpha, aka straight or key alpha.

In an associated alpha image, two facets are represented via the RGB triplet and its associated alpha channel:

Occlusion.
Emission.

In an associated alpha image, the alpha channel could be considered to represent a distillation of the geometry of occlusion covering a given pixel coordinate. 0.5 would represent a baked down 50% occlusion, while the RGB values would represent the degree of emission.

This also means that an RGBA triplet such as 0.0,12.4,10282.9,0.0 is entirely valid, and represents a pixel that is emitting and unoccluding. Naturally occuring things such as candles represent such a combination. Consider the following image:

The above image could not be created using unassociated alpha, and is symptomatic as to why ray tracing renderers cannot generate any alpha model other than associated alpha.

The fundamental difference between the two alpha formats is defined by the two different over operations, as described by Porter Duff.

Associated alpha over:

FG.RGB + ((1.0-FG.Alpha) * BG.RGB

Unassociated alpha over:

(FG.Alpha * FG.RGB) + ((1.0-FG.Alpha) * BG.RGB

The nuances of the difference mean that care should be taken when dealing with alpha and the RGB channels.


Multiple VAO share a VBO Let's say I have a have 10 different terrains. Each terrain have the same x and z coordinates, but they have different y coordinate, normals and colors. The indices used for element rendering is also the same.

The data looks like this:

std::vector<GLfloat> xzCoords;
std::vector<GLuint> indices;

std::vector<GLfloat> yCoords[10];
std::vector<GLfloat> normals[10];
std::vector<GLfloat> colors[10];

And the VBO and VAO IDs:

GLuint VAO_IDs[10];
GLuint VBO_xzCoord_IDs[10];
GLuint VBO_yCoord_IDs[10];
GLuint VBO_indices_IDs[10];
GLuint VBO_normals_IDs[10];
GLuint VBO_colors_IDs[10];

If I want one VAO for each of the terrain deffinitions, how would that look if the VBO for xz coordinates and indices are the same for each?

What I am doing now looks like this:

Setting up the buffers:

for (size_t i = 0; i < 10; i++){
        //VAO
        glGenVertexArrays(1, &VAO_IDs[i]);
        glBindVertexArray(VAO_IDs[i]);

        //VBO
        glGenBuffers(1, &VBO_xzCoord_IDs[i]);
        glGenBuffers(1, &VBO_yCoord_IDs[i]);
        glGenBuffers(1, &VBO_indices_IDs[i]);
        glGenBuffers(1, &VBO_normals_IDs[i]);
        glGenBuffers(1, &VBO_colors_IDs[i]);

        //xz coords
        glBindBuffer(GL_ARRAY_BUFFER, VBO_xzCoord_IDs[i]);
        glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat)*xzCoords.size(), &xzCoords[0], GL_STATIC_DRAW);
        glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 0, (void*)0);
        glEnableVertexAttribArray(0);
        //y coords
        glBindBuffer(GL_ARRAY_BUFFER, VBO_yCoord_IDs[i]);
        glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat)*yCoords[i].size(), &yCoords[i][0], GL_STREAM_DRAW);
        glVertexAttribPointer(1, 1, GL_FLOAT, GL_FALSE, 0, (void*)0);
        glEnableVertexAttribArray(1);
        //normals
        glBindBuffer(GL_ARRAY_BUFFER, VBO_normals_IDs[i]);
        glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat)*VBO_normals_IDs[i].size(), &VBO_normals_IDs[i][0], GL_STREAM_DRAW);
        glVertexAttribPointer(2, 1, GL_FLOAT, GL_FALSE, 0, (void*)0);
        glEnableVertexAttribArray(2);
        //colors
        glBindBuffer(GL_ARRAY_BUFFER, VBO_colors_IDs[i]);
        glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat)*VBO_colors_IDs[i].size(), &VBO_colors_IDs[i][0], GL_STREAM_DRAW);
        glVertexAttribPointer(3, 1, GL_FLOAT, GL_FALSE, 0, (void*)0);
        glEnableVertexAttribArray(3);

//indices
        glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, VBO_indices_IDs[i]);
        glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(GLuint)*indices.size(), &indices[0], GL_STATIC_DRAW);
    }

Rendering function:

for (size_t i = 0; i < 10; i++)
    {
        glBindVertexArray(VAO_IDs[i]);
        glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, VBO_indices_IDs[i]);
        glDrawElements(GL_TRIANGLES, indices.size(), GL_UNSIGNED_INT, (void*)0);

    }

So as you can see, I create a new xz-coordinates and indices buffer for each VAO. I doubt this is the most effective way to do it. How should I do this?

VAO can share VBO's because they do not store vertex data itself but references to Vertex Buffer Objects.

So you can first generate your buffers and upload your buffer data (vertex data and index data) to currently bound buffers.

glGenBuffers(1, &VBO_xzCoord_IDs[i]);
glGenBuffers(1, &VBO_yCoord_IDs[i]);
glGenBuffers(1, &VBO_indices_IDs[i]);
glGenBuffers(1, &VBO_normals_IDs[i]);
glGenBuffers(1, &VBO_colors_IDs[i]);

//xz coords
glBindBuffer(GL_ARRAY_BUFFER, VBO_xzCoord_IDs[i]);
glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat)*xzCoords.size(), &xzCoords[0], GL_STATIC_DRAW);

...

//indices
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, VBO_indices_IDs[i]);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(GLuint)*indices.size(), &indices[0], GL_STATIC_DRAW);

And then generate your VAO's and connect vertex array object data index you provide in glVertexAttribPointer call with currently bound GL_ARRAY_BUFFER.

The GL_ELEMENT_ARRAY_BUFFER can be a little confusing because it doesn't require glVertexAttribPointer call. So how it is connected with VAO? It seems that binding the buffer ties it to the currently bound VAO.

//VAO
glGenVertexArrays(1, &VAO_IDs[i]);
glBindVertexArray(VAO_IDs[i]);

//xz coords
glBindBuffer(GL_ARRAY_BUFFER, VBO_xzCoord_IDs[i]);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 0, (void*)0);

...

//indices
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, VBO_indices_IDs[i]);

glBindVertexArray(0);

And in your rendering call you don't need to bind GL_ELEMENT_ARRAY_BUFFER because it was already connected with VAO.


Alpha blending between two overlapping semi-transparent shapes I am working on a 2D lighting system for my game. My approach has been to:

Create a transparency quad representing 'darkness'.
Create a stencil shader for light sources to clip from the darkness quad.
Create a shader that blends alpha from 0 -> {darkness alpha} to soften the edges of the light source.

This looks like the following:

You can see the problem - the two light source shaders that soften the shadow edges are being blended like normal transparency layers should and causing the colour to intensify. Here is another image without the darkness quad, which might make things a bit clearer:

My question: Is there any sort of alpha blending I can do to make this intersection appear 'normal' i.e. it doesn't intensify the color? If not, is there another approach which could help with this?

Update:

Here is the shader code that generates the fade on the light source - note that the _Radius is set for each light source.

Properties
{
    _Color("Bottom Color", Color) = (1,1,1,1)
    _Radius("Radius", Float) = 1
}

SubShader
{
    Tags
    {
        "RenderType" = "Transparent"
        "IgnoreProjector" = "True"
        "Queue" = "Transparent"
    }

    Cull Back
    Blend One OneMinusSrcAlpha

    Pass
    {
        CGPROGRAM
        #pragma vertex vert
        #pragma fragment frag

        #include "UnityCG.cginc"

        struct appdata
        {
            float4 vertex : POSITION;
        };

        struct v2f
        {
            float4 vertex : SV_POSITION;
            float distance : float2;
        };

        fixed4 _Color;
        float _Radius;

        v2f vert(appdata v)
        {
            v2f o;
            o.vertex = UnityObjectToClipPos(v.vertex);
            o.distance = distance(float3(0, 0, 0), v.vertex);
            return o;
        }

        fixed4 frag(v2f i) : SV_Target
        {
            float distance = (i.distance / _Radius); // 0 -> 1 as progressing to outside
            float normalizedDistance = 1 - distance; // 1 -> 0 as progressing to outside

            // 0 -> 0.43
            fixed colorOffset = lerp(0, _Color.a, distance);

            // return alpha
            return fixed4(0, 0, 0, colorOffset);
        }

        ENDCG
    }
}

I used Stencil Buffer to fixing your problem , you need a way for checking overlapping two or more shapes

 Shader "Custom/SemiTransparent"
{
    Properties
    {
        _Color("Color",Color) = (0,0,1,0.1)
    }
    SubShader
    {
Tags {"Queue"="Transparent" "IgnoreProjector"="true" "RenderType"="Transparent"}
ZWrite Off Blend SrcAlpha OneMinusSrcAlpha Cull Off

        LOD 100

        Pass
        {
            Stencil {
                Ref 0
                Comp Equal
                Pass IncrSat
                Fail IncrSat
            }

            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            #include "UnityCG.cginc"

            fixed4 _Color;

            struct appdata
            {
                float4 vertex : POSITION;
            };

            struct v2f
            {
                float4 vertex : SV_POSITION;
            };

            v2f vert (appdata v)
            {
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                return o;
            }

            fixed4 frag (v2f i) : SV_Target
            {
                fixed4 col = _Color;
                return col;
            }
            ENDCG
        }
    }
}


Radiosity VS Ray tracing Radiosity is basically what allows this:

In a tutorial of Cornell University about Radiosity it is mentioned that:

A ray-traced version of the image shows only the light reaching the viewer by direct reflection -- hence misses the color effects.

However in Wikipedia:

Radiosity is a global illumination algorithm in the sense that the illumination arriving on a surface comes not just directly from the light sources, but also from other surfaces reflecting light.
...
The radiosity method in the current computer graphics context derives from (and is fundamentally the same as) the radiosity method in heat transfer.

And if ray tracing is capable of:

simulating a wide variety of optical effects, such as reflection (diffuse reflection) and scattering (i.e. the deflection of a ray from a straight path, for example by irregularities in the propagation medium, particles, or in the interface between two media)

Has that tutorial not considered these effects or are there radiosity methods that can be used in ray tracing in order to enable them?
If not, couldn't these optical effects simulate radiosity entirely or is the radiosity algorithm more efficient in solving the diffuse reflection problem?

Radiosity does not account for specular reflections (i.e. it only handles diffuse reflections). Whitted's ray-tracing only considers glossy or diffuse reflection, possibly mirror-reflected. And finally, Kajiya's path-tracing is the most general one [2], handling any number of diffuse, glossy and specular reflections.

So I think it depends on what you means by "ray-tracing": the technique developed by Whitted or any kind of "tracing rays"...

Side-note: Heckbert [1] (or Shirley?) devised a classification of light scattering events which took place as the light traveled from the luminaire to the eye. In general it has the following form:

L(S|D)*E

"L" stands for luminaire, "D" for diffuse reflection, "S" for specular reflection or refraction, "E" for eye, and the symbols "*", "|", "()", "[]" come from regular expressions notation and denote "zero or more", "or", "grouping", "one of", respectively. Veach [3] extended the notation in his famous dissertation by "D" for Lambertian, "S" for specular and "G" for glossy reflection, and "T" for transmission.

In particular, the following techniques are classified as:

OpenGL shading: EDL
Appel's ray-casting: E(D|G)L
Whitted's ray-tracing: E[S*](D|G)L
Kajiya's path-tracing: E[(D|G|S)+(D|G)]L
Golar's radiosity: ED*L

[1] Paul S. Heckbert. Adaptive radiosity textures for bidirectional ray tracing. SIGGRAPH Computer Graphics, Volume 24, Number 4, August 1990

[2] The Siggraph 2001 course "State of the Art in Monte Carlo Ray Tracing for Realistic Image Synthesis" says the following: "Distributed ray tracing and path tracing
includes multiple bounces involving non-specular scattering such as E(D|G)*L. However, even these methods ignore paths of the form E(D|G)S*L; that is, multiple specular bounces from the light source as in a caustic."

[3] Eric Veach. Robust Monte Carlo Methods for Light Transport Simulation. Ph.D. dissertation, Stanford University, December 1997


Double rasterization work on triangles edges. I know that fragments are rasterized in a 2x2 quad. That's why the seam between two triangles can be rasterized twice. At least that article states this. I wonder is it still true today on modern hardware?

That's why the seam between two triangles can be rasterized twice.

The seam won't be rasterized twice because of the OpenGL or D3D fill/rasterisation rules.

What I think Humus is alluding to is that a fragment shader may be executed multiple times on pixels spanning the shared edge, which just implies wasted effort.

The moral of the story is "avoid small or long/thin triangles if you want efficient rendering".


Ray tracing - tangent space for a point on a sphere In a ray tracer, given a point on a sphere (point_of_intersection with a ray) and its normal for that point (point_of_intersection - center_of_sphere) how do I calculate the tangent space for that point? Do I need other data to calculate the tangent space?

The tangent space is spanned by the tangent to the point and the bitangent (which is orthogonal to both tangent and normal).

So you need to calculate the tangent which is achieved by calculating the cross-product of the ray-direction and the normal. $T = N \times DIR$
The resulting vector will be orthogonal to the normal and thereby be the tangent.

Now calculate the cross-product of the tangent and the normal $BT = T \times N$ to create a vector orthogonal to both. This vector is the bitangent.

Tangent $T$ and bitangent $BT$ span a plane which is the tangent-space of your intersection-point.


Rounding a corner formed by Arc and Line Given shapes like these, which contain just Arcs and lines, I want to round all its corners with certain radius. I know the vertices of the lines & arcs these shapes.

I can round the corners of a polygon (when two lines meet to form a corner), but can't figure out how to round a corner formed by an Arc and a Line.

I used this post to round the corners of polygon:

Algorithm for creating rounded corners in a polygon | Stack Overflow

Could you please share a code snippet, or just explain/list the math/algorithm behind achieving this? Thanks!

PS: I'm developing a CAD drawing generator using C# & .netDxf library

I suppose you want an arc of C0 and C1 continuity between the line and an arc.

As illustrated above, you already have a vertex A which is the intersection of an edge and an arc of which the center positioned at O and radius equal to R. The question is thus pure mathematical: given A,O,R, edge direction BA, and a corner radius r, find C,B, and T.
For simplicity, here let's assume that the black dotted line has the form of $C_y=kC_x+b$, where k is the same as edge BA while b is easily derived by combining k and rounding radius r. Therefore, three constraint equations can be listed to solve $C_x$:

The distance TC is r: $(T_x-C_x)^2+(T_y-C_y)^2=r^2$
The big arc equation: $(T_x-O_x)^2+(T_y-O_y)^2=R^2$
Point C is on OT: $\frac{T_y-C_y}{T_x-C_x}=\frac{T_y-O_y}{T_x-O_x}$

Solve for Cx and you'll obtain four solutions (I used Mathematica):

$C_x=\frac{-bk+O_x+kO_y\pm\sqrt{-(O_y-kO_x-b)^2+(1+k^2)(R-r)^2}}{1+k^2}$
$C_x=\frac{-bk+O_x+kO_y\pm\sqrt{-(O_y-kO_x-b)^2+(1+k^2)(R+r)^2}}{1+k^2}$

The second expression is out of the arc, so you should use the first solutions. Given the two Cx values, they can be used to derive Cy values to make a candidate list for point C. Pick the correct C using manual check.

Now you have the center C of the round corner arc as well as its radius r. It should be relatively easy to determine point B and point T, and draw an arc through the two points.


Zoom in orthographic vs perspective projection I have read through this article about perspective and orthographic projection matrices.

I started playing with the perspective matrix and as expected if I either increase/decrease the field of view i have a zoom out/in effect. Similarly, if I move the camera position forward/backwards I see the object being close or further away.

However, when I tried the orthographic matrix, since there is no field of view parameter, I tried only moving the camera forward/backwards but my model always looked the same size.

Can anybody explain to me why this is happening?

Perspective projection changes the size of an object as it's distance changes, while orthographic projection does not.  That is part of the definition of those projection types.

To simplify things a bit, a simple perspective projection of a 3d point to a 2d point can be calculated like this:

$x_{2d} = x/z\\
y_{2d} = y/z$

As you can see, the 2d version of the x and y change depending on the value of z.

Orthographic projection on the other hand can be calculated like this:

$x_{2d} = x\\
y_{2d} = y$

In that example, you can see that the x and y values do not depend at all on z, so you can move closer to or farther away from an object in orthographic projection and it's size will not change.

As a fun bit of related trivia, the sun is so far away that when the light gets here, it's all basically going in the same direction.

This is a reason why we have directional lighting, where light beams are always moving in the same way, because that is how sun light works.

Anyways, since the light beams are parallel it means that your shadows in sun light are orthographic projections of your silhouette.  In other words, as you move your hand closer to or farther from the sun here on earth, the shadow of your hand doesn't change size.

However, if you have a more local light source - light a lamp - it casts a perspective projection type of shadow, where the closer to the light source your hand is, the larger the shadow it casts.

This is tied to the definition of an orthographic projection, which is that the focal length is infinitely long.  The sun is far enough away that we can consider it infinity in lighting, for as much accuracy as we care about :P


Skybox is rendered on the model instead of surroundings I'm trying to implement Skybox in DirectX 11 (DirectXTK) but when I'm rendering skybox, then texture is rendered to my sphere model instead of showing skybox outside.

Rendering code:

m_Camera->Render();

m_Camera->GetViewMatrix(viewMatrix);
m_D3D->GetWorldMatrix(worldMatrix);
m_D3D->GetProjectionMatrix(projectionMatrix);

m_D3D->ChangeRasterizerCulling(D3D11_CULL_NONE);
m_D3D->ChangeDepthStencilComparison(D3D11_COMPARISON_LESS_EQUAL);

m_Model->Render(m_D3D->GetDeviceContext());
m_skyboxShader->Render(m_D3D->GetDeviceContext(), m_Model->GetIndexCount(), worldMatrix, viewMatrix, projectionMatrix);

m_D3D->ChangeRasterizerCulling(D3D11_CULL_BACK);
m_D3D->ChangeDepthStencilComparison(D3D11_COMPARISON_LESS);
m_D3D->EndScene();

Skybox.vs

cbuffer MatrixBuffer
{
    matrix worldMatrix;
    matrix viewMatrix;
    matrix projectionMatrix;
};

struct VertexInputType
{
    float4 position : POSITION;
};

struct PixelInputType
{
    float4 positionSV : SV_POSITION;
    float4 position : POSITION;
};

PixelInputType ColorVertexShader(VertexInputType input)
{
    PixelInputType output;

    input.position.w = 1.0f;
    output.position = input.position;

    output.positionSV = mul(input.position, worldMatrix).xyww;
    output.positionSV = mul(output.positionSV, viewMatrix).xyww;
    output.positionSV = mul(output.positionSV, projectionMatrix).xyww;

    return output;
}

Skybox.ps

TextureCube shaderTexture;
SamplerState SampleType;

struct PixelInputType
{
    float4 positionSV : SV_POSITION;
    float4 position : POSITION;
};

float4 ColorPixelShader(PixelInputType input) : SV_TARGET
{
    return shaderTexture.Sample(SampleType, input.position.xyz);
}

Result

A skybox is a (large) cube that encompasses the entire scene and contains 6 images of a surrounding environment, giving the player the illusion that the environment he's in is actually much larger than it actually is.

In order to do this, you need to draw a Cube (or a Sphere) in your Scene that will represent the Skybox. You then need to sample the Skybox Texture (shaderTexture in your shader) to project it into the geometry.
This is exactly what you are doing with your Sphere Model in your Pixel Shader.

The problem you have, is that your Model is too small and your Camera is outside of the Skybox. You need to make the Skybox way bigger, and place your camera inside of it.
The best solution is to center the Sky Sphere about the camera in world space, this way when you move the camera around, the Sky follows the camera, giving the illusion that it's infinitely far away and you can't get closer.

PixelInputType ColorVertexShader(VertexInputType input)
{
PixelInputType output;

// Use local vertex position as cubemap lookup vector.
output.position = input.position;

// Transform to world space.
float4 positionWorld = mul(float4(input.position, 1.0f), worldMatrix);

// Always center sky about camera (You need to pass the camera position).
positionWorld.xyz += eyePositionWorld;

// Set z = w so that z/w = 1 (i.e., skydome always on far plane).
output.positionSV = mul(posW, viewProjectionMatrix).xyww;

return output;

}

To keep the Skybox fragments always on the far plane (z component = 1.0) you can set the z component to w. This will turn into 1 after the perspective division is applied.
With this little hack you can render the SkySphere after having rendered all your meshes, and they would all be rendered on top of it.

You may also want to do the matrix multiplication on the CPU and pass it in as another parameter for performance reasons.


If I have an nVidia graphics card on Linux, do I need Mesa or FreeGLUT or anything like that? The wiki says the following about using OpenGL on Linux:

  Most Linux distributions rely on the Mesa3D project to provide their OpenGL implementation

But it also says:

  OpenGL is included in drivers, so you'll have to make sure drivers are properly installed if you want to enjoy programs using OpenGL. Open source drivers actually make use of Mesa's OpenGL implementation.

So I am a little confused:

If I have an nVidia graphics card and install the nVidia drivers, do I need to install Mesa?

What if I have the card but do not install nVidia's drivers?

What, exactly, is meant by "implementation" in this context?

The Mesa3D project is a software implementation of OpenGL; that is, an implementation that does all the rendering on the CPU with no GPU involvement at all. It's designed in two layers: a backend which renders each frame from some internal state, and a frontend which implements the OpenGL API and sets up the state for the backend to use. Most OpenGL implementations have a similar two-layer architecture, so that the same backend can be used by an OpenGL frontend, an OpenCL frontend, a Direct3D frontend, etc.

The open-source NVidia drivers (and most open-source drivers) use Mesa's frontend to provide the OpenGL API and a backend specific to that GPU. The OpenGL API is quite complicated to implement, so it makes sense for vendors to share an existing front-end in this way. However, closed-source drivers do not use Mesa's frontend and have their own closed-source frontend.

If you install and use a closed-source GPU driver, it will include its own libgl, which doesn't use Mesa at all. You don't need to install anything else to run OpenGL applications. (You might need an extra "-dev" package to get the header files needed to compile OpenGL applications.)

If you install and use an open-source GPU driver, it will use Mesa, but the packages in your distribution should have the dependencies set up correctly, so that if an external Mesa package is needed, it will be installed automatically.

Either way, you shouldn't have to worry about whether the OpenGL implementation you're using is based on Mesa or not.


How was computer graphics libraries or gtk,qt programmed? I am wondering how was first graphic or image libraries created,I know C and Python ,I want to create plotting application,and I cant imagine these libraries(QT,OpenGL,Vulkan) written by C,how can I create my image library,I just want to create basic graphic library.

You're asking about two different types of library. OpenGL and Vulkan aren't just libraries: they're hardware interfaces. They are implemented by GPU vendors so that application developers can access the GPU, like a special kind of hardware driver. They're implemented in C or C++ and the core functionality is implemented by writing some structures in memory that the GPU itself can understand, then writing to registers on the GPU to make it read those structures and do the work. There isn't any C code in OpenGL to transform triangles, compute their projection onto the screen, and shade them, because that work is all done by the GPU.

OTOH, Qt and Gtk are libraries for making applications. A lot of their work isn't about graphics: they provide a framework for applications to manage themes and interactivity, letting the application developer wire together different parts of the application with callbacks. This is just normal library code. They also provide GUI classes such as buttons and checkboxes. These will each have something like a "draw" function which draws the graphics on the screen. A few years ago this would commonly have been implemented using the OS's 2D drawing API (e.g. GDI on Windows), which offers operations like "draw a filled rectangle", "blit an image", but nowadays they're more likely to use OpenGL or Vulkan operations. The UI components are defined in images and they're used as textures to generate the image of the whole window via a sequence of OpenGL draw calls.

You wouldn't implement OpenGL or Vulkan: this is something that GPU vendors create. You can't really do it yourself without detailed knowledge of the GPU hardware. (However, Mesa is an example of a software implementation of OpenGL, which doesn't depend on any GPU.)

If you wanted to implement a GUI library like Qt, you might start by defining an abstract Widget class with a draw method, and then implementing concrete subclasses of it to draw particular GUI components. Add extra methods and functionality as you need it.

To be honest, I don't think that would be a valuable project to work on. There are many smaller GUI frameworks already, and another toy one is not going to improve the world. You might treat it as just a learning project, but I think you'd learn more by studying one of the existing frameworks and writing an interesting GUI application on top of it.


Rotate line around center I have to use a propriertary graphics-engine for drawing a line. I can rotate the whole drawing by its origin point (P1). What I want, is to rotate it around its center point(M). So basically that it looks like the green line (L_correct instead of L_wrong).
I think, it should be possible to correct it, by moving it from P1 to P2. But I cannot figure out what formula could be used, to determine the distance. It must probably involve the angle, with and height... Can anyone give me a hint?

Trick is, to move the entire object so that the point about which you want to rotate is at the center. Then rotate and after that counter move it so that the point is were it was.

In fact this is not so much of a trick, as such, nearly all graphics engines work this way. It is just abstracted away in many cases. Most often you will see it done in matrix notation like this:

$$ T^{-1} R T $$

Where T is to transformation from world to pivot and R is the rotation matrix. Since it can be easily packaged this way you end up with a function called rotate about the pivot point. This is usually the reason why drawing API's allow for a transformation matrix.

Image 1: Rotate about a pivot point, image source available for investigation. Sub images in RTL reading order: Original, inverse transform, inverse and rotate, inverse rotate and move back.