Is vkCmdPushDescriptorSetKHR efficient? I am new to Vulkan. Compared to DX12, the resource binding procedure seems a little bit complicated. In terms of per draw call uniform buffers, I currently have two approaches in mind, and they both seem to work (haven't tested/benchmarked yet). Can someone shine some light on which is the standard way to do this, with pros and cons. You are welcome to provide other ways but I am not considering push constants because I want a universal approach that I can apply to other kinds of uniform buffers as well (i.e. per pass, per frame, per scene and etc.).

approach A) One descriptor set and multiple buffers (one buffer for each object). Bind the descriptor set to pipeline only once before rendering. Bind buffers to the only descriptor set between draw calls using vkCmdPushDescriptorSetKHR.

approach B) Multiple descriptor sets (one descriptor set for each object). (Multiple or single buffers, it doesn't really matter. If single buffer is used, I think it's called a dynamic uniform buffer and all that I need to do is to specify the offset when binding the descriptor set.) Bind buffer/buffers to descriptor sets only once before rendering. Bind the matching descriptor set to the pipeline between draw calls using vkCmdBindDescriptorSets.

I am used to DX12 so approach A seems more natural because in my understanding vkCmdPushDescriptorSetKHR in Vulkan is just like SetGraphicsRootConstantBufferView in DX12, because they both bind buffer to pipeline and this operation is buffered with other commands. But the Vulkan one do feels a little bit slower mainly because 1) it is not a root descriptor, 2) Vulkan does not use GPU address and also 3) a descriptor set write operation is needed.

What you're ultimately trying to do is, between draw calls, change some state that allows one to select which set of read-only data to use in the rendering process. There are many techniques for this.

Push constants can be used to provide an index into an array of arbitrary size stored in an SSBO. Indeed, a single push constant can be used to provide multiple indices into multiple SSBOs, for different frequencies of state (per-frame vs. per-object). The upper 16-bits can be for one frequency, with the lower 16-bits for another.

That may sound slower than other methods, since you're accessing global data rather than an actual UBO. But on some hardware, it won't change anything. AMD hardware for example doesn't actually have any specialized hardware for UBOs, so they implement UBOs as just read-only SSBOs. Even on hardware with dedicated UBOs, the state data you're reading will likely be quickly cached, since all invocations will be reading the same values from the array. So unless your thrashing the SSBO caches, this shouldn't impact performance much.

Dynamic UBO descriptors also exist, given the VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER_DYNAMIC type. Basically, the idea is that, when you call vkCmdBindDescriptorSets, you provide a dynamic offset for any dynamic descriptors. The offset represents a byte offset from the buffer attached to that descriptor to where the UBO data you want to fetch lives. So you would have all of your data in a large buffer, and you would select which particular data you're using for a particular render call by invoking vkCmdBindDescriptorSets with a different dynamic offset.

Now vkCmdBindDescriptorSets may seem like a more heavyweight call than setting push constant. But the implementation has all the information needed to be able to see that you are using the same descriptor as you used in the previous call in that command buffer, and thus not do the actual heavy part of that operation, simply applying the dynamic offsets to any dynamic descriptors. So that part should not be a concern.

Push descriptors are mainly intended for texture/sampler/image bindings, which don't have a convenient mechanism like dynamic offsets that would allow you to quickly switch which image/sampler you're using. That being said, push descriptors are a useful concept even for UBO/SSBOs. Whether they are faster than dynamic descriptors is a matter of profiling.

The principle downside is the obvious: push descriptors is an extension. Dynamic UBO offsets and push constants are mandatory, required by every Vulkan 1.0 implementation. Push descriptors aren't even optional features of Vulkan. They're important enough to get a KHR extension, but they didn't go into Vulkan 1.1 as an optional feature. This is not a theoretical issue: NVIDIA and Intel seem to support it on desktop, but no official AMD drivers do.

So I would go with dynamic UBOs; this is exactly what they were made for, after all.


How to do linear interpolation? Here in my book they have shown interpolation using parametric equation of line but I am unable to figure out how do they found out t=|y1-ys|/y1-y2 , t=|AD|/|AB| , t=|CE|/CB ,t=|EP|/|DE| here is the  pic

Since this is linear interpolation it boils down to solving a linear equation
$y = a + b t$
You can substitute the following $y = y_s$, $a = y_1$(intercept) and $b = y_2 - y_1$ (slope) to get the following
$$y_s = y_1 + t(y_2 - y_1)$$
$$y_s - y_1 = t(y_2 - y_1)$$
$$\frac{y_s - y_1}{y_2 - y_1} = t$$
The same thing happens for points where you can write
$$D = A + (B-A)t$$
$$ (D-A) = (B-A)\cdot t$$
$$ (D-A)^2 = ((B-A)\cdot t)^2$$
$$ \sqrt{(D-A)^2} = \sqrt{((B-A)\cdot t)^2}$$
$$ \sqrt{(D-A)^2} = \sqrt{(B-A)^2 \cdot t^2}$$
$$ \sqrt{(D-A)^2} = \sqrt{(B-A)^2} \cdot\sqrt{ t^2}$$
$$ |D-A| = |B-A| \cdot t$$
$$ t = \frac{|D-A|}{|B-A|}$$


Is back-face culling considered a technique/optimization of ray tracing? I've stumbled upon this question on a CG exam:

Which of the following techniques/optimizations doesn't apply to ray tracing?
A: Back-face culling
B: "Shadow-feeler" rays
C: Recursive tracing of rays
D: Calculation of refraction rays
E: None of the above

B, C and D are obviously false, but what about A? Here, is says that "with ray tracing, this feature is not as useful. Generally with ray tracing, we want geometry in the scene to cast shadows for example, regardless of the orientation of the object's surface with respect to the ray direction however the backface culling option might still be desired for primary rays".
I feel that the question is a bit ambiguous, since back-face culling can be applied to ray tracing but it's not (correct me if I'm wrong) a "native" feature as the other techniques.

I would exclude (A) for the following reasons:

In any modern ray-tracer, the acceleration structure (BVH, k-d tree, etc) holding your geometry is the primary source of culling. The final list of primitives to trace for a given ray may include non-visible triangles, but ideally there should be very few with good acceleration structures.
Ray tracers support many more primitives than just triangles. (How do you back-face cull a sphere?)

The following problems with back-face culling in general are also more relevant to ray tracing:

Transparent materials (such as glass) will make certain objects render incorrectly with back-face culling applied.
Non-closed meshes (like plants) need to render the back-facing triangles.


Possible methods to deal with artifacts when drawing billboards to a cubemap When drawing billboards to a cubemap, the billboard texture was incorrectly aligned on the boundaries of the cubemap between different sides.

My guess is that the false projection of the billboard texture shape using the perspective cameras when drawing the cubemap caused this.

My question is that without using a geometry with dimensions (such as using a plane geometry for the billboard texture to attach) to achieve the correct projection, is there any other methods to reduce the artifacts? e.g. process on the pointCoord?

a screen record showing the matter

Change the projection to always be the same no matter which face is being rendered.

The simplest way is you use the direction from billboard to the camera instead of the direction the camera is facing to rotate the billboard.


How RGB images are formed when normalized between 0 and 1 There is a popular technique of normalising images by dividing each pixel with 255 so that it comes in the range of 0 and 1.
But, then how the same colours are formed when we normalize images.
for instance, there is a pixel with value 0 1 1 and other 0 255 255, I checked on RGB colors chart, both are showing different colors, but if we normalise the second pixel values then its value will be same as the first one and it should produce different results but on plotting the set of normalized pixels, they come out to be same as the previous one.

I have previously asked this question - how RGB images are formed
so deriving out conclusions from the above question.

Let's suggest an analogy. I have a distance of 1 centimeter and 2.54 centimeters. If I transform the latter distance to inches, then it's 1 inch. This has the same value as the first distance.

But 1 inch is still longer than 1 centimeter, no matter the fact that they both use the numeral 1. These numbers are attached to different units, so simply comparing their numerical values is meaningless.

So it doesn't matter if the 8-bit unsigned, normalized value of 255 happens to have the same value of 1. Because you cannot compare them when they are in different spaces like that. Your original values were both in 8-bit normalized, unsigned integer space. If you want to compare them in a floating-point space, you have to transform both of them.


What does GPU assembly look like? I have played around with CPU assembly programming like Nasm, Tasm or Masm, but I'm really curious to know how GPU works now.
However, i'm quite confused when I look on internet. I've heard about Cuda and OpenCL, but it is not what I'm looking for.
I'd like to know how GPUs instructions are in RAM... What are the Nasm and Masm for most GPUs? What is the x86 or Z80 of GPUs (What are the different families of GPU)? Do you know a constructor opcodes reference manual?
I think I really need something to compare between the two Processing Units to make it clear, because GPU assembly programming seems to be an even harder subject to learn from on internet that CPU asm programming. I've also read that "NVIDIA never released details on the instructions actually understood by their hardware", but it seems pretty surprising to me. Full post there: https://stackoverflow.com/questions/4660974/how-to-create-or-manipulate-gpu-assembler?newreg=e31519279ce949f087df6322dbf2bf4d
Thanks for your help!

You're tilting at windmills trying to learn "GPU assembly", and it's due to the differences between how CPUs and GPUs are made and sold.

Each CPU has what's called an instruction set architecture, for example x86 or ARMv8. The instruction set is the interface between the user of the CPU (i.e. the programmer) and the chip. The chip designer publishes the details of the instruction set so that compiler vendors can write compilers to target that instruction set. Any CPU that uses that instruction set can run the same binaries. (Extensions like SSE make that slightly untrue, but they only add new instructions: they don't change the structure of the binary.) When the vendor creates a new processor family, it could have completely different micro-architecture internally, but the same instruction set.

GPUs are not like this at all. For best efficiency, the instruction set is usually closely tied to the micro-architecture of the CPU. That means each new family of GPUs has a new instruction set. GPUs can't run binaries made for different GPUs. For this reason, the instruction set usually isn't published: instead, your interface to the GPU is the driver published by the vendor for each graphics API (OpenGL, Vulkan, DirectX, &c.). This is why the graphics APIs have functions to take the source code of a shader and run it: the compiled shader only runs on the same model or family of GPU it was compiled for.

The closest you get to GPU assembly language is SPIR-V. This is an industry-standard intermediate representation, which GPU vendors are starting to support. It's like a GPU equivalent of LLVM's intermediate representation. It allows you to do the parsing and source optimization parts of compilation up-front to get a SPIR-V file, and then the GPU driver only needs to compile that to the GPU's instruction set at load time.


Image saved as DDS with 16f format in photoshop has strange FOURCC text I am analyzing the hex code for DDS files which are compressed via Nvidia's dds plugin in photoshop.

I noticed something strange when having a 16 bit float file. This is the dialog:

When I look at the header of this file it appears to not follow the DDS spec. According to the DDS docs for the pixel format (https://docs.microsoft.com/en-us/windows/desktop/direct3ddds/dds-pixelformat)
the dwFlags of 0x4 indicates a FOURCC but the next 4 bytes, which usually contain the FOURCC like "DXT1" etc contain only a "o"

AFAIK the for various other formats the locations and FOURCC (when present) are correctly displayed.
Here is a luminance 8 (dwFLags to 0x20000 and no FOURCC)

and here is a DXT1

Could somebody hint what happens for 16f here?

The tool seems to be generating an unofficial extended version of DDS in which the FOURCC code is replaced by a value from the D3DFORMAT enum. The code 0x0000006F translates to decimal 111, which translates to D3DFMT_R16F.

The Microsoft DDS documentation notes that this is seen sometimes, although not recommended: DDS Variants

  There are some common variants in use where the pixel format is set to a DDPF_FOURCC code where dwFourCC is set to a D3DFORMAT or DXGI_FORMAT enumeration value. There is no way to tell if an enumeration value is a D3DFORMAT or a DXGI_FORMAT, so it is highly recommended that the "DX10" extension and DDS_HEADER_DXT10 header is used instead to store the dxgiFormat when the basic DDS_PIXELFORMAT cannot express the format.


Swap buffers twice does not show the first scene If you clear the color buffer and draw a triangle, you have nothing in the buffer that is shown in the screen and a triangle in the other buffer, so you have to swap buffers to show the triangle. If you swap buffers, you have a triangle in your screen and a empty color buffer in the other side. If you swap buffers again you should see and empty screen because you changed the buffer with the triangle by the other empty buffer. Instead of this, the triangle is shown again. Why?

If you swap buffers, you have a triangle in your screen and a empty color buffer in the other side.

No, you do not. Buffer swapping does not necessarily mean that the current contents of the front buffer are preserved. When you do a swap, the contents of the back buffer become the front buffer, but the state of the back buffer is undefined after the swap.

Some implementations use a true swap, where the two buffers really are switched. Other implementations copy the data from the back buffer on a swap, leaving the back buffer as whatever was there before. Still others could do something else.


What is bilateral upsampling? I've worked with various interpolation algorithms including linear, cubic (and bilinear and bicubic), other bezier (and nurbs, etc) based interpolations, lagrange interpolation and some others, but I haven't been able to find any basic info that explains what bilateral upsampling is.

I have been able to find information on Joint Bilateral Upsampling which as best as I can tell interpolates multiple pieces of data in parallel, using hints from each to help the other interpolate better, but am having trouble separating the info of BU and JBU.

Is anyone able to explain the basic ideas of what bilateral upsampling is about?

It seems like you're asking two things.  I can't really speak technically about JBU, but I can give an overview of the necessary concepts and bilateral filtering generally.  You'll probably need to find more details yourself, but this should give a coherent structure to start from.

Fixing "Image"s
Many image-processing people view filtering as either something to be done as a post-process, or as a way to scale images.  The mathematician's perspective is more accurate, descriptive, and complex.
When you have an image file, you don't have a real image.  You have pixel "values", which are strictly valid only at the center of pixels.  Roughly speaking, each pixel has an area that takes on that color.  Mathematically, what you're doing is using a nearest-neighbor reconstruction filter to reconstruct the true image from pixel values.  So an "image" is really a set of samples and a reconstruction filter, typically nearest-neighbor.
Resampling
When you upscale an image, what you're really doing is resampling.  How do you do this?  You take your signal, and generate a new set of samples from it.  But for an image, I just said we don't have the real image; we have to reconstruct it first.  So a typical resampling procedure looks like this:

Take your pixel values.
Reconstruct an image between the values, which live at pixel centers, by using reconstruction filtering.
Measure a new set of samples from the reconstructed image, possibly spaced differently.
Throw away the old samples and use the new ones.

The key point is that the reconstruction filter used is the "filter" used for resampling.  For example, a "bicubic upsample" would use a bicubic reconstruction filter.
It turns out that if you use a sinc function for the reconstruction filter, and you don't shrink the image, you get (theoretically) the same image back.  In fact, if you upscale the image and then downscale it again, you'll get the same answer.  This isn't generally true for any other filter.
Most filters approximate a sinc filter to more or lesser degrees.  Deviations result in image loss.
Filtering
If you use the same before/after sampling rate, you do something called "filtering".  Formally, filtering is resampling without a change in sample rate.  Filtering is a special case of resampling.
The only possible purpose to filtering it to throw away information (e.g. a Gaussian) or possibly also to distort or add spurious information (e.g. a Sobel).
Bilateral Filtering
You can use many different filters for filtering.  Bilateral filtering is one of them.  I don't believe it's considered state-of-the-art for any purpose now (though I don't know what is; probably some newfangled machine-learning thing I expect).
A bilateral filter is a non-linear filter, meaning it doesn't have a nice Fourier representation.  Conceptually, it is like a Gaussian, but the source colors themselves also factor into a pixel's "distance" from others.  This tends to smooth flat areas while preserving edges.
Bilateral Upscaling
By now, I hope you see where I'm going with this: any filter you can use for filtering, you can use for upscaling or downscaling, which are really the same thing (because all of it is resampling).
If you've implemented a bilateral filter, you should be able to rather simply implement a bilateral filter resampler - just as, if you've implemented a Gaussian filter, you should be able to implement a Gaussian filter resampler.
The only difficulty may be the definitions.  Unfortunately, the distinction between filtering and resampling is systemically muddied to occasional non-existence in the image-processing literature.  For implementing a bilateral-type filter, I would rely heavily on its similarities to Gaussian resampling.  IIRC (I haven't read the paper in years) bilateral filtering is posed in terms of pixels, not samples.


Are draw calls executed in parallel or sequentially or both? This a follow up for a question I answered on GameDev SE. The question was simply Is glDrawArraysInstanced in OpenGL parallel when drawing those instances?

My answer was in the lines of "the GPU might execute multiple draw calls in parallel depending on multiple things, for instance if two draw calls use the same vertex shaders." Here is the full answer.

In the comments the OP mentioned, "I think it should be sequential. After all, some of blending mode depends of sequence"

Now, I am little bit confused. Is my answer of it will still parallelize them when it can valid? Can someone clarify this?

The result should be as if it was executed sequentially one triangle at a time. This is important so that each frame is deterministic. If it wasn't then drawing the same frame multiple times could create different results and create a flickering image on screen.

However that does not mean parallelization is impossible. Vertices can be computed in parallel. Their results will be ordered and fed to the rasterizer which will feed the fragment shader stage.

The same happens with pixels. Typically a group of pixels from a triangle can be computed in parallel and blended. A non-overlapping triangle can then also be filled at the same time however if triangle overlaps one currently busy then the result of the fragment shader must be buffered until the corresponding pixels of the previous triangle are done so blending can happen in the correct order.


Register pressure in Compute Shader I'm in the process of writing a Ray Tracer using DirectCompute / HLSL. First, eye rays are generated (one per pixel). Then, rays are traced, shaded and reflected in a loop. Also, shadow rays for each light source are generated and tested for occlusion. As a scene structure, I'm using a KD Tree with Ropes$^1$.

I have analyzed the shader with Joshua Barczak's Pyramid and it seems that I have a major problem with register pressure, mainly vectors. On a Fiji GCN architecture, the shader uses 85 SGPRs and 81 VGPRs, limiting the number of simultaneous wavefronts per SIMD to 3 (VPGR).

Furthermore, I have implemented a simple counter in my shader, that atomically increases a value when a thread starts, decreases it when a thread ends and keeps track of the maximum number of shaders that were concurrently running. I've managed to get rid of some data and bring it up from 8192 to 9216, resulting in a correlated speedup of ~%13. When I use an empty dummy shader, I get 16384 concurrent threads.

I've tried to get rid of excessive variables, especially vectors, by keeping their lifetime as short as possible, and also tried storing some variables in groupshared memory and reading / writing directly from there, all without any change in register count whatsoever.

Are there any practical tips on how to take pressure of the registers? Is this even as much of a problem as I think it is?

I have also thought about splitting my Ray Tracer into various kernels. This should, the way I understand it, take pressure of the registers significantly, but also takes quite some coding effort on my part. Does this sound like an idea worth trying?

$^1$ Popov et al.: Stackless KD-Tree Traversal for High Performance GPU Ray Tracing (2007).

Lowering register pressure doesn't necessarily give you any performance boost though. I recently went through this exercise myself on GCN architectures (for a simple ray tracer) and reduced register pressure so that it increased occupancy from 2 to 4, which had no impact on performance. It's generally a good idea to reduce the pressure if you need to hide memory latencies, but it really depends on what the real bottlenecks in your shaders are.

Since you are working on a GPU ray tracer, you might get better improvement in performance by paying attention to the divergence of threads and try to improve this instead. For example by trying to group rays based on location and direction to reduce divergence in KD-tree traversal per wave. I'm not familiar with the paper you are referring to, and you may already have considered this though.


Can't understand the Importance sampling GGX I've read many articles relating to importance sampling of GGX. However, I still feel confused.

The formula of GGX is :

And many article says we should use this formula to sample:

In the formula, theta is the angle between the normal and the halfvector. So what the formula shows is how to sample a angle between normal and halfvector? But, in the article https://schuttejoe.github.io/post/ggximportancesamplingpart1/
it says "To importance sample D(ω^m) we will use the inverse of the CDF of D(ω^m) to generate a microfacet normal ω^m. If you are familar with using GGX in game rendering this can be thought of as creating a half vector in tangent space." How is theta related to microfacet normal? How can I sample a Li based on the theta?

In microfacet BRDFs, the half-vector is the same as the microfacet normal. The half-vector is exactly the required normal for a microfacet to reflect light from the incident ray to the outgoing ray, by the law of reflection. So, for given incident and outgoing directions, only those microfacets whose normals are aligned with the half-vector are "active".

Conversely, when importance-sampling a microfacet distribution, we wish to select a random microfacet normal, then reflect the incident ray through this normal to produce the outgoing ray. The incident and outgoing rays will then have the chosen microfacet normal as their half-vector.

For an isotropic BRDF, you will sample theta from the inverse CDF, and you must also sample phi, the azimuth angle about the normal, uniformly between 0 and 2π; then convert those into a unit vector using the usual spherical coordinates (as mentioned in the linked blog post), combined with a tangent-to-world transformation. For an anisotropic BRDF, both theta and phi would be supplied by sampling an inverse CDF.


Subsurface Scattering Without Rays? Some games claim to use subsurface scattering but my current understanding of sss is only achieved with ray tracing. How would you sub surface scatter without ray tracing? Do some claim to use sss but really use some approximation trick that doesn't use rays. I can't even imagine doing the below real time.

There are lots of ways to fake SSS with greater or lesser fidelity. A few recent-ish methods:

Screen-space blurring of object lighting (described in detail here)

Using “interior” ambient-occlusion maps generated from a normal-flipped version of the geometry to approximate local thickness, providing an easy way to fake light transmission (more information here, with an addendum here)

There’s also the assortment of techniques in the GPU Gems article PaulHK linked to. One thing I’d add is about the light-falloff-texture approach mentioned at the beginning of it, which Valve used for a couple of interesting effects in Source games in addition to fake SSS—see their $lightwarptexture documentation for some information about that.


How to find vertices on borders of geometry I trying to highlight vertices which are on "borders", imagine half of sphere for example. Also for example I added image geomtry with triangles and highlighted necessary vertices. Any ideas? Thanks

I'm assuming you are rendering this object as a set of triangles. To do what you are trying to do, you will need to store all your triangles (and vertices) in some sort of buffer or list. Then, for each triangle, test if all three of its sides are matched with another triangle side. If so, it is enclosed and inside the object. Otherwise, it is outside the object. Some basic pseudo-code (I apologize for all the foreach loops :) )

foreach t in triangles:
    all_sides_touched = True
    foreach side_t in t:
        this_side_touched = False
        foreach u in triangles.exclude(t):    // run through all other triangles to test
            foreach side_u in u:
                if side_t.equals(side_u):  this_side_touched = True
        all_sides_touched = all_sides_touched and this_side_touched
    if not all_sides_touched:
        foreach side in t:
            foreach point in side:
                // do whatever you want to do with outside vertices


Rendering Equation for photons carrying flux I am trying to understand the mathematical/physical foundations of photon mapping better.

In the forward photon tracing step it is established, that the rays are carrying portions of total power (flux) rather than radiance as it is typically done in other types of ray-tracing algorithms.

However, in all articles and tutorials I could never find a precise explanation how such flux-carrying path interacts with materials? In other words - what is the rendering equation for interaction between flux-path and a surface?

What I typically found is an algorithm, based on Russian Roulette, for building a light path. But not a foundation equation that lead to that algorithm.

In particular, I am interested in:

Should the equation include the cosine of the incident angle or not? I suspect it shouldn't be there, as the power remains constant, but it is spread over a bigger area (thus reducing the radiance at each point, but not total flux).
Should the BRDF function be somehow (re)normalized? For example, for a perfect diffusive material, the BRDF is a constant not greater than 1/pi, so that it is energy conserving. But for a flux-based equation, assuming it is different, energy conservation may set a different maximum value.

Note, the very similar question:

Rendering equation for Photon Mapping

refers to a different step of the photon mapping algorithm. It asks for the rendering equation where the radiance is being estimated based on the nearest stored photons. Instead, I am asking for the flux-surface interaction in an earlier stage of the algorithm, when photons are being generated and propagated.

It'd be hell a lot easier if this were on graphicsexchange, since I can't use latex here but anyways.

In the first pass of Photon Mapping you don't need to use the Flux form of rendering equation. You just divide the original flux coming from the light source among the N photons, then for each photon you use Russian Roulette to determine whether it reflects, transmits or gets absorbed and store the incoming flux.

In the second pass you need to find radiance but you have information in Flux form. The rendering equation can be simply transformed.

So all you need to do is divide the flux stored for each photon by the area. Note that the Area is often denoted as A= pi r^2 Since we are concerned with a hemisphere and the underlying area is just a circle.

About your 2nd point, I don't think the BRDF needs to be re-normalized or whatsoever since the equation is essentially the same. Note that the second term in the last equation is still a radiance written in flux form. (BRDF has units of inverse solid angle thus making the whole term radiance).


Are w3.org docs correct? I'm referring to this section: https://www.w3.org/TR/compositing-1/#simplealphacompositing

Considering the formula co = Cs x αs + Cb x αb x (1 - αs) the docs state then, that co is the

  the premultiplied pixel value after compositing

I would argue that this is not the case, since in this article the resulting color is depicted with a capital C and in the case of premultiplied version, the lower case is used (c). I'm not really sure if this is the case, it's not too obvious from both docs and the article. So, is the result of  co = Cs x αs + Cb x αb x (1 - αs) really a premultiplied value?

Yes, the statement in the W3C doc is correct. You can tell because the background term includes premultiplication by the background alpha αb. If it were non-premultiplied, it wouldn't have an αb in it.


Quick 2-color quantization for small images So I have a small (8x8 pixel) image in 24-bit color. I want to be able to (quickly) generate a color-quantized version of the image that only has two colors.

One naive approach is to cluster a luma histogram of the image (1-dimensional clustering is easier than the 2D or 3D case). The chroma can be generated by averaging the pixels in each cluster. Unfortunately, this technique doesn't really work well if the the image has uniform luma.

Any ideas?

The approach I would use, and indeed have used, would be to

run principal component analysis on your image data to generate "the" main axis
Project your pixels onto that axis (i.e. compute the dot product of each colour value against the axis)
Using the dot prods, sort the the colour values from smallest to largest and
Find the "partitioning" point, i.e. that splits smaller values in one set, larger in the other, that minimises, say, the mean squared error. i.e. each partition is represented by the average of its members and you compute the total error.

This last step, i.e. finding the best partition, can be done in linear time by starting with the "least" item in partition "A", all the rest in partition "B", and progressively shifting the next smaller over from "B" to "A, one at a time. You just need to keep separate sums of the values and sums of squares of the values for each partition and then you can compute the total MSE relatively cheaply.

Note that the function should follow a quadratic curve so it (should) begin decreasing until it reaches the minimum. Once it starts increasing again you can stop the search.

A cheap way to find the principal axis is to first generate the 3x3 covariance matrix and then use the "repeated matrix vector multiplication" trick to find the principal eigenvector (i.e. the principal axis)


How to implement physically based unit in pbr renderer According to this note, Frostbite use photometry unit in their calculation. I am trying to mimic this feature, but I am not sure what transformation need to be done. I will try to explain what I have in mind.

First we do gamma correction(pow(2.2)) to the albedo texture. Then we apply the BRDF that takes luminance as input and will produce output luminance. After that we do tonemapping(say we use filmic tone mapping) and then inverse gamma correct pow(1/2.2) and store it to the framebuffer.

Is this process correct? Should I transform the luminance to radiance first before doing the tonemap or maybe I should convert it even before that when doing BRDF calculation? Do I need to convert the albedo with photometry curve after doing gamma correction?

First we do gamma correction

Make sure you actually need to do gamma correction on the albedo texture (or rather "uncorrection", i.e. making it linear). Not all textures are stored in sRGB color space. Also, you're doing it the wrong way around: Going from sRGB to linear color space you need to do the pow(1/2.2) part here and the pow(2.2) part in the end.

  Should I transform the luminance to radiance first before doing the tonemap or maybe I should convert it even before that when doing BRDF calculation?

Yes you should convert before doing the BRDF calculation. Tonemapping is not a feature that's part of the actual physical simulation of light, therefore it doesn't "need" any physical units. The BRDF however is, and here you should make sure that your units "add up". E.g. you wouldn't calculate the surface area of a rectangle by using meters as the unit for one side and inches as the unit for another side.

  Do I need to convert the albedo with photometry curve after doing gamma correction?

I'm not entirely sure here, but I think you don't need to do that. The albedo color basically tells you, how much of a certain light will be reflected. Since you change your light to photometric units (if you do it as suggested in the Frostbite notes), than I would say this already taken care of. You could also look at it like this: Since the color is a ratio and thus unit-less, there is no need to convert it from unit to another.


Volume of Intersection of Cube and Sphere Is there any way to compute the volume when a sphere intersects a AABB(cube), with different centers. Also i want to be able given a percentage of sphere's volume to  to place the sphere's center in a point in 3D space where it intersects the AABB and satisfies that intersected volume.

There are many ways, but the fastest I think is to create an octtree for your cube and choose a small subdivision level. Then do simple collision detection of the cube to the sphere. If the sphere collides with the cube, subdivide it into 8 smaller cubes. Test each of those cubes and repeat. When you are finished, you will know how many cubes have intersected the sphere, just multiply their dimensions by the number of intersections and you have a close approximation of total volume.

You may want to say that a cube collides with a sphere if its center lies within one radius of the spheres center.

I recently did something similar for a CNC simulator I wrote.

Good luck!

As far as the second part of your question, there would have to be constraints. If the cube is substantially smaller than the sphere, it may not be possible to satisfy your volume requirements. Given that constraints are in place, this method is fast enough that you could just drive your sphere into the cube until your required volume is met.


How to generate OSX Flurry screensaver I want to generate video similar to OSX Flurry screensaver (screenshot, video), but I have no idea how it is made. Does anyone here has an idea about how it's made, its algorithm and method?

Looking at the video you provided, I would tend to think it's something along those lines:

A particle emitter, to which all threads are attached, is moving around. I haven't really figured what its motion is dictated by.
Each thread is made of particles flowing away from the emitter. The thread has a direction, used for the initial speed of the particles. Once created,  they are independent from the emitter.
The particles use a quad or a rectangle textured with a fuzzy ellipse shape (you can see them individually when a thread suddenly changes direction) oriented in the direction of the motion. Their color is based on the duration since their creation, and they use additive alpha blending (the saturation to white is typical). After a certain duration, they fade out and die, only to be recycled into new particles.
I suspect the whole thing might be in a vector field, or something similar blowing wind onto the particles, which would explain why the threads sometimes suddenly change direction.
There might be some feedback post-processing effect to smooth everything out.


Is there a list of OpenGL extension support? I am looking for a list that states which vendor does support which OpenGL extension.

For example I wanted to use ARB_shading_language_include, which was accepted in 2013, but from various forums on the internet I assume it isn't implemented on AMD drivers. But I can't find any official document that states where this extension is supported.

Does such a list exist?

There is no official list, but a community driven one. It is largely up to date:
https://opengl.gpuinfo.org/listextensions.php

As can be seen here AMD doesn't support the ARB_shading_language_include extension at the time I wrote this answer.


Creating a vanishing point perspective shader I have done my due hour of research before asking this question, it's a simple question, but I have found no hint at an answer online.

I want the ability to extend an object infinitely in one direction, such as a tower that appears infinitely high, or a road infinitely long. I have tried my hardest to think up an approach to this that doesn't cheat, but nothing has come to mind.
An example, take a textured cube, and have it appear infinitely tiled in one direction, to its vanishing point:

Is there a way of doing this while retaining lighting and bumpmaps so it seamlessly blends with level geometry? Is there any way to do this efficiently?
Hopefully this isn't a stupid or misplaced question, thanks for your time.

It should be possible to do this properly by using a modified projection matrix for your entire scene that has its far plane at infinity, as detailed here. That should allow you to properly render points infinitely far away from the camera. The presentation says that such a point is represented by a direction (a vector with .w = 0), so it should be as simple as replacing the position of your vertex with the direction vector pointing towards the desired vanishing point.

One caveat is that any interpolation between the vanishing point and any other vertex would be incorrect, because the perspective correct hardware interpolator can't properly handle infinite values. This shouldn't affect lighting assuming normals are the same, but it will make texturing considerably harder. Probably the easiest way here would be to do an inverse projection of the point back to object space in the fragment shader and define the texture coordinates to be based on the object space position of the fragment.


Convert triangulated object to a polygonized object Is there an efficient algorithm to convert a 3D object described with a set of triangles to a 3D object described with a set of polygons. Polygons can have an arbitrary number of points, more is better. I'd like to minimize the number of polygons necessary to describe the 3D object but it does not have to be optimal.

The 3D object should stay the same.

Polygons can be non-planar.

I guess what you wanna achieve is mesh simplification. Then those materials can be useful for you.

http://graphics.stanford.edu/courses/cs468-10-fall/LectureSlides/08_Simplification.pdf

https://pages.mtu.edu/~shene/COURSES/cs3621/SLIDES/Simplification.pdf


Depth of Field in Path Tracing: What do I do with the secondary ray? How can I implement Depth of Field in my Path Tracer? I have read this article which explains the theory behind everything & I already have a good understanding of what this effect needs to achieve based on previous experience with a DSLR but I am a little confused on how to code everything although I understand a most of the calculations involved (after reading the article linked above).

I understand how to calculate the focal point, focal distance and the secondary ray. But I'm confused on how to put this all together to create the 'blur' that is needed. I haven't found much online so I was hoping that someone here could explain that with some code.

Could someone show me their implementation?

The missing step

If you already understand how to generate a secondary ray, then you have already grasped the difficult part. All you need to do now is find the colour that this secondary ray results in. This is exactly the same process as using the primary ray to find a colour, in basic ray tracing.

After repeating this for a large number of secondary rays, the average of all those colours is the colour you assign to the pixel in question (you don't use the primary ray - its only purpose is generating the secondary rays). This is the only difference from basic ray tracing.

Why it works

My personal difficulty when trying to understand this was that I was trying to understand how the light gets from the different points on the lens back to the eye/camera. The thing to understand is that this is only an approximation of a lens, and the rays don't need to consider the eye/camera at all.

The lens is being approximated by a disc.

All these different secondary rays show you what you would see from different points on that disc, if you looked directly towards a certain point (the focal point).

For an object at the focal point, all these different viewpoints will see the same point on the object, and the object will look sharply defined.
For an object behind the focal point, all these different viewpoints will see slightly different positions on the object, and some of them will miss the object and see what is to the side of it. The edges of the object will be blended with the colour of whatever is to the side of it.
For an object in front of the focal point, again the different viewpoints will see slightly different positions on the object, giving blurred edges.

If a primary ray hits an object at the focal distance, but there is a closer object which is not quite in the way, but almost, then some of the secondary rays will hit the closer object before they reach the focal point. This will give a blurred edge to the closer object, partially obscuring the object at the focal point, even though that object itself is in sharp focus.

All of this is as desired. To this level of explanation, this mimics the effect of a real lens, without having to consider refraction at all.


Compute shaders: one-time-only versus persistent buffers When using a Compute Shader, is it possible to keep a buffer in the GPU memory during the whole run-time duration of the application, so whenever the original Compute Shader or other Compute Shaders are executed later they can access data from such buffer and/or update them?

If so, what is the correct way of declaring/referencing buffers when one wants them to be permanent in GPU versus one-time-use only? If that helps, I am using CG in Unity, but an engine/language agnostic answer is fine.

Yes, it's totally possible, and in fact it's preferable to keep buffers in memory if you will be accessing them over the life of your program, rather than deallocating and reallocating them again.

I can't speak for Unity's way of allocating persistent buffers - I work with Unity and do a lot of shader writing in Cg, but most of our stuff is on mobile so compute shaders aren't an option. However, from a recent project I did using OpenGL, a buffer will generally hang around until you tell it not to, and even then, it is not in any way guaranteed to release the memory in a timely manner.

Allocating a buffer in modern OpenGL is done by calling glCreateBuffers and passing it a pointer to a GLuint array which will store your buffer 'names', then calling glBufferData or glBufferStorage to actually allocate and populate the VRAM for that buffer. Calling glBufferData a second time on the same buffer name will allocate new storage for that buffer, and the original storage will be 'orphaned'. It's kind of like reallocating an object in Java, the variable name will no longer point to that object, but other parts of the system may still have references to it (e.g. if it's still being used for a texture read somewhere), so it will not be eligible for cleanup until all references to it are gone.

Calling glDeleteBuffers with the buffer name has a similar effect but without allocating new storage. In both cases, the storage will not be deleted until absolutely all references to it within the system are gone. Even then, it is more like Java's garbage collection than C++'s delete, the system may well hold on to the memory for quite some time in order to reuse it for any future allocations you may make.

The upshot of all this is it's often better to allocate buffers once when you launch your app and cycle different data through them as needed. In OpenGL, a buffer will persist for the lifetime of the application unless you explicitly delete or reallocate it. Unity's rendering system has its own quirks as I'm sure you're aware, but in principle it should behave similarly.


Can raytracing be used to imitate the behavior of light to a higher degree? Raytracing "sees" the pixel on the other side of the screen;
but doesn't this mean the rays traced can be computationally reflected and refracted imitating the properties of actual light?

It'd be exciting if it were possible. Any visual scene possible outside computer graphics will be possible just using physics equations.

You are late by about 35 years. This was addressed in the historical paper "Whitted, T., An improved illumination model for shaded display. Communications of the ACM, Volume 23 Issue 6, June 1980, Pages 343-349", using  ray tracing.

Since then radiosity has been introduced to better deal with diffuse phenomena. Correct physical modeling of the image formation process was discussed six years later in the seminal "Kajiya, James T. (1986), "The rendering equation", Siggraph 1986: 143".


Dynamic Array in GLSL Is it possible to have a dynamic array in a GLSL shader? For instance, what if I have something like this in my GLSL Shader:

uniform int size;
uniform SceneLights lights[size];

void main()
{
    for(int i = 0; i < size; i++) {
        /* Do Some Calculation */
    }
}

And this would be my C++ file:

for (GLuint i = 0; i < pointLights.size(); i++) {
    glUniform3f(glGetUniformLocation(shader, ("pointLights[" + std::to_string(i) + "].position").c_str()), lights[i].someValue);
}

I would like to do this because I have a file with the positions of all the light sources in my scene (this may vary for each scene) so I would like to have a dynamic array which will allow me to send different numbers of lights to my shader.

I don't think uniform arrays can be dynamically sized. In your case you should define the array as the maximum number of lights you will process and then use a uniform to control the number of iterations you do on this array. On the CPU side you can set a subset of the lights[] array according to the 'size' variable.

e.g.

#define MAX_LIGHTS 128

uniform int size;
uniform SceneLights lights[MAX_LIGHTS];

void main()
{
    for(int i = 0; i < size; i++) {
        /* Do Some Calculation */
    }
}

I think (speculate) the likely reason for this is that it would be impossible to determine the location of a uniform if there are variable sized arrays in your uniform list which depend on the value of another uniform.. e.g.

uniform int arraySize;               // offset 0
uniform int myArray[arraySize];      // offset 4
uniform int anotherVar;              // offset ????

GLSL would not know were to place anotherVar, because it would need arraySize to be set with a value before it can compute the offset of 'anotherVar'. I suppose the compiler could be clever and re-arrange the order of uniforms to get around this, but this will fail again if you have 2 or more variable sized arrays..


There are many shading languages built on top of GLSL or HLSL - what problems do they typically solve, and what worthwhile advancements do they make? Whew, that was a long title.

Either way, I'm asking this question, since I like to think about various things, and it occurred to me that there isn't really any simple, open source layers on top of GLSL, even if only to add simple things such as includes, or commonly used functions.

As research of sorts, I'm asking this question, since my own awareness of such languages is minuscule to say the least - I know of bgfx's shading language, and Unity's ShaderLab, but I don't really know what they accomplish - or why - being a relative newbie to computer graphics.

Alternatively, what would your wishlist for a shading language like this be? Mine so far is includes, some compatibility between versions, optional "hidden" inputs that allow easily accessing textures at pixel offsets, or image sizes, etc. and probably passes - for, say, two-pass gaussian blur.

Thoughts?

I think it is fair to say that the reason there are so many niche variations of GLSL/HLSL/Cg/whatnot is because no programming language is nor will ever be a one size fits all tool. Different problems require different tools, so sometimes it is worth the effort of developing a custom built tool if it is going to pay off in the long run.

Stock GLSL is by itself pretty much unusable. It hasn't even acquired much stability across versions, so for a program that targets more than one OpenGL version, some sort of preprocessing is a must. HLSL is a bit more stable across versions, but again, if targeting more than one D3D version, some work will need to done to get good portability.

The kinds of things people usually do are pretty much what you said, like a adding support for basic programming features such as modules and uniform syntax across versions, or even portability across different APIs (GL/D3D) without having to rewrite the shader code. More sophisticated things include fully fledged material systems or things like generating shader programs on-the-fly.

Shading languages will probably get better and more generic in the future, incorporating things that today are commonly hand-rolled as core features. The new GCN architecture is a sign of that. So shading languages will be more usable out-of-the-box a while from now, but custom built solutions will never go away because there's only so much you can generalize.


What happens to float values written to UNSIGNED_BYTE texture? This is more a theoretical question. I'm wondering what happens when I write float values into an empty RGBA texture of type gl.UNSIGNED_BYTE in WebGL 1.0. Obviously, every channel then has 8 bits. But that wouldn't be enough to represent floating point values, would it? Nevertheless it somehow works, more or less. See part of a fragment shader code here:

precision highp float;

uniform vec2 u_Resolution;

void main(void){
   gl_FragColor = vec4(gl_FragCoord.xy / u_Resolution, 0.0, 1.0);
}

When I specify the texture to be of type gl.FLOAT (using the OES_texture_float extension), the results of course look better. But I want to know exactly why they do.

In addition: does the specified precision have an effect on the texture (to be written)?

Unsigned byte textures in a graphics API are usually "normalized"—the byte values are interpreted as ranging from 0.0 to 1.0, with a byte of 255 mapping to 1.0. So no, they aren't truly floating-point values, but the hardware will automatically convert them to/from float when you read/write the texture. If you try to write a value outside the [0, 1] range, it will be clamped.

As for the precision highp part, it doesn't directly affect the texture, but it can certainly affect any calculations of values you write to the texture. For example, to calculate an 8-bit texture you certainly want a minimum of 8 bits of mantissa precision (i.e. lowp)—and you probably want more than that, depending on what calculations you're doing. I'd advise you to check what the actual precision levels are on your device, since they vary widely depending on what GPU you have. See this question for more.


Vector graphics output from shader program I’m fairly new to shader programs in general but I have a question concerning vector graphics. I understand that a pixel shader can generate the color information to draw some sort of geometry on screen. But let’s say you had a shader that created a silhouette (outline) around some geometry. Is it possible for a shader program to generate vector graphics (ie. Curves) that follow that exact silhouette (instead of pixel data)?

Or in a different example, this image was created by using a pixel shader. Would it be possible for a shader to generate the lines on a teapot as actual curve geometry rather than just a bitmap that is mapped to the 3D geometry?

I’ve tried to do some preliminary internet searches but can’t find anything that would answer this type of question. Are shader programs fundamentally different and as such could not generate lines, curves, etc. as a form of output?

In OpenGL, there are 6 different shader types:

Vertex Shader
Tesselation Control Shader
Tesselation Evaluation Shader
Geometry Shader
Fragment Shader (Pixel)
Compute Shader

The first 5 are part of the render pipeline and fulfill a very specific purpose in that pipeline. They are executed in ascending order. The shader types 2.-4. are optional and can be skipped. Between those stages, some operations are performed, that are very specific to rendering. Have a look at this link for further information.

The problem here is: Since the rendering pipeline has a very specific purpose (generating an image of pixels), it is very hard to use it and its shaders for something else than rendering. Won't say it is impossible, but it gets certainly harder and harder the less your goal has something to do with pixel generation.

For general-purpose computations, you can use compute shaders. You can basically compute anything there, so this is probably the shader type you are looking for. However, the way you ask your questions gives me the feeling, that you have some misunderstandings on how rendering works in general and you should probably read some literature about how the render pipeline creates an image from 3d data.

  But let’s say you had a shader that created a silhouette (outline) around some geometry. Is it possible for a shader program to generate vector graphics (ie. Curves) that follow that exact silhouette (instead of pixel data)?

As written above, generally yes. It is possible to write a shader that does that. But probably not with the same technique as the outline is created in the "pixel case". It is certainly much more complicated.

  Or in a different example, this image was created by using a pixel shader. Would it be possible for a shader to generate the lines on a teapot as actual curve geometry rather than just a bitmap that is mapped to the 3D geometry?

Again, yes it is possible for a shader to fulfill this task, but you wouldn't use a pixel (fragment) shader for that. You would also need to approach the whole task very differently.

  Are shader programs fundamentally different and as such could not generate lines, curves, etc. as a form of output?

The shaders of the rendering pipeline are tied to a very specific task and have very specific boundaries on their inputs and outputs. But if you know what you are doing, you can probably hack around those limitations. However, I think it wouldn't make much sense to do so, except you want to prove that you can.
Instead, you can use a compute shader for any computation, that has nothing to do with rendering.

To sum it up

You can use compute shaders to get other output types than just pixels and images made of pixels. However, calculating basic 2d shapes/curves/vector graphics from 3d data will most likly be fundamentally different than generating a corresponding bunch of pixels from that same data. In conclusion, you can't just turn a switch (change 2-3 lines of code) to select which output you like to have.


Kaiser windowed sinc filter for mip mapping These older but informative article talks about how a Kaiser windowed sinc filter is a good way to generate mips.

The Inner Product - Mipmapping - Part 1 | Jonathan Blow

The Inner Product - Mipmapping - Part 2 | Jonathan Blow

Unfortunately, the source code it mentions isn't anywhere to be found.

Can anyone explain how a kaiser windowed sinc filter is calculated and/or give the kernel weights to use for convolution?

I was curious so I tried it. I used the Kaiser Window from Wikipedia: $\frac{I_0\left(\alpha\sqrt{1-x^2}\right)}{I_0\left(\alpha\right)}$

This page has a slightly different formulation using $\pi\alpha$ instead of $\alpha$. Given the comments in the page you linked about the Kaiser window being very similar to Lanczos and how they even overlap in the graph, I suspect they also used the former formula. You can see how close they are if you plot both on Wolfram.

This left me wondering if that's what Mitchell originally meant as it's not "noticably better". I could not see the difference at all with my test texture (it's there, just not noticeable). However, using the second formula, the difference in the plots is much more striking. And there is a noticeable difference in the resulting images. Whether it's better is a matter of interpretation and depends on the filter size.

With $x \in \left[-1,1\right]$ , I used this C++ windowing function:

#include <boost/math/special_functions/bessel.hpp>
float bessel_i0( float x )
{
    return boost::math::cyl_bessel_i( 0, x );
}

float window_kaiser( float x )
{
    const float alpha = 4.0f * M_PI;
    float k =
        bessel_i0( alpha * sqrtf( 1.0f - x * x ) ) /
        bessel_i0( alpha );
    return k;
}

This is the second version. Remove the M_PI to get the first version which is more like Lanczos.

The modified Bessel function of the first kind seems to be a C++17 / boost thing. If you have neither, this quick hack will do:

float bessel_i0( float x )
{
    float r = 1.0f;
    float term = 1.0f;
    for( int k = 1; true; ++k )
    {
        float f = x / float(k);
        term *= 0.25f * f * f;
        float new_r = r + term;
        if( new_r == r )
            break;
        r = new_r;
    }
    return r;
}

Just don't consider that a general replacement for a proper bessel function. However, it's fine for computing the window as far as I could tell.

As the second wikipedia page I linked shows, that window_kaiser can be used on a symmetric filter with $N$ weights by using $x = \frac{2n}{N-1}-1$, where $n$ is $0, 1, 2 \dots N-1$.

Computing the weights of the sinc filter itself is another thing entirely as it depends on filter size, how much you're resizing, etc. I suggest posting a separate question if you don't know how do to that.


Fresnel and specular colour I'm trying to implement Cook-Torrance BRDF and had previously bookmarked this question as it looked very well thought out. Going through it, I'm confused about the Fresnel equation. The author has the equation:

$$
F = c_{spec} + (1 - c_{spec}) (1 - \mathbf{w_i} \cdot \mathbf{h})^5 \tag{1}
$$

and says "$c_{spec}$ is the specular color"

Looking elsewhere, I can see this equation is Schlick's approximation, but in other references, $c_{spec}$ is called $R_0$ and is $\left(\frac{n_1-n_2}{n_1+n_2}\right)^2$ which is clearly a number based upon refractive indices and not a colour.

Does equation $(1)$ make sense with colour?

Yes, because refractive index can vary with wavelength. This is the origin of colored specular reflection in metals such as gold and copper; most other materials have essentially uncolored specular. At each wavelength, the specular reflectance at normal incidence is related to the refractive index at that wavelength according to the formula you mentioned.

In case you're interested, the site refractiveindex.info has data tables and graphs of refractive index versus wavelength for various materials. For example, here's gold:


Why does the integral of NDF over a solid angle equals the area where micronormals belong to that angle? In Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs, Section 2.2, Mr Eric Heitz defines the distribution of normals as:

And then, he goes on with the following assertion:

I fail to understand how can D have such a property. Is it an assumption based on common sense or on geometrical considerations? Or can this equality be proven by manipulating the formulation of the integral?

$D(\omega)$ is defined as the area ($m^2$ unit in the numerator) of the microsurface with normals pointing in the direction $\omega$. $\mathcal{M}'$ is defined as the portion of the microsurface with normals point in the direction $\omega \in \Omega'$. So it's natural that the integral of $D(\omega)$ over $\Omega'$ gives the area of $\mathcal{M}'$.

Substituting the definition of $D$ into (4) gives:

$\int_{\Omega'} D(\omega_m) d\omega_m = \int_{\Omega'} \int_{\mathcal{M}} \delta_{\omega}(\omega_m(p_m)) d p_m d\omega_m$

A fundamental property of the Dirac delta function is that it integrates to 1 over its domain. Unfortunately, we are integrating over $\Omega'$, not $\Omega$. Here I will stop pretending that I am actually able to use formal mathematical terms, and I will just note that the integrand has non-zero values only where both $d p_m \in \mathcal{M'}$ and $d\omega_m \in \Omega'$, which is what (3) says. As long as at least one of our integration domains is restricted to the set we care about, we will get the same result. So I'll just move the prime around a tad:

$\int_{\Omega'} D(\omega_m) d\omega_m = \int_{\Omega} \int_{\mathcal{M}'} \delta_{\omega}(\omega_m(p_m)) d p_m d\omega_m$

Now we can apply the Dirac delta identity, since we're integrating over the whole domain:

$\int_{\Omega'} D(\omega_m) d\omega_m = \int_{\mathcal{M}'} d p_m$

Ta-da!


Synchronizing successive OpenGL Compute Shader invocations I have a couple of compute shaders that need to be executed in a certain order and whose outputs depend on previous inputs. Ideally, I'll never need to copy a buffer client-side and do all of my work on the GPU.

Consider I have two compute shaders compiled and linked as program_one and program_two. Suppose I also have a GL_SHADER_STORAGE_BUFFER that contains the data that is written to by program_one and read by program_two. Can I simply do the following:

glUseProgram(program_one);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, buffer);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, index, buffer);
glDispatchCompute(16, 16, 1);

glUseProgram(program_two);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, buffer);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, index, buffer);
glDispatchCompute(16, 16, 1);

Is it guaranteed that all invocations of the first compute shader will finish before any invocations of the second (to avoid data races between reading and writing buffer)? If not, how do I synchronize them?

No it is not guaranteed, since the OpenGL specification allows that two Compute Shader run concurrently or even in different order.

You need to call glMemoryBarrier(GL_SHADER_STORAGE_BARRIER_BIT) before the second glDispatchCompute to ensure visibility of the writes from program_one.

From the OpenGL.org wiki article on the memory model:

  [...] invocations between stages may be executed in any order. This includes invocations launched by different rendering commands. While it is perhaps unlikely that two vertex shaders from different rendering operations could be running at the same time, it is also possible, so OpenGL provides no guarantees.

From the Opengl.org wiki article on Shader Storage Buffer:

  SSBOs reads and writes use incoherent memory accesses, so they need the appropriate barriers, just as Image Load Store operations.


How can i wrap the earth image around a 3D Sphere using OpenGL, GLFW, GLAD, GLM? Please help me out with this thing and currently i am a little more than a beginner so please tell me in that context. i don't need any obsolete methods for creating this like using GLUT or GLEW.can you just tell me its algorithm to create the outline of it(like using icosphere or triangles) and how and where in the whole program do i implement this algorithm. currently i am referring the "learnopengl.com" tutorials.i know how to create triangles and 3d rotational cube.

I'm not going to post proper working code here just giving you a guideline. There might be an easier way but what I did back in my days was to create a sphere using Triangle Strips. There is an icosahedron approach as well but I haven't looked into that.

So first of all look up on GL_TRIANGLE_STRIPS and how they work. Secondly you need to look up on Spherical coordinates and how to convert them to Cartesian Coordinates.

A brief overview is that Spherical coordinates have parameters (radius, zenith, azimuth). Radius is pretty much self explanatory. Zenith angle is the inclination angle from the pole of the sphere (lines of longitude) While the Azimuth angle runs along the latitude. Zenith runs from 0-PI (assuming 0 starts from 1 pole and PI is the second pole) or -PI/2 - PI/2 (assuming the extreme at both poles and 0 is the center) and Azimuth from 0-2PI (all the way around the sphere).

Now after that you can think of the sphere forming up from concentric Rings. At each fraction of the Zenith angle there is ring that runs around the sphere (the latitude lines). Then you create triangle strips between those rings to fill the sphere up.

So basically you want something like,

zenith = azimuth = 0
ring_increment = PI/num_rings;
sector_inrement = 2PI/num_sectors;

for(i=0; i < PI; i+= ring_increment)
{
    for(j=0; j < 2PI; j+= sector_increment)
    {
        vertex_list.add(convertToCartesianCoord(radius, zenith, azimuth))
        zenith += ring_increment
        vertex_list.add(convertToCartesianCoord(radius, zenith, azimuth))
        zenith -= ring_increment
    }
}

The more the number of rings and the number of sectors the smoother the sphere will be.


First of all - a number must not occur twice, that is implied since we're talking about permutations. So filling the table with a simple random(255) function won't work.

Secondly, you need to ensure that there are no premature recurrence patterns:

Consider the values 1,2,3,4 - the permutation table 4,3,2,1 is not a very good one because of its short cyclic properties, i.e. 1 -> 4, 4 -> 1. Likewise with 4,2,3,1 or 1,2,3,4. The optimal tables take you all the way through all positions: 3,1,4,2 or 2,4,1,3.

This property becomes increasingly  important as you increase the number of dimensions and perform recursive lookups.

However this approach alone may create clusters of too similar values, which may or may not be wanted, which leads me to next point.

Thirdly, When you generate a table with the non cyclic properties, you need to step through remaining unassigned indices in a random manner. When possible constrain the random step distance here to a certain min and max range, e.g. 5..120 to avoid clustered groups of similar values. These numbers are worth experimenting with.


Existing method to automatically fill in this sort of concavity in meshes? I am working with a PWP3D-like algorithm that renders silhouettes of a mesh in order to segment a frame of video and update an object's pose. Because it is only the silhouettes of the mesh that matter, it would be nice to automatically generate simpler meshes where the indents/concavities that do not affect the silhouette are removed. That is, I want to generate a simpler mesh that has the exact same silhouette as the original when viewed at any angle from any position outside of the mesh's convex hull.
Is there an existing technique for doing this? Or even just some definitions other than "indent" that better describe this problem that might aid in a web search for this? I've currently found mesh simplification/decimation techniques that aim to preserve silhouettes, but these are not exactly what I'm looking for right now, and they generally seem to just simplify indents rather than outright removing/filling them. I have an idea for an algorithm that I think would work, but I want to check if anything else already exists.
As a visual example of what I mean, below is a mesh that, if such a technique were applied, would have the orange-highlighted faces removed and be turned into an ordinary cube.

At the same time, though, the problem is not as simple as just taking the convex hull, because this mesh, for example, is fine as it is.

After a fair bit of reading and skimming through papers, I have yet to find a good definition other than "indent" for what I want to remove, but I have found answers to pretty much everything else.
The concept of a "Visual Hull" is what I was looking for regarding a mesh that doesn't have these indents. While the Wikipedia page, and a bunch of other resources, usually just refer to visual hulls created from a finite set of viewpoints, e.g. a set of real-life cameras, Laurentini, who introduced the term "Visual Hull", also introduces a term "External Visual Hull" to refer to one created using every possible viewpoint outside of the convex hull [1]. This will have the exact same silhouette as the original mesh when viewed from any position outside the convex hull at any angle, so it was exactly what I was looking for.
[1] also contains an algorithm for computing the external visual hull for polyhedra, but it is sorta brute-force and is $O(n^{12})$. In [2], Petitjean introduces a much more efficient algorithm for polyhedra, and Bottino and Laurentini also describe a more efficient algorithm in [3]. In addition to these algorithms for polyhedra, other works by Laurentini also discuss algorithms for doing the same for curved objects, surfaces of revolution, etc. I have yet to come across anyone actually implementing these algorithms, though.
I'd like to describe all of these in more detail, but I still need to finish reading through them, and based on some deadlines coming up, I do not think I will have the time to do so for a little while. I hope to eventually come back to this post and edit it to actually describe at least one of the algorithms. However, if someone reading this wants to look these up for themselves, then in addition to the below citations, I would like to note that even if you don't normally have access to papers, e.g. through a post-secondary institution, all of these seem to be publicly available. You can download [1] from ResearchGate here, download [2] from ResearchGate here (though I have no clue why different authors are listed for it than Petitjean), and [3] can be found in Google Books.
Citations:
[1] Laurentini, Aldo. (1994). The Visual Hull Concept for Silhouette-Based Image Understanding. Pattern Analysis and Machine Intelligence, IEEE Transactions on. 16. 150-162. 10.1109/34.273735.
[2] Petitjean, S. (1998). A computational geometric approach to visual hulls. International Journal of Computational Geometry & Applications, 8(04), 407-436.
[3] Bottino, Andrea & Laurentini, Aldo. (2006). Retrieval of Shape from Silhouette. 10.1016/S1076-5670(05)39001-X.


Is it possible to bind dynamic texture samplers to a shader in DX12? So usually texture samplers are set directly in the D3D12_ROOT_SIGNATURE_DESC structure in DX12, at least that's how I've been setting them. I was wondering if it's possible to put them in an upload_heap and map a CPU pointer to it or something similar to make it possible to modify the samplers dynamicaly?

Yes it is possible by putting Samplers in descriptor tables to dynamically assign and index into them. Source: https://youtu.be/Wbnw87tYqVg?t=754

Here's an example that uses the HLSL root signature syntax to do this here:
https://youtu.be/Wbnw87tYqVg?t=5277


How to calculate matching roundness of two offset rectangles? I have two rectangles — one with a fill (blue) and one with a stroke (red). The red rectangle is being offset (depending on the stroke width) so that it appears snug and outside the edge of the blue rectangle.

This set up works great except when I need to round the corners.

As you can see I'm getting gaps between the rectangles and I'm stuck trying to figure out the math to calculate the right roundness amount for the outside rectangle (red).

Any ideas?

What you (probably) want to achieve is something like this:

When having a closer look at one of the corners and add a few lines, we see this:

The black lines indicate that the center points of the circles along the borders of the red and blue boxes is the same. If the outer radius of the red box, for example, is $50px$, and the distance between the outer borders of the red and blue boxes is $25px$, this leads to a border radius of $50px - 25px = 25px$ for the blue box. The distance, obviously, is dependent on the stroke width of your red stroke.

Note: When creating the above images, I simply drew two filled boxes with rounded corners. So first a filled red box and then a smaller filled blue box above it. This also avoids some pixels being left white due to rounding errors (no pun intended).


Process of compute shader in OpenGL I'am curious about compute shader in OpenGL.

Let's assume the number of points (vec4) is 900

and  the work group size(= the number of work items) is 256

Then, We would have four work groups because 900/256 = 3.xx, so need to plus 1.

The code is as follows:

glDispatchCompute(4, 1, 1);

and shader is as follows:

layout(local_size_x = 256, local_size_y = 1, local_size_z = 1) in;
layout(std430, binding = 0) buffer Particles {
    vec4 partices[];
};

In this case, I wonder how to work fourth work group.
In my thought, the fourth work group have overflow problem because the total number of points is 900.

Do we need to deal exceptional case(overflow)?

For example, if idex > total num, then 'pass'

Or

The Opengl deals this problem automatically?

(ex, There is no problem situation.)

I tried to check by myself. Even though I make overflow situation, there was no warning.

The Opengl deals this problem automatically?

How could it? At no point do you inform OpenGL that you only want 900 invocations. You have exactly 2 mechanisms to control the number of invocations: the work group size in the shader, and the work group count in the dispatch call. That determines how many invocations there are.

If you want a variable number of invocations with a fixed work group size, then your shader has to be given the actual maximum number of operations, and you have to explicitly not do anything for invocations beyond that size.


Instanced Stereo Rendering vs. Multiple Command Buffers Source
In this webpage from Nvidia, the author(s) seems to imply that you could create a command buffer for each eye on separate threads. However, I don't see the benefit to this over instanced stereo rendering since you must also handle synchronization with multiple command buffers.
Possible Synchronization Issues
For synchronization (feel free to correct me if I'm wrong), I was thinking about when using a framebuffer for each eye, kind of the "naive" way of rendering for VR. I suppose though you could just render to one framebuffer like with stereo instanced rendering. Regardless, the issue with synchronization that I'm concerned about would be if one eye finishes considerably faster than the other eye. It seems that with stereo instanced rendering, this would be less likely to happen since the workload is more divisible. The other issue I see where synchronization could cause a performance hit is that you must wait for both threads to complete sending their respective command buffer before sending it again. Arguably, this might not be a huge concern, but it seems unnecessary nonetheless.
Spatial Locality Issues
It also seems that since there isn't a way to guarantee when an object is going through the pipeline that you may get additional cache misses on the GPU compared to instancing here.
Question
Is there any advantage to using multiple command buffers for stereo rendering compared to instanced stereo rendering?

Yes, there are advantages. You can render different command buffers to different queues.

Modern NVIDIA hardware, in particular, offers a full sixteen separate queues that are capable of rendering independent graphics operations. By putting each eye in a separate queue, you may be able to use the available hardware resources more efficiently.

Then again, it may not. It all really depends on how things work out down in the lowest levels of hardware.

  Regardless, the issue with synchronization that I'm concerned about would be if one eye finishes considerably faster than the other eye. It seems that with stereo instanced rendering, this would be less likely to happen since the workload is more divisible.

It would basically never happen with instanced rendering. But you say that like it's a good thing.

If one eye happens to have more scene complexity than the other eye, then that means instanced stereo rendering will be wasting a lot of vertex processing on vertices for the "weak" eye that will never be seen. Whereas if you built the command buffers separately, you would only submit the stuff that matters for both scenes.

If you submit both CBs to the same queue, the total time spent processing those CBs can be less than the time spent processing the larger CB, since it isn't computing a bunch of triangles that won't be culled. If you submit the two CBs to different queues, how that balances out depends entirely on how well the hardware can adapt to the workload it is given. If it can adapt well, then you'll be fine.

So for this particular case, the multiple CB setup may represent a more efficient use of hardware resources. Granted, this particular case is rather rare.

Synchronization for this case is really not a problem. If one eye is so complicated relative to the other that you drop a frame, then it doesn't matter which form you're using to render. You're going to drop a frame.

With Vulkan, rendering entirely separate stuff to multiple queues isn't difficult. Indeed, the only real synchronization you might need would be to synchronize the presentation of the swapchain images, which is easily done with semaphores.

  The other issue I see where synchronization could cause a performance hit is that you must wait for both threads to complete sending their respective command buffer before sending it again.

No, you don't. If you're reusing CBs, you can always use the VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT, which allows you to submit a CB that's already in use. And if you're not reusing CBs (that is, your CBs aren't static), you don't care; just double-buffer your command pools so that you aren't trying to reset a CB that's being consumed.

The latter is the general standard method for rendering dynamic scenes with Vulkan.


Half-Edge Data Structure with holes I am trying to implement next/prev of edges in half-edge data structure. But I ran into this question. If there are two holes with a common vertex, will the closed loop of the edges run as in the first picture? Or will it be two separate cycles, as in the second picture?

In 3ds Max, when the border tool is used, it selects the edge loop as in the first image. Please help me to understand this.

The cube-like mesh shown in the images is a non-manifold mesh. Officially half edge data structures do not support such a topology. However, this case can be supported by continuing the boundary half edges through the non-manifold node. This is apparently the way it is done in 3DS Max, and also Blender. You can look at this article for several extension to the data structure to support non-manifold meshes.


Topology: Polygonal Meshes, Euler's Formula Firstly, I would like to tell this is my first question on stackexchange and I am a senior school student. Please bear with me, if I am not able to express myself clearly.

I am trying to solve a question based on euler's formula: V - E + F = 2 (1 - G)

I have been given a closed mesh of a sphere. Number of Edges and Faces and number of Vertices with valence 4 are given.

Now, I have to compute V, x_1, x_2 and x_3. The vertices for valence 4 are some how not printed in my book and I assume them to be 137 (274/2) relating to table on the right. I know F = 2 V, but 275 = 2V gives a decimal point number which is not possible !

Pardon for my brevity and correct me if I am not clear :)

Let g be the number of vertices of valence 4. A closed sphere has G=0 and we have the equation V-E+F=2. Now V=x1+g+x2 and F=275.

As a first step let us count the edges. If we multiply each face by its valence and sum those up, we get twice the number of edges (as each edge is shared by two faces), so:

(1)   2*E = (274*4 + x3*1)

Immediately we can see that 2 divides x3.

Similarly if we multiply each vertex with its valence we get again twice the number of edges, as each edge is shared by two vertices, so:

(2)   2*E = 3*x1 + 4*g + 5*x2

And again we see that x1 and x2 must have the same parity.

Plugging the first of this equation (with the very first ones) into V-E+F=2 we get

(3)   2 = V-E+F = x1 + g + x2 - (1096+x3)/2 + 275 = x1 + x2 - x3/2 + g + 548

Equating the two equations we derived above we get

(4)   1096 + x3 = 3*x1 + 4*g + 5*x2

Now the system of equations (3)&(4) has all the information encoded (excluding the parity information), so it appears you don't already have everything you need to know to determine these four unknowns.

I made a small Haskell program that does some bruteforcing with these conditions: Try it online!

The first solution it does get is (0,732,2564,0) = (x1,x2,x3,g)


Deriving radiance from irradiance and intensity I'm reading PBRT and am stuck in chapter 5.4 about radiometry. In particular:
We define the irradiance as the average density flux arriving at a surface with units $\frac{W}{m^2}$. So for a point light source, we have: $E = \frac{\Phi}{4 \pi r^2}$ since the area of a sphere is $4 \pi r^2$. Where $\Phi$ is the flux or power.
A (to me) similar concept is intensity which is the amount of power per angle. Again, for a sphere with a point light at the center, this is $I = \frac{\Phi}{4 \pi r^2}$ with the unit $[\frac{W}{sr}]$ (watt over steradian)
Now, the book defines radiance for a point $p$ as $L = \frac{d\Phi}{d\omega dA^\perp}$ in units $[\frac{W}{sr\cdot m^2}]$. Here, $\omega$ is the direction where the light comes from, $A^\perp$ is the projected are of $A$ as seen here:

This means that practically, when I implement a point light source with a given power that shines at a point $p$, I need to do the following to arrive at radiance:

Divide by $4 \pi r^2$ to convert power into $[\frac{W}{sr}]$, or in other words, intensity.
Given intensity, I need to divide it by $4 \pi r^2$ and multiply by $\cos \theta$ to arrive at $[\frac{W}{sr\cdot m^2}]$, the final radiance. The multiplication by $\cos \theta$ is to project $A$ to $A^\perp$ and is the dot product of the surface normal $n$ with the direction $w$ (as both are normalized).

For both calculations, $r$ is the distance between the light source and my point $p$.
However, when I look at the source, this is not what happens. The point light returns intensity divided by $r^2$ as seen here:
return I / DistanceSquared(pLight, ref.p);

and the integrator then multiplies it with the dot product (and the brdf) in the whitted integrator
L += f * Li * AbsDot(wi, n) / pdf;

So what is wrong in my derivation? Why do we "only" divide once by $4\pi r^2$ (to get the Intensity I) and not twice? Aren't we missing either the power per area or the power per steradian?
sources:
http://www.pbr-book.org/3ed-2018/Color_and_Radiometry/Radiometry.html

Your definition for radiant intensity is wrong: it should be just $\Phi / 4\pi$. There are only $4\pi$ steradians in a sphere no matter how big it is, so $r$ doesn't come into it.
Also note that you can't calculate radiance for a point source—it would be infinite, due to the fact the point source emits a finite amount of flux compressed into zero size. It subtends zero solid angle, from the receiver's point of view, so the $d\omega$ factor in the denominator of radiance would be zero. The usable quantities are radiant intensity in a certain direction from the point source (which could vary with direction, for a non-omnidirectional light), and irradiance in a certain direction and distance. As you've seen, irradiance is obtained from radiant intensity by dividing by $r^2$—you could think of this as "area per steradian", as it's the conversion factor from $4\pi$ steradians to $4\pi r^2$ area of a sphere; then the units work out.
Where radiance would actually show up is when dealing with an area light rather than a point light. Then you would have the flux being distributed over a finite solid angle from the receiver's point of view, and you'd get the incident irradiance by integrating the light's radiance over that solid angle. The emitted radiance would be $\Phi/(2\pi A_\text{light})$, assuming it's emitted uniformly over the surface of the light and into all directions from each point. This is only $2\pi$ since it's only emitting into the outward-facing hemisphere. Also note that you don't do any division by $r^2$ for radiance—with area lights, the distance attenuation comes naturally as a result of the light subtending less solid angle from the receiver's point of view, when the receiver is farther away.


Why replicating the higher bits of RGB565 when converting to RGBA8888? I have seen in some computer graphics software's code bases that sometimes the higher bits of RGB565-format image data are replicated into the lower bits when converting to higher-bit-depth format RGBA8888.

I have found for example the comment by user "eq" in this gamedev.net thread:

  I prefer to replicate the higher bits into the undefined lower bits:
  R8 = (R5 << 3) | (R5 >> 2);

However I do not understand the reason behind.

What use the purpose of replicating those bits into the converted data?

There's actually a reasonably good mathematical reason for doing bit replication:

First note that the n-bit string, $N$, actually represents the value $\frac{N}{2^n-1}$ and we want to produce the m-bit string, $M$, where $n<m$ and
$$\frac{N}{2^n-1}\approx\frac{M}{2^m-1}$$

We first scale numerator and denominator with
$$\frac{N.(2^n+1)}{(2^n-1)(2^n+1)}\approx \frac{M}{2^m-1}$$
and this simplifies to
$$\frac{N.(2^n+1)}{2^{2n}-1}\approx \frac{M}{2^m-1}$$

In your case, $n\in \{5,6\}$ and $m=8$ and we can "stop" here, but but the process can be repeated, (ad nauseum), if m >> n.

We next make the approximation...
$$\frac{N.(2^n+1)}{2^{2n}}\approx \frac{M}{2^m}$$ which simplifies to
$$\frac{N.(2^n+1)}{2^{2n-m}}\approx M $$

Note that $N.(2^n+1)$ is equivalent to repeating the n-bit string, to create a 2n-bit string, and the division shifts off the $2n-m$ LSBs to leave an M bit result.

QED

Of course, the  'correct' calculation is $M=\lfloor(\frac{(2^m-1) N}{2^n-1}+\frac{1}{2}\rfloor$ but this approximation, in general, works most of the time. Of course there are times when it's inaccurate, but IIRC only by one bit and relatively infrequently.


OpenGL animation - turn into mp4 movie Does anyone know how to do run an OpenGL animation and then convert it into a mp4 movie?  I've been trying with apitrace and ffmpeg but I get horrible quality output...

It's not too difficult to set up your app to stream frames directly to ffmpeg. This blog post by Miles Macklin shows how, and I've used this method in my own apps.

You spawn ffmpeg as a child process from your app, and at the end of each frame you just read back the GL framebuffer and write its contents to ffmpeg's stdin. The relevant code bits are excerpted here.

Initializing ffmpeg:

// start ffmpeg telling it to expect raw rgba 720p-60hz frames
// -i - tells it to read frames from stdin
const char* cmd = "ffmpeg -r 60 -f rawvideo -pix_fmt rgba -s 1280x720 -i - "
                  "-threads 0 -preset fast -y -pix_fmt yuv420p -crf 21 -vf vflip output.mp4";

// open pipe to ffmpeg's stdin in binary write mode
FILE* ffmpeg = _popen(cmd, "wb");

Writing a frame:

glReadPixels(0, 0, width, height, GL_RGBA, GL_UNSIGNED_BYTE, buffer);
fwrite(buffer, sizeof(int)*width*height, 1, ffmpeg);

You can adjust the quality by altering the -crf option in the command line (lower numbers are better quality). Don't forget to change the resolution (-s) and framerate (-r) to match what you want, as well.

Depending on the quality, how fast your PC is, etc, it might not quite work in real-time. It's a good idea to set up your app to use a fixed per-frame timestep when you're in video capture mode, so the resulting animation doesn't depend on how much slowdown there was in the capture process.


Raytracing: why are the spheres in the image below appear stretched? Some context. Above are the code and the resulting image for it in Peter Shirley's Raytracing in one weekend's book. As you can see from the code, he adds in some spheres. And yet in the final image there are 2 ellipsoids side by side. I just can't fathom why it is so, no matter how many time I looked at the code. Can anyone explain to me? Here's the link to the source code.
https://github.com/petershirley/raytracinginoneweekend

Shapes appearing stretched in the periphery is a consequence of perspective projection. The wider the field of view (FOV) is, the stronger the stretching effect gets.

To demonstrate the effect I wrote a quick example on ShaderToy: https://www.shadertoy.com/view/MltBW2

As you can see on the images below (corresponding to FOV of 40, 80 and 120; if I didn't make any mistake), this becomes especially noticeable with wide FOV, like 90° and above.


Given a background image B and a composie image of A OVER B, can we somehow recover image A? I often find myself in the situation where I have some old image where I composed a translucent overlay on top of a background, but then lost (or more likely discarded) the overlay. I still have pre-operation copies of the background, but have not managed to figure out how to perform an inverse operation to calculate the original overlay. Best results would be an operation I could perform in Gimp or Image Magick, but I could write an uncomposer if I had to, so long as I knew what to calculate.

Just to confirm, assuming non-premultiplied alpha, at one point you performed the following, per pixel operation:
$$ Result_{RGB} = Src_{A} * Src_{RGB} + (1.0 - Src_{A})* Background_{RGB} $$

You say you've lost the $Src_{ARBG}$ data,  but have retained the $Background_{RGB}$ and the $Result_{RGB}$, but want to recover the $Src$ data?

Unless you can make some other assumptions, you might be out of luck. It's a rather underconstrained problem.
Jim Blinn notes in "Fun with Premultiplied Alpha" that there are some commercial systems that are used to 'de-bluescreen' but they may require the operator to identify, say, what is 'Source' and 'Background'.

As an extreme example, you may have a pixel in the result which exactly matches the background but the source pixel may also have been the same colour.

If, on the other hand, $Result$ and $Background$ don't match, then you know that the source is involved but if the $Src$, $Result$, $Background$ RGB values are all collinear, then there could be multiple possible solutions.

Having said this, there may be some heuristics that perhaps could be used, but it'd require some thinking.
Perhaps, pixels which differ and are not collinear could be used to estimate $Src_A$, with some level of 'certainty', and this could be 'filtered' to neighbouring 'uncertain' pixels and the process repeated.
Non-constant pixels in the destination with a local neighbourhood which all match the background might be assumed to have an alpha of 0.0.


How can I make my ice cubes look real? I can model ice cubes as slightly misshapen transparent cubes with the refractive index of water, but they don't look convincing. They look like lumps of glass rather than ice.

Looking at real ice cubes I can intuitively describe some differences but I don't know what physical properties to change to match them:

Ice cubes are wet. Mine look like dry glass.
Ice cubes are transparent in places and not in others.
Ice cubes often have cracks that are visible despite not separating.

In this instance I am trying to model ice cubes on a surface (in air, not floating in water).

What techniques do I need to include in order to increase the realism?

I am not looking for real time techniques, just to produce still images. I would like the ice to be photorealistic even close up, and to cast realistic caustics and shadows.

I've tried using curved edges and coating my ice cubes with a thin layer of transparent material to simulate a melted layer of water, but it doesn't seem to give the impression of being wet. I've also tried embedding a transparent sphere half the size of the cube at its centre, with a fog effect, but it doesn't blend into the cube naturally - it just looks embedded. Even a series of nested spheres with gradually increasing fog still doesn't look right.

Two big ones you're missing:

Angle-dependent reflection.  This is one possible cause of your "transparent in places and not in others" effect, and the most likely cause of the missing wetness.
Ice cubes usually have air bubbles trapped inside.  This shows up as a white volumetric haze denser in the center of the cube (for small bubbles) or distinct bubbles (for large ones).  This is the other likely cause of your "transparent in places and not in others".

The techniques for modeling these depend on what rendering method you're using.


Alpha component in color buffer affects RGB components? I'm looking for a way to make my surface semitransparent, and for now i ended up using this configuration:

eglChooseConfig(eglDisplay, new int[] {
            EGL10.EGL_ALPHA_SIZE,   8,
            EGL10.EGL_RED_SIZE,     8,
            EGL10.EGL_GREEN_SIZE,   8,
            EGL10.EGL_BLUE_SIZE,    8,
            EGL10.EGL_NONE
}

My default configuration was:

eglChooseConfig(eglDisplay, new int[] {
            EGL10.EGL_RED_SIZE,     8,
            EGL10.EGL_GREEN_SIZE,   8,
            EGL10.EGL_BLUE_SIZE,    8,
            EGL10.EGL_DEPTH_SIZE    16,
            EGL10.EGL_NONE
}

As you can see, i added EGL_ALPHA_SIZE attribute and removed EGL_DEPTH_SIZE, since my application is a simple 2D drawing on a surface and it doesn't need a depth buffer. It works, but for some reason results in somewhat unexpected results for this simple fragment shader:

precision mediump float;
uniform sampler2D uSampler;
varying vec2 vTextureCoord;
void main() {
    vec4 originalColor = texture2D(uSampler, vTextureCoord);
    gl_FragColor = originalColor + vec4(0, 0, 0, -0.5);
}

The alpha component seems to be affecting rest of the parts of the color and changes it at some extent. The first image below is the original one, second is what I have, and the last one is what I want to achieve.

Any advice is appreciated, thanks!

For the sake of clarity

OpenGL ES Version: 2.0

It looks like you're drawing with pre-multiplied alpha. This means that the red, green, and blue channels have been multiplied by the alpha channel. So your shader should look more like this:

precision mediump float;
uniform sampler2D uSampler;
varying vec2 vTextureCoord;
void main() {
    vec4 originalColor = texture2D(uSampler, vTextureCoord);
    vec4 tempColor = originalColor + vec4(0, 0, 0, -0.5);
    tempColor.rgb *=  tempColor.a;
    gl_FragColor = tempColor;
}

I believe you also need to set glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA) to get the correct blending (and of course, glEnable(GL_BLEND)).


How to blend World Space Normals I am trying to blend two world space normals inside a shader. One comes from a tangent space normal map converted into world space using a classic TBN matrix and the other one is a mesh normal map in world space.

I found some interesting resources here :

Blending in Detail
Shadertoy - Normal Map Blending

But those blending technics seem to be only available for tangent space normals, especially the Reoriented Normal Mapping (RNM).
I tried to apply the RNM technic with unpack already done.

n1 += vec3(0, 0, 1);
n2 *= vec3(-1, -1, 1);

return n1 * dot(n1, n2) / n1.z - n2;

But this doesn't give expected results and I don't get why. Is there a way to apply the RNM blending on world space normals ?

Thanks a lot.

Edit: Here are some output and the result I am having with RNM. The A normal is not disturbed by the B normal, it gives some strange massively reoriented results.

float3 nv = dot(normal, viewDir);
color.rgb = nv;

Huge thanks to @MJP who answered this.
The aim is to avoid the simplification made when using tangent space normals.
Here is the paper : Blending in detail

But only implement equation (4) which gives you this.

float3 ReorientNormal(in float3 u, in float3 t, in float3 s)
{
    // Build the shortest-arc quaternion
    float4 q = float4(cross(s, t), dot(s, t) + 1) / sqrt(2 * (dot(s, t) + 1));
    // Rotate the normal
    return u * (q.w * q.w - dot(q.xyz, q.xyz)) + 2 * q.xyz * dot(q.xyz, u) + 2 * q.w * cross(q.xyz, u);
}

Original answer is here.