Components that can be used for 3d shadow analysis, sunhours of surfaces throughout the year in a 3d model? I want to develop a component for 3d shadow analysis of 3d city model. I decided to use ray tracing for this purpose. For every time step (for sun's dynamic position during the year) rays will be sent from sun (point light source) and with ray intersection tests surfaces which are and how long are in shadow will be determined. Are there any components for this purpose such as sun's position, ray tracing etc. or do I have to code everything from beginning to the end? Any answer would be very helpful.

I use Nvidia OptiX for Ray casting and tracing purposes. It is based on CUDA and gives you a library that runs on GPU and is capable of building and traversing efficient acceleration structures. You just supply a few programs such as intersection tests and shading and OptiX does the rest.
I personally think it is really easy to work with it but your milage may vary.
With this it should be very doable to make your project and even with decent running speed.


Use big frame buffer to control a small display I am really new in computer graphic area.
Suppose I have 1024*1024 frame buffer, and I want to use it to control a 512*512 display. What shall I do?
In my opinion, I will only read the information in 512*512, such as from (0,0) to (0,511) then go to next line, (1,0) to (1,511)...
Is this an usual way to solve the problem?

As stated, the question is a bit generic. It all depends on what part you want to display in the 512x512 display. If you have a 1024x1024 frame buffer from some source and only want to display 1/4 of the information, by all means copy the first 512 elements from the first 512 lines. If you want to support panning, you'll start at a different index in a different line. If you want to support scaling (with or without filter), you'll use a bigger or smaller set of source pixels.

It also depends on the type of the buffer. If it's just raw bytes, your suggested approach is basically the only way to go, but if it's a DirectX/OpenGL texture, or part of some other graphics API (Windows Bitmap, graphics engine, etc), there are usually specialized approaches to get a rectangular subset of a buffer.


Are some 3D objects "solid"? Do they have internal density? If so, when, and in which file formats? I am more or less a complete novice in the field of 3D modeling. I understand the basics of a polygon mesh and a texture; however, it was always my understanding that 3D models were, effectively, "hollow" inside the mesh, in the same way that a plastic skeleton Halloween decoration is hollow. It possesses a surface, but no internal density. This understanding has always worked well for me, as I primarily work with photogrammetry-created 3D models, for which this is undoubtedly true.
In my latest project, however, I am being asked to work with 3D models created using CT scan data and Slicer. Unlike photogrammetry images, CT scans do care about internal density and structure. I know Slicer can be used to create 3D models from CT scan images. What I'm uncertain about, however, is whether Slicer creates truly "solid" models from this data -- models which have internal density rather than simply a mesh "skin", where the "inside" of the model is programmatically distinct from the "outside" -- or whether it simply create a multilayered but still fundamentally hollow mesh (or series of meshes) based on boundaries between areas of different densities.
Is this a meaningful distinction? Am I fundamentally misunderstanding how the surface/internal geometry of 3D models works? Any advice is welcome.
(I took my best guess as to where to ask this question -- if this is outside the purview of this site, please let me know! If you know another place where I could go asking for answers, I would also appreciate that information. Thank you so much!!!)

Assuming what you’re referring to is this Slicer and the models are the ones produced by its Model Maker module: it looks like it’s creating hollow, surface-only models.

Specifically, judging by the crunchiness of the mesh shown in the top right of this screenshot, it looks like it’s using marching cubes to create a mesh from the isosurfaces at given density thresholds in the volume.
Most 3D model formats—especially the ones used for real-time rendering—are surface-only; as graphics hardware gets more powerful, though, there’s been wider use of volume rendering, often via raymarching, to render volume data directly rather than having to go through the intermediate meshing step. One common format for volume data is OpenVDB; it’s also possible to embed such data in Alembic files.


How can I use multiple texturing with shadow mapping? So, recently I followed this tutorial about shadow mapping, but in the tutorial it doesn't texture the object.. When I try to add a texture, it's distorted by the TexGen. Also I've tried to use glActiveTexture() with no success(dunno exactly how glActiveTexture() work).

So can anyone help me by adding textures into above code base? From there, I can learn how to use glActiveTexture() and overcome that TexGen issue.

Thanks in advance.. To note, I'm not using GLSL.

If multi-texturing is supported, texture coordinate generation will affect the active texture unit (quote from the documentation, emphasis is mine):

  When the ARB_multitexture extension is supported, glTexGen sets the texture generation parameters for the currently active texture unit, selected with glActiveTexture.

So you simply need to make sure texture coordinates generation is disabled for the color texture, but enabled for the shadow texture. As you guessed, this is done with glActiveTexture(), by writing for example:

glActiveTexture(GL_TEXTURE0); // Assuming the color texture to be #0
glDisable(GL_TEXTURE_GEN_S);
glDisable(GL_TEXTURE_GEN_T);
glDisable(GL_TEXTURE_GEN_R);
glDisable(GL_TEXTURE_GEN_Q);

glActiveTexture(GL_TEXTURE1); // Assuming the shadow texture to be #1
glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGenfv(GL_S, GL_EYE_PLANE, textureMatrix.GetRow(0));
glEnable(GL_TEXTURE_GEN_S);
// [...]

This being said, you're adding to yourself a lot of trouble by sticking with fixed pipeline instead of using GLSL.


Reading from buffer versus calculating on the fly performance I am creating a fast fourier transform algorithm for the compute shader - i am no expert on how GPUs really run optimally so thought i would ask here.
I have the option to calculate on the fly or precompute a lot of the trig function (cos and sin) values and store it in a read only float buffer.
My question is, though i am aware cos() and sin() is fast, how do they compare to simply getting the value precomputed in a buffer instead?
I don't mean a LUT here i do not need to interpolate between values from the buffer so i just get them values directly. I don't know how to really bench test shaders properly so i am unsure which is faster.
Perhaps some one knows more info on this?

My question is, though i am aware cos() and sin() is fast, how do they compare to simply getting the value precomputed in a buffer instead?

This will depend on your shader code and GPU model.

GPUs utilize latency hiding by executing independent ALU instructions while waiting for data to arrive from memory.
Accessing memory is usually the slowest operation on a GPU.

If your shader code allows for latency hiding, it may be faster.
The best option here is to profile the shader performance.

I don't know how to really bench test shaders properly so i am unsure which is faster.

I think the easiest way would be to download RenderDoc and get a frame capture of your application.
After that, you click on the timing icon (shown red) to get some measurements.

Although these are not super accurate measurements, they're good enough and sort of representative of the application performance.
For precise measurements, you'll need your GPU vendor's graphics profiling tools.

AMD has Radeon Graphics Profiler (RGP)
NVIDIA has NSight
INTEL has Graphics Performance Analyzers

Update: Below are some captures from RGP, showing GPU assembly instructions and instruction timings of various instruction types. You can see latency hiding in action below:

vmem : texture read
smem : constant buffer / descriptor<texture/buffer> read
ALU  : math ops (arithmetic / logic)
sync : waits for vmem/smem instructions to finish loading data from memory

Another capture showing instruction timings while first two texture loads not having latency hiding.


WebGL - issues doing a simple texture downsample I'm working on an image processing app which uses WebGL to apply shaders to a bunch of input photos. At one point in the algorithm, I need to create some image pyramids where I downscale the input photos by a factor of two each time, until I reach the smallest possible level.

For my downscaling routine, I'm using a really simple process:

Create a new, blank texture using texImage2D which is half the size of the original.
Attach this texture to a frame buffer and use it as a render target (bindFrameBuffer). I also set the viewport to match the new texture's size.
Draw a flat quad whose vertices cover the screen-space coordinates (-1 -> 1) using a 'copy-paste' shader which samples the original (full-size) texture. Note that the texture parameters are always NEAREST and CLAMP_TO_EDGE.

Here's the source code for my fragment shader:

precision mediump float;

uniform sampler2D u_inputImage;  // Original full-size texture

varying vec2 v_texCoords;

void main()
{
    gl_FragColor = vec4(texture2D(u_inputImage, v_texCoords).rgb, 1);
}

The result looks sensible, but the quality of the downscaled texture is pretty poor. In the following example, you can see how it looks jagged compared to the original, particularly along the edges of the table on the bottom-right. There are also some glitchy pixels on the top of the chair, and under the door handle:

Does anyone know why this simple method doesn't seem to work? Is there a better way of creating a downsampled texture using WebGL/OpenGL?

The downscaled picture is not smooth because you use "nearest" filtering.

Each block of 2x2 pixels in the original image is transformed into a single pixel in the downscaled image. Put another way, each pixel in the downscaled image represents the contents of 4 original pixels. Your current solution considers just one of those 4 pixels and ignores the others. This is what produces blockiness in the downscaled picture.

To fix this, each downscaled pixel must contain information from more than one original pixel. The simplest way to do this is by using a "box filter" where each downscaled pixel is computed as the mean of the 4 original pixels it covers. This can be achieved in a single texture fetch with linear filtering and the proper sampling position:

+---+---+
| A | B |  // ABCD => original pixels
+---X---+  // X => sample position in the middle of the 4 pixels (texels)
| C | D |
+---+---+

Linear filtering sampled at position 0 will give (A+B+C+D) / 4 or the average value.

Another computer graphic term for what you're doing is "mipmap generation". There are other filters which give better quality than the simple box filter I described.


Algorithms to Remove High Frequency Noise from Path Tracing I have been working on my own renderer for a while, and I'm wondering if there's any way to remove the Monte Carlo noise from the rendered image, besides waiting for a long time for it to converge?

The way I found is to blur the image, which is not really helpful, since it reduces the quality/sharpness of the image a lot. And I can achieve the same thing by rendering a small image with more samples, then scaling it up.

Is there any algorithm designed to deal with noise in the image in path tracing?

There are, and I am looking forward to seeing the specifics of other answers, but one way to deal with this is to not have the noise (or as much noise) in the source data to begin with.

The noise is coming from the fact that there is high variance in the rendering - the number of samples you've taken haven't converged enough to the actual right answer of the integral, and so some pixels are too high/bright and some are too low/dim (in each color channel).

The problem is this: If you use white noise random numbers to do your sampling, you may get samples clumping together like the image below.  Given enough samples, it will converge, but it will take a while before it gives good coverage over the sampling space.  Find a region of empty space in the image below (like in the lower right) and imagine that there was a small, bright light there and that the scene was dark everywhere else.  You can see how not having any samples there is going to make a problem for rendering.

Alternately, you could sample at even intervals like the below, but that will give you aliasing artifacts instead of noise, which is worse.

One idea is to use low discrepancy sequences and do quasi monte carlo integration (https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method).  Low discrepancy sequences are related to blue noise, which has only high frequency components.  Going these routes, you get faster convergence of $O(1/N)$ instead of $O(\sqrt{N})$.  These give better coverage of the sample space, but since there is some randomness (or random like qualities) to them, they don't have the aliasing issues that regularly spaced sampling does.

Here is a "jittered grid" where you sample on a grid, but use small random offsets within a cell size.  This was invented by pixar and was under patent for a while but is no longer:

Here is a common low discrepancy sequence called the Halton sequence (basically a 2d version of Van Der Corpus)

And here is a poisson disc sampling, using Mitchel's best candidate algorithm:

More information, including the source code that generated these images can be found here: https://blog.demofox.org/2017/05/29/when-random-numbers-are-too-random-low-discrepancy-sequences/


Flexible Vertex Format Constants in Direct3D 11 How do you migrate Flexible Vertex Format Constants (or FVF codes) from DirectX 9 to Direct3D 11?

Old code:

#include <d3dx9.h>

struct Vertex {

public:
    Vertex() : p(D3DXVECTOR3(0.0f, 0.0f, 0.0f)), n(D3DXVECTOR3(0.0f, 0.0f, 0.0f)), tu(0.0f), tv(0.0f) {}
    Vertex(D3DXVECTOR3 p, D3DXVECTOR3 n, float tu, float tv) : p(p), n(n), tu(tu), tv(tv) {}

    // Position of the vertex (in world space)
    D3DXVECTOR3 p;
    // Normal of this vertex
    D3DXVECTOR3 n;
    // Texture UV coordinates
    float tu, tv;
};

// D3DFVF_XYZ: Vertex format includes the position of an untransformed vertex.
// D3DFVF_NORMAL: Vertex format includes a vertex normal vector.
// D3DFVF_TEX1: Number of texture coordinate sets for this vertex.
#define VERTEX_FVF ( D3DFVF_XYZ | D3DFVF_NORMAL | D3DFVF_TEX1 )
#define VERTEX_FVF_SIZE D3DXGetFVFVertexSize( VERTEX_FVF )

New code without D3DX:

#include <DirectXMath.h>
using namespace DirectX;

struct Vertex {

public:
    Vertex() : p(XMFLOAT3(0.0f, 0.0f, 0.0f)), n(XMFLOAT3(0.0f, 0.0f, 0.0f)), tu(0.0f), tv(0.0f) {}
    Vertex(XMFLOAT3 p, XMFLOAT3 n, float tu, float tv) : p(p), n(n), tu(tu), tv(tv) {}

    // Position of the vertex (in world space)
    XMFLOAT3 p;
    // Normal of this vertex
    XMFLOAT3 n;
    // Texture UV coordinates
    float tu, tv;
};

// ??

P.S.: Can someone with enough reputation points create and add a directx9 tag.

Vertex formats are specified in D3D11 using the input layout. An example input layout description for your vertex might look like:

D3D11_INPUT_ELEMENT_DESC aInputDescs[] =
{
    { "POSITION", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, UINT(offsetof(Vertex, p)), D3D11_INPUT_PER_VERTEX_DATA, 0 },
    { "NORMAL", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, UINT(offsetof(Vertex, n)), D3D11_INPUT_PER_VERTEX_DATA, 0 },
    { "UV", 0, DXGI_FORMAT_R32G32_FLOAT, 0, UINT(offsetof(Vertex, tu)), D3D11_INPUT_PER_VERTEX_DATA, 0 },
};


What is the oldest hardware able to feasibly support Vulkan? What is the oldest hardware or minimum requirements conceptually able to support a fully hardware-accelerated Vulkan implementation? I am not referring to what implementations are currently available or likely to be written, but what could possibly be written.

Is there a specific set of features that must be provided by hardware in order for the manipulation primitives provided by Vulkan to be used?

Can Vulkan drivers be written for every video card that supports OpenGL?

For example, could one create a hardware Vulkan driver for the Number Nine Imagine 128, with 4M VRAM?

I apologize for how naive my question must be, and thanks in advance for any reply.

Khronos publishes a list of Vulkan compliant hardware here. The list also shows compliance to different flavors of OpenGL. I don't know all the card models, but for NVIDIA at least the GeForce GT 630 is supposed to work with Vulkan.

To answer your question a bit more, to support Vulkan a video card must support a certain number of programmable shader stages. Shaders is probably the biggest evolution of graphic cards in the last 10 years in my opinion. This already eliminates the graphic card you mentioned in the question.

This early Vulkan presentation from Khronos, states that "Vulkan 1.0 provides access to OpenGL ES 3.1 / OpenGL 4.X-class GPU functionalitybut with increased performance and flexibility." That can give you a basis for finding which individual features a graphic cards must theoretically support to have a Vulkan driver. See the new features introduced in OpenGL 4 at Wikipedia.

Among them:

Tessellation shaders
Indirect draw commands
Transform feedback (more flexibility in transform feedback)
Texture buffer objects (new texture formats only)

See also the sections "New Features" of the OpenGL 4.x specs.


Performance difference in OpenGL compute shader vs Vulkan Compute Shader vs Cuda/OpenCL and others I have been told there is no performance difference if you are skilled enough when it comes to gpu computing. And cuda only performs better because nvidia wants it to. I have also read graphic api's compute shaders are not as good as OpenCL/cuda. I don't understand how are compute shader different from Cuda/OpenCL and why would they be slow (if they are). I wanted to know this because OpenCL is difficult to understand and setup for me. Will there be a performance hit on using OpenGL compute shader and will Vulkan compute shader perform better than OpenGL's?

In general, you should not see significant performance differences running identical compute shaders via one API vs. another; in the end they’re running the same instructions on the same hardware. It’s possible for vendor-provided toolkits like CUDA or MetalPerformanceShaders to have more efficient implementations of a given algorithm on that vendor’s hardware than a more generic version of the algorithm, because they’ve been designed and tuned for the strengths of that hardware, but there’s nothing in principle that would prevent you from developing the same efficiencies on your own with enough optimization work.
Long story short: work with whichever API you find easiest to use, and invest some time in learning how to profile and tune your code for the hardware you plan to run it on.


Controlling "zoom" and "position" of a cubemap in shaders I am trying to mimic reflections in image composition using shaders.

I have gotten this far:

In this image the floor is "reflecting" the back wall, however the reflection is uncanny because the scale and the positions of the objects are very wrong.

For example the plant is nowhere even close to where it should be. Assuming I have a ground truth set of values for the camera parameters, is there a way I could "move" the reflections to better match what I expect to see? It doesn't have to be perfect, just better than what I have.

The way I am generating the image is the same as learnopengl's code:

void main()
{
    float ratio = 1.00 / 1.52;
    vec3 I = normalize(Position - cameraPos);
    vec3 R = refract(I, normalize(Normal), ratio);
    FragColor = vec4(texture(skybox, R).rgb, 1.0);
}

EDIT:

An important thing I should mention is, I don't have geometry, just a still image and information of where the floor is. So I don't have any depth, normals, or anything like that for most of the image.

Reflections based on a cubemap best represent a reflection at infinity. That is, since the only factor that a cubemap provides is direction, the cubemap acts as though it were at a distance from the reflective surface where only the direction to the reflecting objects matters. And that only happens if the reflecting objects are infinitely far from the reflective surface.

That works fine for reflecting distant environmental details like skies or distant landscapes. But it's not going to work with anything even remotely close to the reflective surface. And if the reflecting object and a reflective surface touch, then you can basically forget about it.

Cubemap reflections are simply not the right tool for this scene. You will instead have to employ other strategies for reflections, like rendering the back wall under the reflective surface (using transparency of the reflective surface to create the appearance of reflection), or doing screen-space reflections or something like that.

  I don't have geometry, just a still image and information of where the floor is. So I don't have any depth, normals, or anything like that for most of the image.

If you don't have geometry information or any equivalent to it, then you're pretty much out of luck. Accurate reflection of a scene requires knowing where the scene being reflected is relative to each point on the reflective surface. Even if the surface of reflection itself is flat, the reflected scene isn't. If you don't have some approximation of geometry information, and you cannot generate such a thing, then you cannot generate an accurate reflection.

If computer vision algorithms are available to you, I would suggest trying to fine one that can reconstruct the depth information of the scene from a single image. If that's not feasible, then you're going to have to live with an inaccurate reflection in one way or another.


Understanding of Microfacet monte-carlo path tracing I am trying to implement BRDF microfacet path tracing and I think I need a little push.
I will explain what I understand so that you can correct my mistakes.

I start with simple rendering equation:

Le and Li are radiance we have no control of so I cannot change anything about them. The only thing I can change is function F which is called BRDF. Specifically, I am using this Cook-Torrance microfacet specular BRDF I found here.

For functions F, G and D I choose these formulas.

Let's start with the Fresnel function. Greater the angle between H and Wo (or Wi, it should be the same) the more light is reflected. When the angle between H and Wo will be 90° then the function will return 1. That means that all of the light is reflected. That is understandable, but when I will look directly down to normal the result will be 0.14163. is really so little light reflected back? I thought that Fresnel gives percentage between reflection and refraction on the water surface for example. Is this right?

Following with function G - Geometric shading. I want to use the Smith shadowing model because I want to improve it later for multiple scattering BSDF. The equation goes like this:

As G1 can be used more types of functions, but I like GGX. D_GGX is evaluated for both incoming light and outgoing light. It describes how much light will NOT be blocked by surface imperfections.

And last function D - Normal distribution function. With this one, I have the biggest problems. It describes the probability that a new microfacet normal will be pointing to the given direction. This depends on roughness. So perfectly smooth surface will have every normal pointing upward, but as the surface gets rougher and rougher normals will start to point to different directions. With 100% rough material the chance of normal pointing a given direction should be uniform over the whole hemisphere. Because we are talking about probability the integral of probability over hemisphere should be 1. as this equation says, but I don't understand why there is a dot product of N and H.

Now I will explain how I am using these equations.

I start with generation microfacet normal H with distribution D_ggx. Spherical coordinates a generated like this: (epsilon is a random number from the interval (0;1>. I can't use 1 because we cannot divide by 0)

With H defined I can reflect Wo and get Wi. H is now half vector between Wo and Wi. Now I calculate F and G. I am not using D anymore because I already used it for generating the microfacet normal H (that is against what I read in this answer).  I divide them by 4 * dot(H, Wi)dot(nWo). This final number should be weight describing how much light [Li(Wi)] is reflected towards Wo. Together with a dot product of n and Wi from rendering equation. I think I have this part wrong.

I am not using any importance sampling for now because I think it is not necessary to work. Am I right? Of cause with importance sampling, the image would converge faster. This is my final equation.

Of cause, as you might guess when I implement this it returns almost totally black image. If you don't find any mistake in the math then the error has to be in code (but I will bet a 10$ it is a math problem)

If you need any specification of my decisions or goals I will happy to answer you.

Some more clarity on the fresnel.

F tells you how much light is reflected, and (1 - F) therefore tells you how much light is transmitted through the layer/material/surface.

Generally we put fresnels into two types Dielectric(water/glass) and Conductor(metals).

The approximation you are using is for a dielectric fresnel, so what you are actually modelling is a thin layer over the top of your diffuse. In renderers this layer is normally called the coat or clearcoat. It can be used to emulate laminated materials or a coating of water/sweat.

If you want to render metals you would need to swap out the fresnel for a conductive one. There are better approximations, but an easy artistic approximation is to swap the F0 of your schlick approximation with the rgb of your material. You can layer a diffuse below this if you need to but you may find with conductive it's not nessisary.

http://www.codinglabs.net/article_physically_based_rendering_cook_torrance.aspx

This is a good article on the topic for further reading.


Ray tracing scene with 2 lights and sphere I'm writing my implementation of ray tracer and I've got result which you can see at this image.

So you can see that some dark parts are overlapping. And I have them even in the part where it should be bright in my opinion. There are two lights one from above and the second one from right-bottom corner.

And I have two questions. Is it right? I really tried to google some references but failed. I'm not still sure that results in my imagination are correct ones.

And the second one where does it come from?

I use usual lambertian shading model and have something like this in my code:

pixel = (0, 0, 0)
normal = get normal of sphere in point of intersection;
normal = unit vector of normal;
for each light from light_set {
    light = unit vector of light;
    pixel = pixel + diffuseColor * max(0, scalar multiplication of light and normal);
}

I'm really struggling to find out the reason where this strange thing is coming from. But couldn't find it.

UPD0: added some of my real code. The part which is described here is next:

Material mat = surfaces[closestSurfase]->getMaterial();
Color pixel = Color(0, 0, 0);
Vector3f normal = surfaces[closestSurfase]->getNormal(point).getUnitVector();
Vector3f view = -ray.getDirection().getUnitVector();
for (size_t i = 0; i < lights.size(); ++i) {
    Vector3f normal_light = lights[i]->getDirection().getUnitVector();
    Vector3f specVect = (normal_light + view).getUnitVector();
    float intenst = lights[i]->getIntensity();
    pixel = pixel + mat.getDiffuse() * std::max(0.0f, normal_light.dotMultiply(normal)) * intenst;
}
result->setPixel(x, y, pixel);

Or you can find more here: https://github.com/minebv/ray_tracer/tree/lambertian_shading_fix/src

Described above is from Camera's method getRender.

UPD1: It looks like the problem in the mixing colors in gamma space instead linear one, as answered @PaulHK .

But I've still some kind of problem. Here you can see 2 renders.

At the left one I make all work after transformation all colors into linear space, I mean it's like this in my code:

pixel = pixel + mat.getDiffuse().toLinear() * std::max(0.0f, normal_light.dotMultiply(normal)) * intenst;

At the right one I make transformation after multiplying:

pixel = pixel + (mat.getDiffuse() * std::max(0.0f, normal_light.dotMultiply(normal)) * intenst).toLinear();

And I didn't forget to make pixel into gamma space again after all job.

As for me, the right one is more correct. But then there is a question: why should I make multiplication on angle into gamma space? Or is it my incorrect way to think about what is right?

UPD3:it looks like the whole thing really was about mixing colors in gamma space instead of linear one. Here is result with ambient light.

Also about overlapping lights, it was my incorrect way of thinking. I finally found reference material where they have the same thing. Found it at blender docs.

Thank everyone for answers.

I think what you're describing is what happens when you mix colours in gamma space. You should be using linear colour space when mixing.

When reading colours of lights and materials, convert them from gamma->linear using the inverse gamma function.

colourLin = pow(colourLight, 2.2);
... do lambert calculation here and accumulate in 'colour' variable ..

// On output (linear->gamma conversion)
gl_FragColor = pow(colour, 1.0/2.2);

There is an in depth chapter in GPU Gems on this topic: The Importance of Being Linear.


Why not calibrate all display devices to sRGB? Why do we not calibrate all devices to sRGB in the factory? The assumption is that every monitor is sRGB anyway. So why not calibrate them to that? That would make people see same color as intended on a wide range of devices.

Simple answer: because it costs. :) Some higher-end monitor models are indeed calibrated at the factory (and come with a calibration report sheet) but it would not be cost-effective to do so for all models in the product line.

Color reproduction varies even between batches of the exact same panel model, so it isn't possible to just use a set calibration for a given monitor model, it need to be done on a unit-by-unit basis. So to save costs, they instead determine the color reproduction curves of the "average" panel and use that for an ok but inconsistent result.


Pixel dispersion (dissolving) algorithms This YouTube video of a flip-dot display (physical b/w pixels) shows the reverse of an effect that might be called dissolve or dispersion, i.e. a text emerges from noise by pixels moving in to form letters. Here's a short cropped clip showing just the "crystallization" (wouldn't that be an appropriate name?):

(source: BREAKFAST NY)

Looking for more info on this or similar effects I found After Effects: Text That Blows Away Like Sand which has a Shatter effect, apparently a 3D particle system with physics simulation.

For a particle system, I can imagine that "dissolving" every pixel (of a character) randomly wouldn't be too hard. Reversing the effect as shown in the video could be done using a precalculated sequence or even just animated sprites for each letter. Or, each pixel in the noise gets chosen to gravitate towards a letter (seems more complicated, may require clustering).

Then I found an implementation and live demo:

Particles text effects

Uses particles with a seek behavior to make up a word.
The word is loaded into memory so that each particle can figure out their own position they need to seek.
Inspired by Daniel Shiffman's arrival explantion from The Nature of Code. (natureofcode.com)

https://www.openprocessing.org/sketch/377231

The reference resolves to

http://natureofcode.com/book/chapter-4-particle-systems
http://natureofcode.com/book/chapter-6-autonomous-agents/ (6.4 Arriving Behavior)

I think the effect is very similar, so I think they used a particle system.

Q: Is there an approach that does not require particles (i.e. less memory-expensive)?

Just for comparison, here's an attempt at creating a "normal" animated noise dissolve (random dither), as suggested in a comment, but I think it's obvious that the pixels in the original do move and it's not just a binary blending effect:

It requires at least one random value for each pixel: on which frame of the animation to become active.

This is what trichoplax's solution looks like (pseudorandomly swapping 10,000 neighboring pixels in each of 100 frames, and then reversing it using the same pseudorandom number sequence, just reversed):

A very simple low memory approach

If you really want to use as little memory as possible, it can be done with not much more memory than that required to store a single image (the first frame) provided it is acceptable to do some preprocessing in advance.

If you copy the following jumbled image, this jsfiddle will take it as input:

It will then move the pixels around one step at a time so they drift into place to give the original image again:

This does not require any memory for tracking particles. The algorithm is very simple:

Pick a pixel at random
Swap it with one of its 4 neighbours
Repeat a million times

No additional memory is required as no particles are being used. We're simply swapping pixels at random, with no destination locations in mind. The key is in choosing that apparently jumbled initial image to have the pixels in exactly the right places such that applying a million random swaps will happen to leave all the pixels in exactly the right places to give the desired text.

Setting up the initial jumbled image

To do that, this preparatory jsfiddle takes an image as input and outputs a jumbled image that is precisely arranged to work as input for the main jsfiddle.

This one takes more memory, but is still a very simple algorithm:

Generate a large number of random numbers and store them
Using these numbers in reverse order* (using the last first):

Pick a pixel at random
Swap it with one of its 4 neighbours
Repeat a million times

This is exactly the opposite of what the main jsfiddle will do, so the jumbling will be completely reversed, restoring the original image. You can paste in a colour image, or if you want just text with only black and white pixels that will work too. The example image shown above is between these two extremes, as it has greyscale pixels for the antialiasing, which wander around alongside the black and white pixels.

A million pixel swaps works for fairly small images. For larger sizes this won't jumble them enough to completely obscure the original image, so you would need to adjust the number of swaps.

Languages without reseeding

Note that this only works because the pseudorandom number generator is reseeded at the start of both algorithms, to ensure that they are both working with exactly the same list of "random" numbers. Most languages will allow you to reseed the random number generator, making this an easy approach. JavaScript does not, so the jsfiddles include an implementation of xorshift to allow reseeding.

Even with a language that does allow reseeding, if the hardware is particularly limited then you may want to consider xorshift as it is very fast and uses only 4 bytes of memory for its state. It's also only a few lines of code.

*Note that the order isn't quite the same as just reversed. In this example code, three random numbers are used for each pair of pixels to be swapped. So the order of each triple must be kept the same to avoid choosing different pairs of pixels. For example:

[1, 2, 3], [4, 5, 6], [7, 8, 9]

must be reversed to

[7, 8, 9], [4, 5, 6], [1, 2, 3]

in order that the same x value, y value and direction be chosen for each pixel pair when running in reverse.


How to work around the pointer limitations of GLSL? (I am new to Computer Graphics in general)
I am learning how to ray trace from a book called ''Peter Shirley - Ray Tracing in One Weekend''. In the book, the code is written in C++. I have intermediate-advanced level understanding of good ol' C, but I've never written even simple programs in C++. I don't pretend to understand everything the code says in the book. But, that is the fun part -- figuring it out.
But, for ray tracing, even C is slow. And writing GLSL fragment shaders is fast, even on my potato PC. And GLSL is closer to C than to C++. So, I'm trying to follow through the book using GLSL. But, GLSL has many limitations and I was hoping if you could teach me how to get around those limitations.

Pointers. GLSL doesn't have pointers. In the book, there is a function
bool hit_sphere(const vec3& center, float radius, const ray& r)
I believe that ''address of'' operator just points to the variable that is passed when the function is called. Without pointers, how to call a function by reference in GLSL?

Late binding. In the book, every object is a class which exposes an interface function called hit which returns if a ray hits the object. How does one implement that in GLSL structs?

I'll get to the specifics of your question presently, but first it is important to discuss the broad thesis of your idea: porting C++ code as is to GLSL.
This is a bad idea. The execution model of shaders is fundamentally different from the way a normal program (even a multi-threaded program) works. Attempting to port such code from one execution model to another is not a good idea. Your best bet is to try to make the algorithms involved fit the nature of the rendering pipeline. That is, you need to understand how rendering works, then try to apply the mechanics of rendering to the problem you're trying to solve.
For example, in regular CPU code, you might implement a ray-tracer by iterating over a bunch of locations on the screen and doing ray-scene intersection tests for each of them. Doing that iteration on the GPU in GLSL code is wrong. Instead, you should map rendering concepts to the needs of a ray-tracing algorithm. You might have each fragment shader do a single ray/scene processing step, with the fragment shader inputs being a direction from the camera. There would be no explicit loop over the screen's pixels; the loop happens implicitly in the rasterizer. The rasterizer generates a fragment shader invocation for each pixel-sized area of a rendered object. You would simply render a single quad that covers the whole screen.
Things like that are how you need to think of these kinds of problems in a GPU-centric computation model.

GLSL does not have pointers. But it does have shader storage buffer objects, which allow you to define arbitrarily large regions of memory that the shader can read from. A "pointer" in this model is just an index into an array of data structures. You have to do the indexing explicitly in GLSL code.
Similarly, any form of "late binding" can just be a switch statement based on some value in a data structure. The different casees define which function to call.


Precision problem on AMD I have a problem with precision on AMD in shaders (hlsl). In vertex shader I calculate UVs for particle using modulus operator on float. With simple % there is a problem on Nvidia and on AMD (e.g. 7 % 7 = 7) so I have written modulus as:

float Mod( float x, float y )
{
     return x - floor(x / y) * y;
}

This have fixed the problem on Nvidia but AMD still returns wrong values.

Any idea what is going on? And how to fixed it?

I know I can use:

float( int( x ) % int( y ) )

but I think it is slower and I wonder if there is any better way.

In HLSL, you can use the built-in fmod function for this.

I'm not sure what's going wrong with your implementation, though. It looks mathematically correct. It's conceivable you're encountering a precision issue with the subtraction, depending on the inputs.

BTW, an alternative implementation which avoids the subtraction is:

float Mod(float x, float y)
{
    return frac(x / y) * y;
}


Extract visible vertices from a 3D geometry model Let's assume we have a 3D model consisted of XYZ-formatted vertices (such as point cloud model). My problem is given the position and normal of camera or camera projection (i.e. where I am looking at the points- usually given by my API), how can I extract the visible points and remove the non-visible points?

For instance here points corresponding to the black point cloud box and the back surface of the red box are removed. In point clouds, points have a size. So let's assume they have a size, say 5 pixel radius just for modeling purpose.

P.S. It doesn't have to be perfect. Just any approach to somehow remove the non-visible points so to decrease the total points with not much impact on the visible points will work.

Here is a test point cloud model to work with: https://www.dropbox.com/s/dfhyy9xa5i9o2fz/testPLY.ply?dl=0

Idea A:
Draw an invisible mesh that will occlude the points we don't want.

Create a mesh from the point cloud.
Render that mesh to a depth buffer but not to the color buffer.
Render the point cloud using a depth test "closer or equal".

This approach should give the expected result, but the problem with it is the first part, which is not trivial at all.
Idea B:
Approximate the mesh with fat particles, that are big enough to hide points in the background.

Pick a size that is roughly the typical distance between two points.
Use that size as a particle diameter.
Render the point clouds using such large particles, to a depth buffer but not to the color buffer. Maybe offset that point cloud a little, so it's a bit farther from the camera.
Render the point cloud using points this time, and using a depth test "closer or equal".

This method should be a lot simpler to implement than A, because there is no need to triangulate anymore. The result will not be perfect though: some points that should be visible might disappear, and some that shouldn't be visible will remain visible. Tweaking the particle diameter and offset will yield different results.
Idea C:
As a very cheap trick, just render the point cloud with a fog function. The farther points are from the camera, the less visible they are. This method won't hide any point, but still help reading the volume.


Reducing artificial rings in mean curvature of mesh I have an isosurface/mesh (generated via marching cubes) of a micro-CT scan. Originally the surface looks very noisy, which becomes prevalent when the mean curvature is visualized:

To reduce the noise I've applied Gaussian smoothing to the imagedata and regenerated the mesh. This time, artificial looking rings appear:

I would like to reduce these circles/rings/terracing artifacts to obtain a more smooth distribution of the curvature values.
I was wondering if anybody would know the name of this problem since I'm struggling to find anything similar online. All help is appreciated!

This is just an educated guess, but the artifacts in your visualization of the curvature values reminded me of Moiré patterns. The images in the Wikipedia article don't show circle patterns like the ones you are observing, but I have seen similar patterns before when I tried to visualize highly oscillating functions. (Edit: The function you are visualizing, i.e. the curvature, is likely not oscillating quickly. However, the following example still illustrates that it may not only be the discretization or mesh generation algorithm that is problematic, but also the computation of the curvature.)
For example, if you try to visualize the function $f(x,y) = |\cos((x^2+y^2)^2)|$ for $x$ and $y$ ranging from $-3.3$ to $3.3$ on a grid of $256\times 256$ pixels, what you get is the following image:

The Moiré patterns appear in this image since the function $f$ is only evaluated at a single point (x,y) for every pixel, although each pixel covers not just a single point but an area. A better visualization of the function $f$ would involve integrating over the corresponding area for each pixel, which would result in a uniform gray color near the image boundaries reduces the Moiré patterns.
My guess is that there is something similar happening in your situation. As for solving your problem, this essentially means that the curvature is computed in an erroneous way if the discretization is too coarse. So you could either try to improve the algorithm that is used for the computation of the curvature or simply use a finer discretization. Unless, of course, if the rings really are a feature of your surface, in which case they will not disappear as lightxbulb pointed out.
In case you are interested, I have attached the C++ code that I used to generate the above image. Note that if you increase X and Y while keeping lX and lY constant, the discretization is refined and the Moiré patterns will disappear.
#include <fstream>
#include <vector>
#include <string>
#include <cmath>

using namespace std;

void WriteGrayscalePPM(const std::string& file, const std::vector<unsigned char>& values, const unsigned int X) {
  const unsigned int Y = values.size ( ) / X;

  std::ofstream image(file.c_str(), ios::out | ios::binary);
  image << "P6\n" << X << ' ' << Y << "\n255\n";
  for (unsigned int i=0; i<X*Y; i++) {
    unsigned char color[3] = {values[i], values[i], values[i]};
    image.write((const char*)color, 3*sizeof(char));
  }
}

int main(int argc, char **argv) {
  const int X = 256;
  const int Y = 256;
  const double lX = 3.3;
  const double lY = 3.3;

  std::vector<unsigned char> values(X*Y, 0);

  for (int y=0; y<Y; y++) for (int x=0; x<X; x++) {
    double xd = lX / (X/2) * (x - X/2);
    double yd = lY / (Y/2) * (y - Y/2);
    double d = sqrt(xd*xd + yd*yd);

    values[x + y*X] = 255*abs(cos(d*d*d*d));
  }

  WriteGrayscalePPM("moire.ppm", values, X);

  return 0;
}


Vulkan clear color is wrong I am following https://vulkan-tutorial.com/Introduction to get myself familiar with the Vulkan API. I have gotten to the point where my application is able to draw a triangle. But for some reason the clear color is way brighter than the RGB value that i specified in my code. I am setting the color like this:
    VkClearValue clearColor = {0.2f, 0.2f, 0.2f, 1.0f};

    VkRenderPassBeginInfo renderPassBeginInfo = {
        .sType = VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO,
        .renderPass = renderPass,
        .framebuffer = swapChainFramebuffers[i],
        .renderArea = {
            .offset = {
                .x = 0,
                .y = 0
            },
            .extent = swapChainExtent
        },
        .clearValueCount = 1,
        .pClearValues = &clearColor
    };

An RGB value of [0.2, 0.2, 0.2, 1.0] or [50, 50, 50, 255] should be a very dark gray, almost black. Instead i am getting is this:

This looks more like an RGB value of [150,150,150,255]. To get a color close to what i want i have to instead go with something like this:
VkClearValue clearColor = {0.02f, 0.02f, 0.02f, 1.0f};

So it almost seems like everything is shifted by a decimal. Any idea what might cause this issue?

This looks like the VkFormat being selected for the VkSurfaceFormat, most likekly VK_FORMAT_R8G8B8A8_SRGB. Try VK_FORMAT_R8G8B8A8_UNORM for the VkFormat with a corresponding colorspace (like VK_COLOR_SPACE_SRGB_NONLINEAR_KHR). That should give the dark color you are expecting.


draw a chord in three.js as in d3.js I'm starting to learn three.js and want to create a chord diagram as possible in d3.js, but from ground up. I've created a scene and have circle segments on xz axes. But I got stuck on the chords. I looked up d3.js code (path+quadratic curve) but need some good example or explanation how to do it in three.js.

What I'd like to do is to have chords with stroke and fill joining my circle segments with different widths at both ends, nicely curved and thinner in the middle (same as d3.js) with a way to set z-index and opacity in the future.

Some example could be really helpfull, thanks.

I want to achieve something like this:

Ok, I've found the solution myself.

It goes like this:

// create shape
shape = new THREE.Shape();

// move it to first point of the chord
shape.moveTo(0+(RADIUS-2)*Math.cos(-1.4639537206164), 0+(RADIUS-2)*Math.sin(-1.4639537206164));

// create all the parts afterward
shape.absarc(0, 0, (RADIUS-2), -1.4639537206164, -1.3041396376011, true);
shape.quadraticCurveTo(0, 0, 0+(RADIUS-2)*Math.cos(4.5262534414805), 0+(RADIUS-2)*Math.sin(4.5262534414805));
shape.absarc(0, 0, (RADIUS-2), 4.5262534414805, 4.6513392175639, true);
shape.quadraticCurveTo(0, 0, 0+(RADIUS-2)*Math.cos(-1.4639537206164), 0+(RADIUS-2)*Math.sin(-1.4639537206164));

// create geometry
geometry = new THREE.ShapeGeometry( shape );
// create mesh
mesh = new THREE.Mesh( geometry, new THREE.MeshBasicMaterial( { color: 0x000000, side: THREE.DoubleSide } ) ) ;
// rotate
mesh.rotation.x = 90 * Math.PI/180;
// add to scene
scene.add( mesh );

1) create Shape object
2) move offset to the first point on the arc
3) create arc with the same center and smaller radius as circle (start angle of the first point, end angle second)
4) use quadratic curve with control point (circle center) and the third point
5) use arc the same way as before
6) complete shape with quadratic curve and the last point
7) set geometry, material, add it to scene


How to detect a screen in a photo Is there a way to make a difference between a photo of an actual object and a photo of an object through a computer or smartphone screen.

For example: how can I detect that the second image is taken from a screen?

Thank you very much!

The first things I would try is to see if smartphone pixels are visible (if the photo is high res) or if strange aliasing occurs (your example image shows both).

Another classical test (but not adapted to any scenes) is to detect perspective bias: if the camera is not exactly parallel to the smartphone screen, then 3D objects projected on the image on screen are projected again to the camera captor through a slightly different perspective. If the scene contains spheres, discs (physical or as reflects, for instance), perpendicular angles, or any similar invariant, then you can detect this.   (this kind of trick is also used to detect images photoshoped by composition of several images).


What are Metal and Vulkan? I keep coming across the terms 'Metal' and 'Vulkan' on this site.
Guessing this to be some CG software, I'm interested to know what these are exactly and what the intended and most common usage is.

A quick search shows me a video about why metal in CG looks fake: https://www.youtube.com/watch?v=NjbLYFXrfFU

Obviously not the answer, most answers talk about compute kernels. I'm familiar with kernel inversion in machine learning but not like Android kernels and the such. Hope you can help me out. Your input would be most valuable.

Metal and Vulkan are low-level 3D graphics APIs, in much the same way that OpenGL and Direct3D are 3D graphics APIs. Metal is Apple's API, usable on iOS and MacOS. Vulkan was developed by the Khronos Group, who also oversees the various forms of OpenGL. Vulkan is directly usable on all non-Apple platforms, and there is the MoltenVK project which provides a translation layer from Vulkan to Metal (since the APIs are not that different), thus allowing Vulkan applications to run even on Apple platforms.

Metal, Vulkan, and Direct3D v12 represent a dramatic departure API-wise from prior low-level graphics APIs. They use a command-buffer-based interface, where you record rendering commands into a command buffer object, then execute those commands at some later period. The more immediate style of APIs like OpenGL and D3D-pre-12 have no intermediary command buffer. When you call a rendering function, it is given to the GPU "right then" and that's the end of it.

These styles of API are important because they allow threaded creation of rendering commands. In the immediate APIs, you can only effectively have one thread issuing rendering commands. In command-buffer APIs, multiple threads can build multiple CBs simultaneously, and then one thread submits them all as a single bunch of work. This allows better utilization of CPU resources for CPUs with lots of cores.


Degenerate case of normal computation in marching cubes I'm implementing the marching cubes, reference is this. I managed to get the geometric reconstruction (the mesh) however I do struggle with the normals.

In section 4 they explain the gradient can be firstly estimated using central difference using 8 cubes sharing the vertex $(i,j,k)$. However consider for example case 1 (figure 3) in the paper assume for example you have 8 cubes with one vertex "on" and all the other "off". All the central difference (assuming binary values) would give you 0, therefore I wouldn't be able to estimate the normal in the cube vertex, and later linear interpolate.

Is there some subtlety I'm missing? I wouldn't personally use central difference, but assuming I'm sweeping in a certain direction and then interpolate a point in a cube edge I would use for example forward difference, which to me make more sense, however I'm not 100% sure of the "sense" of this.

In their scheme they give a good explanation of why their approach make sense.
Not sure I can justify mine.

The paper doesn't say to compute the normals for the 8 "cubes" around the cube in question. It is saying to compute the normals for all the vertices of the cube you are testing. To create those normals the algorithm would need density values from the surrounding cubes but not the normals of the surrounding cubes.

The paper labels those vertices v1 to v8. So if v1 is the only vertex whose density value is 1 and the density value for all 7 other vertices is zero then v3,v6,v7 and v8 would have normals of zero (assuming the adjacent cubes were empty). But v1,v2,v4 and v5 would have non-zero normals since v1 is guaranteed to have a non-zero density and at least 1 of x,y or z would be non-zero. Interpolating those normals to the triangle with vertices at e1,e4 and e9 should give a normal pointing roughly at v7.

When reading a paper like this one (ie one that has been used around the world to visualize life saving data, and help countless people) if I think I have found an error I stop and take a long hard look for my mistake instead of looking for the mistake of the authors. Just saying.


Problem using Decaling with Stencil to avoid Z-fighting I need to create a decaling into a surface like a mesh, and to avoid Z-fighting I've been using Stencil Buffer to Turn-off depth test and make the decaling over the surface the Stencil Op permits. In fact that decaling fails when we turn the camera in a such way that the viewer has more than one surface point "competing" for the same pixel. This happens when the surface isn't a plane, in our example we have a mesh representing a terrain. The problem is that we turn-off the depth test and the decaling object is always visible, even when we have a surface point closer to the camera. Is there someone here that had the same problem ? Do I have to quit stencil solution to avoid Z-fighting ?
Without problem:
The Problem:

It sounds like this code is using the stencil buffer to mask off the locations to draw the line. This isn't a "typical" decal approach. Normally the decal is draw directly into the scene and "depth bias" is turned on for the graphics state (OpenGL) or the graphics pipeline (Vulkan). Then turned off when done, again for OpenGL.
A solution that doesn't change the current rendering approach (which i guessed at) is to use a depth buffer when rendering the terrain. Then pass that depth buffer as an input to your decal shader, read the depth buffer and compare it to the depth of your current fragment. Then discard the fragment if the depth test fails. Basically, it is doing depth buffering by hand. It isn't to hard to set up and there are lots of tutorials out there to help get you going.
It also might be possible to just change the stencil buffer based on depth values this would be dependent on how the stencil is being generated. If it is regenerated each frame this could be a quick and easy solution.
On the other hand, if you are drawing lines or polygons directly, then you really should look into using depth bias settings. Assuming you are using OpenGL this is turned on before rendering the decal then turned off after rendering it. The hardware then computes an offset depending on how far the camera is away from the object up to some limit (which is also programable). This approach would allow the depth testing to be left on, it is view independent so it works from any direction, and it would allow the decal to be textured.
Also: The issue described here isn't really z fighting. Z fighting usually occurs when two fragments are being rendered to the same location and quantization causes the fragments to "fight" for visibility. As the camera moves one of the two fragments wins the "fight", but the winner keeps changing depending how much floating point error there is.


How to assign/calculate triangle texture coordinates Below is a triangle with its vertices and uv coordinates. These are read from
the input model file.

I don't understand why the texture comes in different orientation than the image.

here is the method to calculate uv coordinate for the hit point(ray-triangle code from Moller, Trumbore):

     public bool GetIntersection(Vector rayOrigin, Vector rayDirection, ref
              float hitDistance, out Vector uv)
    {
        uv = null;

        // begin calculating determinant - also used to calculate U parameter
        Vector pvec = rayDirection.Cross(edge2);

        // if determinant is near zero, ray lies in plane of triangle
        float det = edge1.Dot(pvec);

        const float EPSILON = 0.000001f;
        if ((det > -EPSILON) && (det < EPSILON))
            return false;

        float inv_det = 1.0f / det;

        // calculate distance from vertex 0 to ray origin
        Vector tvec = rayOrigin - verts[0];

        // calculate U parameter and test bounds
        float u = tvec.Dot(pvec) * inv_det;
        if ((u < 0.0f) || (u > 1.0f))
            return false;

        // prepare to test V parameter
        Vector qvec = tvec.Cross(edge1);

        // calculate V parameter and test bounds
        float v = rayDirection.Dot(qvec) * inv_det;
        if ((v < 0.0f) || (u + v > 1.0f))
            return false;

        Vector uvHit = uvVectors[0] * u + uvVectors[1] * v + uvVectors[2] * (1 - u - v);
        uv = new Vector(uvHit[0], uvHit[1], 0);

        // calculate t, ray intersects triangle
        hitDistance = edge2.Dot(qvec) * inv_det;

        // only allow intersections in the forward ray direction
        return hitDistance >= 0.0f;
    }

And here is the method in ImageTexture class to convert the hit point uv coordinate:

    public Vector Interpolate(Vector uv)
    {
        float u = uv[0] - (float)Math.Floor(uv[0]);
        float v = uv[1] - (float)Math.Floor(uv[1]);
        u *= bmp.Width - 1;
        v *= bmp.Height - 1;

        Color cuv = bmp.GetPixel((int)u, (int)v);

        return new Vector(cuv.R, cuv.G, cuv.B);
    }

Also how are the uv coordinates that come from the input model are supposed to be calculated? If this is a big mesh(a terrain) with many triangles how is the uv coordinates for each triangle vertex is calculated?

Thanks.

UV or barycentric coordinates represent a point on a triangle as a weighted average of the three vertexes. You can use those coordinates to smoothly interpolate any value over the triangle, like texture coordinates or the normal vector. So think of it like a two-step process. We map the world coordinates of the intersection point to uv coordinates in "triangle space", and then use those to map to another space, in this case "texture space". Your ray hits triangle ABC and you determine the uv coordinates of the intersection point, let's say (0.3, 0.5, 0.2). Intuitively, we're sort of saying the intersection point is 30% A, 50% B, and 20% C. Each vertex would have a texture coordinate, i.e. a pixel in the texture image, associated with it (that's part of modeling / setting up your scene, not something calculated at runtime). Mix the texture coordinates at A, B and C in that ratio and you'll get the point on the texture image associated with that point on the triangle.

This scratchapixel page has some good diagrams and clear explanation.

edit: I'm sorry, I missed the part about how to calculate the coordinates in the first place. The page I linked explains it better than I could anyway.


Open GL Shader Storage Buffer Objects to replace Vertex Attributes At first: I already posted this on the stackoverflow community. I'm new and didn't know about this community here. I apologize.

I basically got the same question as the guy who asked here: https://stackoverflow.com/questions/36457138/dynamically-sized-arrays-in-opengl-es-vertex-shader-used-for-blend-shapes-morph.
Especially his last unanswered question bothers me too.

So I also want to use an arbitrary number of blendshapes for each mesh I'm processing. At the moment I'm using a fixed number and treat the shapes as vertex attribute. The advantage here is that I always have the relevant data for the current vertex availiable. Now, if I want to use an arbitrary number of shapes, I figured I'd use SSBOs since their clue is exactly what I want: Dynamically sized data. However SSBOs are, as far as I understand it, static and for each processed vertex in the shader I have the blendshape data for the whole mesh availiable. That means I would have to introduce some kind of counter and try to pick the correct data piece out of my SSBO for each vertex.

Is this understanding correct? I'm really not sure whether this is the optimal solution, maybe you can give me some hints.

Thank you for your answers!

The counter you want is built in the shader as gl_VertexID.

However you have 16 attributes of 4 components each to work with. Take 2 components for the texture coordinates and you have 62 float values to play with. If you need 6 components per blend shape (3 pos + 3 normal) you get 10 blend shapes to work with per object per frame. If you leave off the normal as static then you get up to 19 shapes.

Times when you want to blend between more than 10 shapes should be very rare. And even then you can probably get away with making another shape to get the resulting shape you want.


Using RGB texture format for image load/store As stated in documentation possible values for layout qualifiers of image formats are (for example floating point):

Floating-point layout image formats:

rgba32f
rgba16f
rg32f
rg16f
r11f_g11f_b10f
r32f
r16f
rgba16
rgb10_a2
rgba8
rg16
rg8
r16
r8
rgba16_snorm
rgba8_snorm
rg16_snorm
rg8_snorm
r16_snorm
r8_snorm

So we can use R, RG and RGBA formats. But why RGB (except r11f_g11f_b10f) is not allowed?

RGB8 format and similar are often not supported by GPUs (or only supported for a subset of GPU operations) because they don't have a power-of-two number of bytes per pixel. RGB8 would be 24 bits or 3 bytes per pixel. All the formats in that list have 1, 2, 4, 8, or 16 bytes per pixel.

Using a power of two simplifies addressing calculations in the hardware, because it can be implemented using a shift instead of a full multiply. (A shift by a fixed value such as 2 bits can even be implemented in hardware without any actual "calculations" at all, just by routing wires.) It also means that pixels are always nicely aligned with respect to boundaries of memory pages, cache lines, etc., which no doubt simplifies many things in the hardware.


How are volumetric effects handled in raytracing? How are volumetric effects such as smoke, fog, or clouds rendered by a raytracer?  Unlike solid objects, these don't have a well-defined surface to compute an intersection with.

Overview

The appearance of volumes (also called participating media) in nature is caused by tiny particles, such as dust, water droplets or plankton, that are suspended in the surrounding fluid, such as air or water. These particles are solid objects, and light refracts or reflects off of these objects as it would on a normal surface. In theory, participating media could therefore be handled by a traditional ray tracer with only surface intersections.

Statistical Model

Of course, the sheer number of these particles makes it infeasible to actually raytrace them individually. Instead, they are approximated with a statistical model: Because the particles are very small, and the distance between the particles is much larger than the particle size, individual interactions of light with the particles can be modeled as statistically independent. Therefore, it is a reasonable approximation to replace the individual particles with continuous quantities that describe the "average" light-particle interaction at that certain region in space.

For physically based volumetric light transport, we replace the inconceivably many particles with a continuous participating medium that has two properties: The absorption coefficient, and the scattering coefficient. These coefficients are very convenient for ray tracing, as they allow us to compute the probability of a ray interacting with the medium - that is, the probability of striking one of the particles - as a function of distance.

The absorption coefficient is denoted $\sigma_a$. Say a ray of light wants to travel $t$ meters inside the participating medium; the probability of making it through unabsorbed -- i.e. not hitting one of the particles and being absorbed by it -- is $e^{-t\cdot\sigma_a}$`. As t increases, we can see that this probability goes to zero, that is, the longer we travel through the medium, the more likely it is to hit something and be absorbed. Very similar things hold for the scattering coefficient $\sigma_s$: The probability of the ray not hitting a particle and being scattered is $e^{-t\cdot\sigma_s}$; that is, the longer we travel through a medium, the more likely it is that we hit a particle and are scattered into a different direction.

Usually, these two quantities are folded into a single extinction coefficient, $\sigma_t = \sigma_a + \sigma_s$. The probability of traveling $t$ meters through a medium without interacting with it (neither being absorbed or scattered) is then $e^{-t\cdot\sigma_t}$. On the other hand, the probability of interacting with a medium after $t$ meters is $1 - e^{-t\cdot\sigma_t}$.

Rendering with Participating Media

The way this is used in physically based renderers is as follows: When a ray enters a medium, we probabilistically stop it inside the medium and have it interact with a particle. Importance sampling the interaction probability$1 - e^{-t\cdot\sigma_t}$ yields a distance $t$; this tells us that the ray traveled $t$ meters in the medium before striking a particle, and now one of two things happens: Either the ray gets absorbed by the particle (with probability $\frac{\sigma_a}{\sigma_t}$), or it gets scattered (with probability $\frac{\sigma_s}{\sigma_t}$).

How the ray is scattered is described by the phase function and depends on the nature of the particles; the Rayleigh phase function describes scattering from spherical particles smaller than the wavelength of light (e.g. our atmosphere); the Mie phase function describes scattering from spherical particles of similar size than the wavelength (e.g. water droplets); in graphics, usually the Henyey-Greenstein phase function is used, originally applied to scattering from interstellar dust.

Now, in graphics, we don't normally render pictures of an infinite medium, but render media inside a scene consisting of hard surfaces as well. In that case, we first fully trace the ray until it hits the next surface, completely ignoring the participating medium; this gives us the distance to the next surface, $t_{Max}$. We then sample an interaction distance $t$ in the medium as described before. If $t \lt t_{Max}$, the ray hit a particle on the way to the next surface and we either absorb it or scatter it. If $t \geq t_{Max}$, the ray made it through unscathed and interacts with the surface as usual.

Outlook

This post was only a small introduction to rendering with participating media; among other things, I completely ignored spatially varying coefficients (which you need for clouds, smoke, etc.). Steve Marschner's notes are a good resource, if you're interested. In general, participating media are very difficult to render efficiently, and you can go a lot more sophisticated than what I described here; there's Volumetric Photon Mapping, Photon Beams, Diffusion approximations, Joint Importance Sampling and more. There's also interesting work on granular media that describes what to do when the statistical model breaks down, i.e. particle interactions are no longer statistically independent.


OpenGL Unable to retrieve the values of data sent to shader using textures I've been following these tutorials and i can't figure out how to send data to shaders using textures.

https://en.wikibooks.org/wiki/OpenGL_Programming/Scientific_OpenGL_Tutorial_02

In this tutorial they are using GL_LUMINANCE which has been removed from OpenGL 3.1 and above. I was wondering what should i use instead of GL_LUMINANCE? I tried GL_RGB, GL_RED but i was unable to retrieve the values.

What GL_LUMINANCE as a texture format actually does is store one component of data per pixel that is used for the red, green and blue color components. So, a classic greyscale image (It doesn't use it for the alpha component, though, that's why there was also GL_LUMINANCE_ALPHA for two compnent storage and GL_INTENSITY that broadcasts it to all components).

When using shaders you don't actually need this distinction anymore, though, since you can do the swizzling yourself in the shader. So you just use a proper typed 1-component internal format, most probably GL_R8 (i.e. 8-bit normalized fixed point red) in your case, but depending on your exact data type requirements really. And for setting it you would (in that case) use GL_RED as the format of the data in your RAM storage. So something like:

glTexImage2D(GL_TEXTURE_2D, 0, GL_R8, <width>, <height>, 0, GL_RED, <type>, <data>);

And in the shader you can do with that red color component whatever you want. If you just need a single value per pixel, you use that. If you need a grey color (i.e. GL_LUMINANCE behaviour), you swizzle it to the RGB components, and if you need it as color and opacity, you swizzle it into all four components. Or whatever else you want to do.

float data = texture(...).r;
vec4 luminance = texture(...).rrra;
vec4 intensity = texture(...).rrrr;

Though, this requires your shader to be somewhat aware of what texture you're working with. But if you want to use a more generic shader working with a 4-component color from an arbitrary texture, this might be too much of a hassle. In this case what you can also do is have the texture itself control its own swizzling behaviour. This is an OpenGL 3.3+ feature and can be set with glTexParameter and for good old GL_LUMIMANCE behaviour this would be:

glTexParameter(..., GL_TEXTURE_SWIZZLE_R, GL_RED);
glTexParameter(..., GL_TEXTURE_SWIZZLE_G, GL_RED);
glTexParameter(..., GL_TEXTURE_SWIZZLE_B, GL_RED);
glTexParameter(..., GL_TEXTURE_SWIZZLE_A, GL_ONE);

For a broadcast into all components, i.e. GL_INTENSITY behaviour, you can use the shortcut

glTexParameter(..., GL_TEXTURE_SWIZZLE_RGBA, GL_RED);

So then you can retrieve a normal 4-component color from your texture in the shader and get the 1-component it's storing conveniently broadcast into the RGBA components of the result.


phong equation of illumination specular component I'm stuck on this question asking me to calculate the intensity of a pixel using Ray Tracing. It gives values such as $I_i$ but doesn't give any coordinates.

I know for the diffuse component, $I_ik_d(n,l)$ can be written as $I_ik_d(\cos\theta)$ but for the specular component $I_ik_s(h\cdot n)$ is there any other way to write it so that it doesn't require any vectors such as $h$ and $n$?

This is the question:

Assuming that the classic Phong model is desired here, the dot product that goes into the specular calculation should be R·L, rather than N·H (which was introduced by Blinn). That is, Phong calculates specular using the angle between the reflected eye vector and the light vector.

In the diagram you posted, these vectors are shown and the angle is given as 15°. So, R·L = cos(15°) = 0.96 according to the table below the diagram.


What is principal vanishing point in perspective projection? I am unable to visualize what is principal vanishing point w.r.t to x axis ,y axis z axis  do someone can post some simple  diagram to understand what is it? Also I have question in my text book
'        Now the question is how to calculate vanishing point and what is the line A' B ?Solution is also given which is (0,0,0) as vanishing point but I am unable to understand how do they get it , figure (k)shows it as (0,0,2) ,also what is d in the solution?

Basically whenever you project a set of parallel lines in 3D space on a 2D image plane, you'll see that these lines don't remain parallel in 2D space and tend to meet at a certain point which is the so called vanishing point. Picture a long railway, both the rail lines meet at the horizon.
Now one confusion arises that the question doesnt give you a set of parallel lines, But it doesn't matter as long as a single line segment is provided. All you need to do is just picture the line segment extending to infinity. So for example, picture the point $B$ at infinity and imagine drawing a line $BE$. The point at which this line intersects the image plane will be your vanishing point.
There is a simple pattern that applies to every case like this. Just shoot a ray in the direction of the vector $\vec{AB}$ from the point $E$. The point at which this ray intersects the image plane will be your vanishing point which as you can see is $(0,0,0)$.
Now imagine why this works. Picture the point $B$ extending up, and at each instance start drawing segments $BE$. You'll see that this line segment is moving towards the $Z$ axis.
So if you are projecting a point $B$ at infinity then the projecting line or segment $BE$ gonna become parallel to the line $AB$. However, it can't go past the $Z$ axis, as this will start going in the opposite direction.


Skia example code not writing an image I am just trying to make a simple Skia program. A very simple program, but as I run this code, the output image file never writes anything. I am running this code on VS 2015 and included the skia.lib and OpenGL32.lib static libraries.

Can someone tell me why this simple example does not work?

#include "skiaIncludes.h"

void draw(SkCanvas* canvas) {
SkPaint paint;
paint.setColor(SK_ColorWHITE);
canvas->drawPaint(paint);
}

int main ()
{
try {
    FILE * pFile;
    char buffer[100];

    pFile = fopen("myfile.png", "wb");

sk_sp<SkSurface> rasterSurface(
        SkSurface::MakeRasterN32Premul(300, 300));
    SkCanvas* rasterCanvas = rasterSurface->getCanvas();

    //rasterCanvas->save();
    rasterCanvas->translate(SkIntToScalar(128), SkIntToScalar(128));
    rasterCanvas->rotate(SkIntToScalar(45));
    SkRect rect = SkRect::MakeXYWH(-90.5f, -90.5f, 181.0f, 181.0f);
    SkPaint paint;
    paint.setColor(SK_ColorBLUE);
    rasterCanvas->drawRect(rect, paint);
    //rasterCanvas->restore();

    sk_sp<SkImage> img = rasterSurface->makeImageSnapshot();
    sk_sp<SkData> png(img->encode(SkEncodedImageFormat::kPNG, 100));
    rasterCanvas->drawImage(img, 0, 0);
    png->MakeFromFILE(pFile);

    //sk_sp<SkImage> bmap = SkImage::MakeFromBitmap(bitmap);
    //SkData * encBmap = bmap->encode(SkEncodedImageFormat::kJPEG, 100);
    //encBmap->MakeFromFILE(pFile);

    fclose(pFile);
}
catch (std::exception ex){
    printf(ex.what());
}

return 0;
}

Header file :

#pragma once

#include "core/SkData.h"
#include "core/SkImage.h"
#include "core/SkStream.h"
#include "core/SkSurface.h"
#include "core/SkCanvas.h"
#include "core/SkGraphics.h"
#include "core/SkImageEncoder.h"
#include "gpu/gl/GrGLInterface.h"
#include "gpu/gl/GrGLExtensions.h"
#include <fstream>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <exception>
#include <iostream>
//#include "gpu/gl/GrGLUtil.h"

This never writes anything to the output file because you haven't included any code to write to the output file. The only time you touch the file after opening it is in the line

png->MakeFromFILE(pFile);

which reads from the file into the SkData instance. It won't even do that in this case, because the file handle has to be open for reading only.

I'm no expert on Skia, but you probably want to use SkFILEWStream to write the file. Or just call png.bytes() and write the bytes to the file using the standard library.


OpenGL specular shading - Gradient banding issues I use Phong shading as my lighting model. Nevertheless, the specular shading produces gradients in the light cone:

Hopefully you can see the gradient starting from the lower left corner expanding along the cuboid.

Here is the shader code that produces this specular light:

vec3 viewDir = normalize(-FragPos);
vec3 lightDir = normalize(-light.direction);
vec3 reflectDir = reflect(-lightDir, normal);
float spec = pow(max(dot(viewDir, reflectDir), 0.0), material.shininess);

By the way, I'm doing my lighting calculations in view-space.

Is there any chance to make the lighting smoother?

This is a common problem with very subtle gradients. The problem is that the image is displayed with 8 bits per component and this is not quite enough precision to make a gradient that is smooth to the eye under these conditions.

However, the underlying calculations being done in the shader are at higher precision (e.g. float = 24 effective bits of mantissa), so you can apply dithering to improve the appearance of the image.

You can do this by adding a slight amount of random noise to the value of each pixel in the shader. The noise can come from a texture, filled with random grayscale values in [0, 1] and mapped 1:1 to screen pixels. In the shader you remap this to the range [−0.5/255, 0.5/255] and add it to the output color.

Here's a Shadertoy demonstrating this. The left side has no dithering; the right side uses the procedure just described. As you can see, the banding vanishes on the right.

Dithering effectively allows you to represent intermediate values between 8-bit integer levels. For instance, an area that the shader computes as 82.3 (out of 255) would normally be uniformly rounded down to 82, but with dithering it will come out as a random mix with 70% of the pixels set to 82, and 30% set to 83. The eye averages over this very slight, high-frequency noise and it becomes practically invisible.


Realistic chainmail reflections for videogames If I were to create a chainmail that has realistic reflections, where would I start learning about what mechanisms to use? Normal cloth seems too simple, and the reflections look bad (they have to be in the texture, or the cloth would have to be metalic). If I did it as a series of toroids, physics would have problems with colisions, or they would just overlap weirdly. Maybe there is a hybrid approach? Where can I read about this problem?

It seems that you're mixing physics/engine programming with graphics programming. I wouldn't say that dealing with collisions is related to rendering.
Also, I don't really understand your problem with toroids, but in case, I found one paper related to this term - "3D ChainMail: A Fast Algorithm for Deforming Volumetric Objects" by [Gibson1996]. I would say that if you're interested in creating clothing then you'll have to deal with deformations and animations and not only reflections. And, in case if you'd like to just work with reflections/rendering, then you don't need to deal with physics in any way. Static mesh would be enough, but I'm just speaking based on informations that you've provided until this point.

Normal cloth seems too simple, and the reflections look bad (they have to be in the texture, or the cloth would have to be metalic)

I wouldn't agree with that and I don't really understand what you've meant by that. In many cases, reflections are indeed in the texture. Or more specifically - cubemap, local environment map, irradiance volume and many other names, used for explaining GI voxel-based methods. https://www.kostasvardis.com/files/research/PhD_Thesis_Kostas_Vardis_book_online_lowres.pdf - [Vardis2016] PhD thesis is nice place to check for methods that you might be interested in. I wouldn't say that normal cloths are too simple. I was never really interested in that area but there are number of papers and overall, creating realistic clothing in not an easy task.

Normal cloth seems too simple, and the reflections look bad

I can only guess that by "bad", you've meant that there are little to none reflections. That's because most of clothing is rather rough and you've got little materials with big specular term in everyday life. That's why there is little reflection to it.
I don't know what is your current level of knowledge. Also, I don't know if you're going to creating real-time interactive application or if you're going to create offline renderer. In any case - if you have Turing graphics card or newer, I would suggest using Ray Tracing for most realisting reflections. You can start with cwyman tutorials - http://cwyman.org/code/dxrTutors/dxr_tutors.md.html also you should read microsoft's and nvidia's introduction to DXR. Also, here is a relatively easy path-tracer, created by Matt Petineo - https://github.com/TheRealMJP/DXRPathTracer
If you get through above tutorials, then you can start reading about denoising and state of the arts methods introduced in recent years - https://alain.xyz/blog/raytracing-denoising Raytracing is a subject, which exists since the begining of computer graphics, so there are a lot of papers related to that. But I think that MJP's example is good enough for start. Also, read related papers that he's stating in github and his code. They're mostly related to avoiding artifacts and fireflies.


Are vertices reprocessed per-poly for indexed meshes? I've been digging a bit into what actually happens at a hardware level on the GPU, and found NVidia's Life of a triangle which explains the pipeline pretty well, at least for green boxes. One thing I wasn't clear on is what happens with indexed meshes, when the same vertex is used for a bunch of different triangles. Since data is generally not persisted for any longer than necessary in a stream processor, I'm guessing the vertex is simply destroyed after being rasterized, then fetched and run through the vertex shader again whenever it appears in a new triangle. Can anyone confirm this? Also, what happens in line-strip or triangle-strip modes? Does the GPU persist the transformed vertex data somewhere until the 2 or 3 relevant primitives have been rasterized in these cases?

The key expression you may be looking for is "Post Transform Cache".

This is usually effective because of the natural vertex reuse, but reordering your mesh, e.g. with Hoppe's method (note: the page also lists some more recent work) or Forsyth's can improve things further.


Suggested image preprocessing before applying threshold I am trying to vectorize the contours (skewed rectangle) of the TV in the following original image.

My first step is applying threshold. Tried selection of threshold values based on what skimage has to offer, but non of them managed (for example) to separate correctly the little fawn to the left, or the TV stand from the TV border. For example this is the result using yen's method:

Can you suggest any preprocessing that may help before applying a threshold ?

In your case, I don't think a pre-processing is necessary. You may find the Hough transform useful in your case.

Since your image is big, it can be difficult to have a straight line due to the noise and blur.

This is what I got using a sobel contour detection with a smaller version of your image.

This is maybe a good start to use a line detection algorithm.

A second solution would be to use color segmentation. If you assume that your screen would always have black border you can work something in the LAb or HSV color space.


Why is cos() in the denominator in the reflectance equation and in the numerator in the shading equation? Background: I'm reading https://learnopengl.com/PBR to learn about PBR. In the theory slide (https://learnopengl.com/PBR/Theory), the author has the reflectant equation as

The theta (the angle between normal vector and the incoming light) is in the denominator. However, on the lighting slide (https://learnopengl.com/PBR/Lighting) he has the familiar code for the radiance of a light source, which makes the $\cos(\theta)$ on the numerator.

vec3  lightColor  = vec3(23.47, 21.31, 20.79);
vec3  wi          = normalize(lightPos - fragPos);
float cosTheta    = max(dot(N, Wi), 0.0);
float attenuation = calculateAttenuation(fragPos, lightPos);
float radiance    = lightColor * attenuation * cosTheta;

Can someone explain this fine point to me?

The first is a definition of radiance in terms of the flux - it's a the derivative of flux with respect to both area and direction. It's simply a relation between $\Phi$ and $L$. It simply says that if you integrate radiance over the area of some surface, and for each point you integrate over all possible directions, then you will get the flux arriving/leaving that surface.
The second part comes from the restriction of the rendering equation to a single point light source.
Remember that:
$$L(x,\omega_o) = L_e(x,\omega_o) + \int_{\Omega}{f(\omega_o,x,\omega_i)L_i(x,\omega_i)\cos\theta_i \,d\omega_i}$$
Where $x$ is a point on some surface, and $L(x,\omega_o)$ gives you the radiance incoming from $x$ in the direction $\omega_o$. This radiance is the sum of emitted radiance from $x$ in direction $\omega_o$: $L_e(x,\omega_o)$, and the scattered radiance by $x$ in direction $\omega_o$. The integral is necessary, since light may arrive from any direction at $x$, so you need to integrate over all directions. $f$ is the brdf, which really tells you how much of the light arriving from direction $\omega_i$ is reflected towards $\omega_o$ at $x$. $L_i(x,\omega_i)$ is the incoming radiance from direction $\omega_i$ at $x$, and the cosine is there to account for Lambert's law. What happens when you evaluate only direct lighting from a point light is that the integral is over a domain that is a single point, so you get only the integrand part, which you have in your code. Note that this is just an intuitive explanation, there are various issues with radiance not being defined for point lights.


How does the Painter's Algorithm handle transparency? I was wondering if anyone could explain how the Painter's algorithm would handle transparent objects? Can the Painter's algorithm handle transparency?

The common way to render transparent polygons in a rasterizer is by use of Alpha Blending, which basically combines the colour of the supposedly transparent pixel with the colour of the background at that pixel's position, this way rendering the pixel transparently onto what was already rendered before. This technique, however, requires your polygons (or at least the transparent ones) to be sorted from back to front, at least in general non-trivial cases, since you can only combine the transparent colour with what has already been rendered behind it.

But since the Painter's Algorithm already requires ploygons to be sorted from back to front (which is pretty much the entire essence of the algorithm), this just fits naturally into the rest of the algorithm. Thus, supporting transparency becomes rather straight forward by simply doing any variant of alpha blending of the pixel with the background instead of overwriting the background.


Looking for a compact file format for outputting 3D geometry with color support I am looking for a file format to output some 3D geometry. In my case, it only needs to handle very simple shapes - uniformly spaced and sized cubes at points in space. So far I've ended up just representing them as triangles or quads depending on what I can use.

I've tried Binary STL format which is decently compact, but has no support for colors.

Wavefront Object format (OBJ) and the material library (MTL) work well for color support, but there is no binary version as far as I know - making it very bulky.

Is there a file format that:

Is compact in size
Has some way to support colors
Can be opened by software like 3D Viewer, Paint 3D, or other common tools?

The PLY file format is a simple format similar to OBJ that allows arbitrary attributes (like colours) and has a binary mode.


Problems with Subdivision Surfaces It sounds like Subdivision Surfaces are better than NURBS but not as good as T-splines. I would like to know some of the disadvantages of Subdivision Surfaces, what they can't do or what they do inefficiently.

I am considering what type of system to use. Whether it be B-splines, NURBS, subdivision surfaces, or T-splines, or something else. Here I am mainly wondering how good subdivision surfaces are.

Both T-splines and subdivision surfaces are capable of handling an arbitrary topology input mesh, whereas NURBS can only handle meshes with regular topology.

Complex NURBS objects are therefore made out of multiple regular meshes. These meshes are trimmed and fit together to form a single complex object. However, along the trimming lines of the mesh discontinuities can occur. This is because the boundary curve of a trimmed NURBS patch is not in general a NURBS curve. This complicates the joining of several NURBS patches, potentially sacrificing water tightness.

Subdivision surfaces are able to handle meshes with arbitrary topology. This means your mesh can contain extraordinary faces (faces with more or less than four sides) and extraordinary vertices (vertices with a valency other than 4). The generated surfaces are in general curvature continuous in regular regions and tangent plane continuous at extraordinary vertices. The evaluation of a subdivision surface is an iterative process. Although regular regions can be converted to B-spline patches problems occur when rendering the irregular regions as an infinite number of B-spline patches will be generated. The OpenSubDiv library by Pixar is a good start for speedy subdivision as it calculates many things on the GPU.

Most subdivision surfaces are based on uniform subdivision and some work has been done to handle non-uniform and rational subdivision, but it is quite extensive and has some limitations.

T-splines can do everything NURBS can do as well. The most striking feature of T-splines is the ability to handle T-junctions, as well as arbitrary topology, without sacrificing in surface continuity. This allows the designer to insert more detail in some places without it effecting the mesh in other regions. In this way it is possible for a very organic design process. Take for instance this video. A downside of T-splines is that it is a patented technology.


Phong and the Rendering Equation: What's with the cosine? Ignoring emission and shadowing for simplicity, the rendering equation can be stripped down to:

$$L(x, \, \vec \omega) = \int_{\Omega}{f_r(x, \, \vec \omega^\prime, \, \vec \omega) \, (\vec \omega^\prime \cdot \vec n) \, d\vec \omega^\prime}$$

where $f_r$ is the BRDF and $(\vec \omega^\prime \cdot \vec n)$ is the cosine of the angle between light and surface normal.

For the Lambertian diffuse BRDF $\frac{\rho}{\pi}$ this is fine - we're multiplying it with the cosine, seeing Lambert's cosine law in action.

Now for the specular part of the Phong BRDF, you would commonly calculate a pixel's color like this:

$$I_{out} = I_{in} \, k_{specular} \, (\vec w \cdot \vec r)^\alpha$$

where $I_{in}$ is the incident light, $k_{specular}$ is the specular reflectivity of the surface, and $\alpha$ is the Phong exponent. Note the absence of $(\vec \omega^\prime \cdot \vec n)$ in this equation.

To use the rendering equation and end up with the same pixel value as the common Phong equation above, we'd need to write:

$$f^{Phong}_r(x, \, \vec \omega^\prime, \, \vec \omega) = \frac{1}{(\vec \omega^\prime \cdot \vec n)} \, k_{specular} \, (\vec w \cdot \vec r)^\alpha$$

Note the cosine in the denominator that cancels out the cosine in $L(x, \, \vec \omega)$.

In his derivation of the Phong normalization factor, Fabian Giesen states:

  [...] this is the derivation for the original Phong formulation, where the $R \cdot V$ term is not multiplied by $cos \theta$. If you write that version of the Phong model as a BRDF, you end up with a $cos \theta$ in the numerator to cancel out the $cos \theta$ factor in the reflection equation. This numerator is complete nonsense physically, so the modern formulation of the Phong model removes it.

I don't understand why he speaks about the numerator here, instead of the denominator. Is this a slip in the article, or a misunderstanding on my part?

What is the 'modern formulation' of Phong? What does it mean that cancelling out is 'complete nonsense physically'? With the normalization factor, $\frac{\alpha + X}{2 \pi}$ when should you use $X=1$ and when $X=2$?

Here is a path traced side-by-side comparison of a reflective sphere, Phong exponent 100, $X=2$ for the normalization. The left image has the cosine cancelled out, while the right image includes it, noticably darkening the borders.

The cosine term in the rendering equation is to account the amount of light reaching the surface, and leaving it out from the rendering equation is what he refers as "complete nonsense", which is right. The original Phong model didn't have the cosine term in the rendering equation, so if you want to model the original nonsensical Phong with proper rendering equation, you need to hack it by adding the cosine denominator to the Phong BRDF.


Instagram's fade effect What is the pixel-level explanation of the fade effect, that was first popularized by Instagram? I want to implement this effect programmatically—that is, I want to understand what this effect is doing to each pixel.

Also, if anyone could point me to resources from which I can learn more about how image filters are implemented, that would be great.

The effect in the photo is very close to a simple scale-bias per pixel. After a bit of tweaking, I found that applying the transformation: $x' = 0.77x + 38$ to the raw pixel values (as bytes) gives something quite close to your output:

In this case the effect of the scale-bias is to reduce the contrast (scale factor < 1) while keeping the overall brightness about the same.

Example Python code (with numpy and pillow) to do this transformation:

import numpy as np
from PIL import Image
a = np.array(Image.open("input.png"), dtype=np.float32)
b = a * 0.77 + 38
Image.fromarray(np.uint8(np.rint(b))).save("output.png")

For learning how to implement many basic image processing operations, I've found these Image Processing Operator Worksheets from the University of Edinburgh helpful.