Which technique is fast to generate real-time reflections? I want to know about some technique with which i can generate nice reflections on the surface. I know one conventional technique where we invert an object and render it and blend it with the floor but in this technique each object has to be rendered twice which is not good. Is there any other technique to achieve the same result?

Usually modern game engines use Screen Space Reflections which is some kind of screen space raymarching.

Basically, a simple way to achieve this is to shoot rays from the G-buffer. This means you use your depth buffer as a height field by raymarching any ray you cast. Then sample the normal at this position and take the final composed lighting from you previous frame and project it.

Although it has the usual drawbacks associated with any screen space method. This means you only have what is visible in front of you (screen space) to compute your reflections, so anything outside the screen would require tricks to have them get reflected on your current frame (e.g. you can combine this with environment mapping reflections and such).

You can find a few articles about that and presentations on the Internet.


What is the technique to generate an oblique 3d cone? Oblique meaning not a regular cone; its apex makes an angle to the base.

An alternate way of visualizing this: consider a sphere centered at the origin. The sphere traces a circle on the XY-plane. Now consider a point A anywhere on the sphere, and another point B on the circle. Now as B traces around the circle, length AB varies. I an trying to draw the solid (or surface) with AB as the generator.

The pics and video below would give you the final idea of the solid I want to generate:

https://youtu.be/Z8RAn6Dkz7w - animation

https://ggbm.at/vrEC7tDs - geogebra worksheet (3d)

The general equation for a upright cone with the tip at $Z=0$ is

$$ 0 = X^2 + Y^2 - Z^2$$

(Ignoring the points for which $Z>0$). If you consider slices for some constant $Z$ you get circles centered on the line $X=Y=0$. To make an oblique cone with circles along the line $X=aZ,Y=bZ$ we can just shift the coordinates accordingly and get the equation

$$ 0 = (X-aZ)^2 + (Y-bZ)^2 - Z^2$$

(again for $Z<0$). You can make it narrower or wider by scaling the $Z$ coordinate. To position the tip at some other point $(x,y,z)$ you can just shift the whole thing using the transform $(X',Y',Z') := (X-x,Y-y,Z-z)$.

With the same idea you can easily parametrize the (upright) cone by

$$(u,v) \mapsto (u\cos(v),u\sin(v),u)$$

or the oblique cone by

$$(u,v) \mapsto (u\cos(v)-au,u\sin(v)-bu,u)$$


Uniform not set, despite valid location I try to write a simple triangle mesh renderer with OpenGL.

void ShaderProgram::SetUniformMatrix4(std::string name, glm::mat4 matrix)
{
  GLint location = glGetUniformLocation(program_, name.c_str());

  if (location != -1)
  {
     glUniformMatrix4fv(location, 1, GL_FALSE, &matrix[0][0]);
  }
}

Is my uniform related code. I made sure that glUseProgram(program_) is called before this method is executed.
Also the glm::mat4 does indeed contain the correct values and program_ has a value of 3 so it seems valid.
The location comes out as a 0

Here how I call that method:

shader_->BindProgram();
shader_->SetUniformMatrix4("matModel", transformation);

texture_->BindTexture();
mesh_->Draw();
texture_->FreeTexture();

shader_->FreeProgram();

Now I have following simple Vertex Shader

#version 330 core

layout (location = 0) in vec3 vertexPosition_modelspace;
layout (location = 1) in vec3 vertexNormal_modelspace;
layout (location = 2) in vec2 textureCoords_modelspace;

uniform mat4 matModel;

layout(std140) uniform Camera{
  mat4 matView;
  mat4 matProjection;
};

out vec2 textureCoords;

void main(){
  textureCoords = textureCoords_modelspace;
  gl_Position =  matProjection * matView * matModel * vec4(vertexPosition_modelspace, 1.0);
}

I see nothing on the screen which is the main error.
When Debugging with Nvidia-NSight I find that all values except the matModel are correctly set. The matModel contains all zero which seems to be why the model isn't drawn.

Why is the matModel always zero, even when I try to set the uniform?

It seems that nothing was wrong with the code I posted.
I deleted my complete build and rebuild it, and now it seems to work.
I don't see anything on screen but i think that error is unrelated.

I can see the matrix in the nsight debugger.


How to convert image results into data? I have made an 3d solar exposure analysis for a 3d city model that determines how long a surface receive direct sunlight throughout the year. Results are coloured surfaces and legend of coloures that explain how long the colour means. You can see the results in the attaching file. How can I report the results. How can I convert visualization into data? Because I want to query the results of analysis

I think getting the real data with high precision is not easy without some API.
The first idea would be just to manually use some color based selection tool that you can tell to select all areas in the screenshots according to the lookup table.
But that would not result in some 3D data, but only segmented images where you can annotate the values.

Another method would be to make a UV-map of the mesh, and then by making screenshots from various regions try to extract the color-coded texture.
Then you could read the interpolated color-values of the texture and map them back to the actual values.

The last but maybe strangest idea would be to make a huge number of screenshots and then use some multiview object reconstruction algorithm (I don't know if there is software or if you have to write it yourself.). There are algorithms that not only reconstruct the geometry, but also the texture.
This would then give you a model with the color-texture that again you could map back to the values.

But ideally there would be some export function in the software you used. Maybe this is a question you should ask the developer of the software. Maybe they have this function somehow but it is a bit hidden.


XMMatrixPerspectiveFovLH is broken? I've been trying to build a simple DirectX renderer from the ground up (was an OpenGL guy before but i figure it's good to know different APIs). I've been following the guide at rastertek.com to learn how to get the basics up and running.
I got as far as this lesson, following the guide verbatim except for variable names, and ran into a really weird bug. The renderer class contains 3 XMMatrix instances, world, perspective and orthographic, but trying to initialize them would crash my application - it ran fine once i commented out these lines :
m_projMat = XMMatrixPerspectiveFovLH(fov, aspect, nearplane, farplane);
m_orthoMat = XMMatrixOrthographicLH(width, height, nearplane, farplane);
m_worldMat = XMMatrixIdentity();

At first i thought it was the dreaded SIMD alignment issue, but I'm compiling 64-bit so that should be taken care of. After a bit more tinkering i found out the ortho and world matrices were fine, it was just the projection matrix that was causing problems. Assigning to this matrix with the other XMMatrix... functions also works ok.
So:
m_projMat = XMMatrixPerspectiveFovLH(fov, aspect, nearplane, farplane);

fails, but:
m_projMat = XMMatrixPerspectiveLH(fov*aspect*nearplane, fov*nearplane, nearplane, farplane);

works! The right-handed version also fails. Also, the failure only happens in release mode, not in debug. I'm using Visual Studio 2015 community and the SDKs that installed with it.
Has anyone else encountered this bug, or have any idea what's going on here?

Problem

XMMATRIX and XMVECTOR (which use __m128 under the hood) require 16 byte alignment. The C++ compiler can automatically ensure this for XMMATRIX and XMVECTOR data allocated on the stack, but can't ensure this for XMMATRIX and XMVECTOR data allocated on the heap.

In your specific situation (referring to the RasterTek tutorial), three member variables of your D3DClass class are declared to be of type XMMATRIX and you construct one instance of this D3DClass class on the heap by using the default operator new (due to the absence of a custom one). Using the default operator new does not guarantee this 16 byte alignment. Even if your code would have run fine using the default operator new, you could still face this issue after adding some extra member variables in the future.

Solution 1

The most easy fix in this specific case, is to allocate your D3DClass class instances on the stack instead of on the heap. This, however, does not prevent you or future users of your code to allocate an instance on the stack instead.

Solution2

The programmers of the DirectXMath library advise the general programmer audience to use XMMATRIX and XMVECTOR for calculations only, and to not use them as containers for storing persistent data (i.e. class member variables). For the latter, they advise to use XMFLOAT4, XMFLOAT4x4, etc. The latter structs do not impose the 16 byte alignment restriction and their instances can simply be loaded in SIMD registers by using XMLoadFloat4 for XMVECTOR, XMLoadFloat4x4 for XMMATRIX, etc.

  However, often it is easier and more compact to avoid using XMVECTOR
  or XMMATRIX directly in a class or structure. Instead, make use of the
  XMFLOAT3, XMFLOAT4, XMFLOAT4X3, XMFLOAT4X4, and so on, as members of
  your structure. Further, you can use the Vector Loading and Vector
  Storage functions to move the data efficiently into XMVECTOR or
  XMMATRIX local variables, perform computations, and store the results.
  There are also streaming functions (XMVector3TransformStream,
  XMVector4TransformStream, and so on) that efficiently operate directly
  on arrays of these data types.

Solution 3

Ensure the 16 byte alignment by overwriting operator new/operator delete, operator new[]/operator delete[]. We declare a template class which does allow heap allocation with arbitrary alignment restrictions:

#include <malloc.h>

inline void *AllocAligned(size_t size, size_t alignment = 16) noexcept {
    return _aligned_malloc(size, alignment);
}

inline void FreeAligned(void *ptr) noexcept {
    if (!ptr) {
        return;
    }

    _aligned_free(ptr);
}

template< typename DataT >
struct AlignedData {

public:

    static void *operator new(size_t size) {
        const size_t alignment = __alignof(DataT);

        // __declspec(align) on DataT is required
        static_assert(alignment > 8,
            "AlignedData is only useful for types with > 8 byte alignment.");

        void * const ptr = AllocAligned(size, alignment);
        if (!ptr) {
            throw std::bad_alloc();
        }

        return ptr;
    }

    static void operator delete(void *ptr) noexcept {
        FreeAligned(ptr);
    }

    static void *operator new[](size_t size) {
        return operator new(size);
    }

    static void operator delete[](void *ptr) noexcept {
        operator delete(ptr);
    }
};

This code snippet is based on the DirectXTK and is also included in my own codebase (including documentation).

This data structure can now be used as follows:

__declspec(align(16)) struct Transform final : public AlignedData< Transform > {
    XMMATRIX m_transform;
}

A Transform structure can now be allocated on the heap with 16 byte alignment, since our custom operator new will now be invoked:

Transform *transform = new Transform();

If you use instances of the Transform struct as member variables in other structs of classes, the alignment restrictions remain of course. So instead of having only alignment restrictions for XMVECTOR and XMMATRIX, you will have now alignment restrictions for XMVECTOR, XMMATRIX and Transform.

Smart Pointer Pitfalls

Be careful if you use std::shared_ptr and especially std::make_shared. The latter allocates the data block together with the control block without using customly defined allocators such as our custom operator new.

This means that this method will not work:

template< typename T, typename... ConstructorArgsT >
inline std::shared_ptr< T > MakeShared(ConstructorArgsT&&... args) {
    return std::make_shared< T >(std::forward< ConstructorArgsT >(args)...);
}

but you should rather use an explicit invocation of operator new (which is also the case for std::make_unique):

template< typename T, typename... ConstructorArgsT >
inline std::shared_ptr< T > MakeAllocatedShared(ConstructorArgsT&&... args) {
    return std::shared_ptr< T >(new T(std::forward< ConstructorArgsT >(args)...));
}

std::unique_ptr and especially std::make_unique will work as intended since the latter explicitly calls operator new (due to the absence of a control block).

RasterTek

RasterTek's second series of D3D11 tutorials is a subset of RasterTek's first series of D3D11 tutorials with the minor difference that the latter uses the obsolete D3DXMath. The transition is from D3DXMath to DirectXMath is actually pretty straightforward. Therefore, it can be interesting to take a look at the first series as well (which I ported to Visual Studio 2017, run fine in both Debug and Release configurations, and can be founded as well in this repository).

If you want to stick to DirectXMath, you can also take a look at the D3D11 tutorials of Microsoft itself (explanation + code). (I also created a repository with some code refactors of these tutorials).


How to create specular surface in Mitsuba Renderer I am a newbie in computer graphics and I am trying to develop a special case of path tracing really slowly! But first: I came up studying the Mitsuba renderer and one of its uses in the so-called GDPT (Gradient Domain Path Tracer). I am trying to build a Cornell Box in this framework and my silly problem arises in realizing which kind of material is a "mirror". In simpler words: I want a mirroring sphere in my scene but I don't manage to create it. According to the documentation that you can find here (http://www.mitsuba-renderer.org/releases/current/documentation.pdf) we can have all the different surfaces explained in section 8.2, which are:

Smooth Diffuse material (diffuse)
Rough diffuse material (roughdiffuse)
Smooth dielectric material (dielectric)
Thin dielectric material (thindielectric)
Rough dielectric material  (roughdielectric)
Smooth conductor (conductor)
Rough conductor material (roughconductor)
Smooth plastic material (plastic)
Rough   plastic material (roughplastic)
Smooth dielectric coating (coating)
Rough dielectric coating (roughcoating)

Can you please help me understand what set of parameters will make a mirroring surface material?

Usually for each kind of material there is a reference XML code (which is what I am using to build the scene). Therefore, the main question remains understanding how to build a mirror, but please take those examples in considerations because the parameters in those are the only things I can tune. Thanks in advance!

You want to use the "conductor" plugin to create a smooth conductor / metal type BRDF. See section 8.2.6 in the documentation.

As far as Mitsuba is concerned, a mirror is a special instance of a metal, where the material is specified as "none" (which corresponds to using complex IOR of $ \eta = 0 $ and $ k = 1 $).

Here is an example in xml:

<shape type="...">
    <bsdf type="conductor">
        <string name="material" value="none"/>
    </bsdf>
<shape>


splitting hue & saturating and leaving value behind In an HSV model, the Hue is the color model. So when we split the image or remove H, should we get an image without rgb colors?
Similarly, saturation is the gray shade. I read that images become more grey and faded in this case. So what should happen when we remove both, hue and saturation, leaving only the vue component behind?
Any example images would be nice.

Removing hue and saturation ("desaturating") leads to a grayscale image with the same luminance as the original colors, for instance:

(source: my own photo)
It is not possible to remove hue alone without removing saturation, as any saturated color must be saturated in some direction (hue). Nor is it possible to "split" the image into HSV "components" in quite the way you can split into RGB components. Hue works like an angle of rotation around the "color wheel", while saturation is the radius from the center:

(image from here)


Can you do GPGPU on integrated graphics like Intel HD 630 on Core i3? The Core i3 says it has "integrated graphics" with Intel 630.

Does this mean it is like a mini GPU?

In which case can use things like CUDA to speed up C sharp programs?

Is there much software that takes advantage of integrated graphics? Like does photoshop access the integrated graphics?

CUDA is only available on nvidia cards.

The open technology is OpenCL: https://es.wikipedia.org/wiki/OpenCL

Another alternative is to use Compute Shaders. DirectX, OpenGL, and Vulkan all support compute shaders


After a deformation operation on polygons, how can I check for and fix inverted polys? I'm doing a quick and dirty automated deformation routine on a polygon body.  If it was a tree, and my axis was in the center of the tree, I would like to bend the body by bending the axis.  I would use a simple influencing algorithm to determine vertex point xyz displacement.

On the concave side of the bend, how can I check for polygons that have overlapped or passed through each other, and how can I fix such polygons?

Thank you!

If you have an oriented triangle mesh (a non oriented surface would be the Moebius Strip for instance), you can check the triangle inversion by looking at the signed area of triangles. The signed area is positive when the vertices are in CCW order and negative if they are in CW order. If your triangle vertices are all sorted in CCW then all triangles with negative area are inverted. Fixing that depends on the deformation technique, I don't know your case.

Checking self-intersection is more involved. You should actually do Collision Detection of the mesh with itself. Collision detection is fast but need to implement proper data structures such as AABB Tree for narrow the amount of triangle-triangle intersection tests. Collision Detection is usually performed using very low-poly version of the meshes. In some computer games, the bounding volume hierarchy (spheres, AABBs, etc) that approximate the mesh is the only thing considered for collision, no the mesh itself.

Once a collision is detected, there are several ways to response, applying a  bouncing force for instance, is common.


Physically based rendering and ray tracing I've got a lot of confusion and I need to clarify some terminology and put together the knowledge.
If I say that an engine is a ray tracer (so it uses the ray tracing algorithm to render the scene), is it automatically a physically based engine? I mean, ray tracing is by its definition physically based or there could be some ray tracers that are not physically based? (and, symmetrically, are there some non-ray tracers that are physically based?)
Also, is it true that "physically based" means also that "resolves the light transport equation" ?

"Physically based" is not a very well defined term, so it's difficult to answer this question exactly.

In general, "physically based" refers to the fact that the algorithm in question is derived from physically based principles. It's not physically correct (because we can't afford that) and some approximations usually have to be made, but it's well known what those approximations are and ideally there is some intuition as to what kind of error it introduces.

This is in contrast to ad hoc models, which are usually made by an artist who observes an effect in real life and tries to write a shader or similar that somehow mimics the look of it. Usually ad hoc models are simpler and cheaper and tend to be the first solutions to turn up for a given problem, but they don't offer any particular insight into what's actually happening. It's also practically impossible to say how accurately such a model is able to reproduce the effect that it attempts to simulate.

In the context of rendering, a "physically based renderer" would therefore be simply something that renders an image using physically based principles, which is a very vague classification. A ray tracer is not inherently physically based, and most early ray tracers in fact used ad hoc models for lighting and similar. From my personal experience, "physically based rendering" used to usually refer to solving the rendering equation. However, it seems that in recent years, many game engines have claimed this term as well to mean "we do energy conservation" or "we no longer use phong".

So really, there's no hard classification of what "physically based rendering" means, and using ray tracing by itself does not make a renderer physically based. In offline rendering, this term is still mostly used to refer to renderers that solve the rendering equation, whereas in real-time rendering, it more likely refers to the use of microfacet models or similar. But it's possible that the meaning of this term will change over the years.


Export Model Textures Per Face (Note that as far as attempts go, I can't find anything that helps me)

I have a 3D polygon, and for simplicity's sake it's going to be the one below:

It is textured using the UV map below:

Side Note: I just found the texture on Google. It is not being used in any manner other than solely to demonstrate my question.

I plan on 3D printing the polygon. Since I suck at painting, I want to print the textures for each face so I can glue/decal them onto the correct faces.

My question is: How can I go from a UV texture map to individual polygon faces (and 1 texture per face) that is correctly scaled to fit the face. I want something like below:

Is there any way to do this? I don't care about width * height as I can manage that on my own, but I need the textures to be distorted just like how they are on the corresponding face of the polygon.

UV mapping assigns fixed positions in the texture to each vertex. You'll have to distort the image so that it maps to a constant area in "model size" so to speak. I found the following other question that will probably give you a solution: https://blender.stackexchange.com/questions/22872/unwrap-uv-map-proportionally-to-object-size

It won't give disjointed parts like in your image, but I think it will still provide what you're looking for.


What is OpenGL's paradigm? OpenGL is not object-oriented, but more like "switches you turn on"?

Like

glEnable(...);
list_id = glGenLists(1);
glNewList(list_id, GL_COMPILE);
glBegin(GL_TRIANGLES);
...

It seems like one's turning on switches on some machine which is contained somewhere, but it isn't seen as an object in object-oriented sense?

Is there a name for this kind of programming or library paradigm?

I'm using it from C++, but perhaps the way it's used is "C-style"?

It's generally referred to as a "state machine". It has, as you noticed, a bunch of switches and parameters. All of them are globally accessible to any code that has access to the current context. Your program sets them and then issues draw calls. Anything you don't change, will remain in that state until you do change it.

It is also a Hardware Abstraction. The same OpenGL routines should have roughly the same effect regardless of the hardware it's running on. If you upload 3 vertices for a triangle and then draw them, that should produce similar results on an ATI, Nvidia, or Intel GPU, for example. You don't need machine-specific instructions for each one.


What is a tbuffer and what is their use case? In HLSL, we have cbuffers and tbuffers.
MSDN's description for tbuffers is the following:

A texture buffer is a specialized buffer resource that is accessed
like a texture. Texture access (as compared with buffer access) can
have better performance for arbitrarily indexed data. You can bind up
to 128 texture buffers per pipeline stage.

what is the difference to cbuffers? in cbuffers i can access the data arbitrarily, too.
When should i use a tbuffer and when should i use a cbuffer?

Keep in mind, that I'm still learning and I have no professional capacity in terms of graphics programming. So you might want to continue research with provided resources.

t*#*: A register for a texture buffer (tbuffer).[1]

To start with - texture buffer is what you can see in hlsl code as: Texture2D texName : register(t0) or as Texture2D texName : register(t0, space0). If I remember correctly, spaces were added in DirectX 12, maybe they're available in DX 11.3, but that's different discussion. Back to my point - you're using texture buffer every time you're binding texture to your shader and sending texture data to GPU. Therefore, using PSSetShaderResources [2] will be most trivial usage of texture buffer, espacially in DX11 code. You can use constant buffer to store pixel data and then send it to GPU, but why should you bother, when Texture2D or TextureCube and other data structures are already in hlsl?

cbuffer (DX11/DX12) forces you to pack data in 32-bit groups [3]. You might have experienced it, if not, I encourage you to experiment a little. In the past I didn't follow this rules, having that packed like this, which causes problem with incorrect data in your shader:
cbuffer buf : register(b0)
{
    float3 a;
    // Here is where problem occurs
    float2 b;
    float3 c;
};

Correct packing rule would be:
cbuffer buf : register(b0)
{
    float3 a;
    float padding_a;
    float2 b;
    float2 padding_b;
    float3 c;
    float padding_c;
};

Of course, I am using worst possible example here, because correct data is 8 bytes and you're sending additional 4 bytes (whole hlsl vector) as padding. But you get an idea. It might be easier to sort data with bigger buffers, especially if you're using uber buffer in your hobby project.

Additional problem in DX12 is that cbuffer has to be 256-byte aligned [4], unless you're using root constants, which allows you to send not aligned data, but they have other limitation. I was only talking about them with other developer, so I don't really have sources except [5]. There was something about size of buffer and that root constants are working magic in AMD drivers, but it isn't something that I really need right now, as I am focusing on learning, not extreme optimization of code.

Shader Model 5 allows you to use new data structures [6]. I wasn't working with compute shaders, but for DXR hlsl code, I am using RTTexture2D as UAV. However, I am passing indices, vertices and RaytracingAccelerationStructure as texture buffers. I didn't see anyone to use those three as cbuffer. I guess it's possible, at least for indices and vertices, but I don't see a reason to do so:
RaytracingAccelerationStructure SceneBVH : register(t0);
ByteAddressBuffer indices : register(t1);
StructuredBuffer<Vertex> vertices : register(t2);

Indices are simply 16 or 32 bit index for accessing vertex data. I don't see a reason to hassle with creating cbuffer for single array and having to deal with padding and providing correct size of buffer. However, I am really interested if there is a reason to do it sometimes.
Same with vertices - in rasterization, you're using structure which don't have to follow packing rules e.g.:
struct VertexInputType
{
    float3 position : POSITION;
    float3 normal : NORMAL;
    float3 tangent : TANGENT;
    float3 binormal : BINORMAL;
    float2 uv : TEXCOORD0;
};

So for raytracing, I am just using struct with same data and I'm accesing it through structured buffer, regsitered as texture buffer:
struct Vertex
{
    float3 position;
    float3 normal;
    float3 tangent;
    float3 binormal;
    float2 uv;
};
StructuredBuffer<Vertex> vertices : register(t2);

Regarding RaytracingAccelerationStructure - it's built-in hlsl type, no need to hassle with creating cbuffer.

I'm sorry for speaking from DX12, DXR perspective, but that's something that I'm currently working on, so it's a simplest way to provide examples from that area.
[1] https://docs.microsoft.com/en-us/windows/win32/direct3dgetstarted/work-with-shaders-and-shader-resources#read-from-the-constant-buffers
[2] https://docs.microsoft.com/en-us/windows/win32/api/d3d11/nf-d3d11-id3d11devicecontext-pssetshaderresources
[3] https://docs.microsoft.com/en-us/windows/win32/direct3dhlsl/dx-graphics-hlsl-packing-rules
[4] https://social.msdn.microsoft.com/Forums/sqlserver/en-US/78dec419-6827-40fc-a048-01c3cccb92ef/directx-10-11-and-12-constant-buffer-alignment?forum=vsga
[5] https://docs.microsoft.com/en-us/windows/win32/direct3d12/using-constants-directly-in-the-root-signature
[6] https://docs.microsoft.com/en-us/windows/win32/direct3d11/direct3d-11-advanced-stages-cs-resources


What is the basic idea of denoising? When doing random sampling, you can get noisy results if you don't use enough samples like in the shadows below.

I know that if you use high frequency noise (aka blue noise), the noise pattern can be high frequency content only, which can then be low pass filtered (aka blur) to remove the high frequencies and leave just the lower frequency content which would be less noisy.

Is denoising ALWAYS about doing a low pass filter / blur? Or are there other ideas and techniques for removing noise?

Is denoising ALWAYS about doing a low pass filter / blur?

No, but this is the most obvious technique. A good denoiser isn't just a filter that runs on the image, but actually performs the reconstruction; i.e. it's a function from random samples to an image, not a function from an image to an image.

  Or are there other ideas and techniques for removing noise?

Yes. For years, the main family of denoise techniques worked in two parts. First, a line detection algorithm would detect the true edges in the image. Then, the output in this would set weights in a filter kernel to perform some kind of guided blur, so that the blur would be strongest (i.e. remove more frequencies) where there are no edges, and weakest (i.e. allow high frequencies) where there were edges. This would remove more noise from gradients while preserving edges.

Nowadays, deep learning has provided a new family of techniques. Using the same technique that's producing the noise (i.e. path tracing if you want to denoise path-tracer renders), produce lots of image pairs where one has high noise (few samples) and one has low noise (high sample rate). Feed these pairs into a convolutional network, to train it to produce the low-noise images from the high-noise images. Then feed your high-noise image into the trained network, and enjoy the low-noise output. Variations on this technique include having the network produce a filter kernel for each image patch, instead of learning the output image directly; or classifying an image patch according to its structure (what edges are present), so similar patches can be handled similarly. Some networks can even take the raw samples as input and directly produce a denoised image; others can guide your adaptive sampling algorithm.


How to handle a parametric equation in raytracing? After taking a look at the Mobius strip, I noticed its equation is really simple and tried to add it into my Raytracer.

I tried a "naive" way by simply generating N triangles attached to each other to obtain the desired shape. While this approach works, the result it not really pretty:

(By the way I probably have an issue with my normals but I don't know where it comes from.)

I tried it with PovRay and the result was astonishing. Perfectly smooth strip made in a far far FAR smaller time than mine. I'm pretty sure Povray is well optimized but I also think it won't generate triangles like I did.

In case that might help, here is the actual code used (C++) :

float step = .1f;
float halW = 0.5f;

_facets.clear();

auto lambda = [this] (float v, float t) {
  Vec_t p;

  float cdv = Tools::Cos(2 * v);
  float sdv = Tools::Sin(2 * v);
  float ctv = Tools::Cos(v);
  float stv = Tools::Sin(v);

  float c = 2 + t * ctv;

  p.x = c * cdv;
  p.z = c * sdv;
  p.y = t * stv;

  return p;
};

for (float v = 0.f; v < Globals::PI; v += step)
{
  if (v > Globals::PI)
    v = Globals::PI;

  for (float t = -halW; t < halW; t += step)
  {
    if (t > halW)
      t = halW;

    Vec3 p1 = lambda(v, t);
    Vec3 p2 = lambda(v + step, t);
    Vec3 p3 = lambda(v, t + step);
    Vec3 p4 = lambda(v + step, t + step);
    _facets.emplace_back(p1, p2, p3);
    _facets.emplace_back(p3, p2, p4);
  }
}

TL;DR

How can I handle parametric surfaces like this one in raytracing?

Edit
After letting the above algorithm run for about 20 hours, I got a way prettier result (with 3 torsions instead of 1)

Many years ago I worked on a ray tracer that handled parametric surfaces, so this is unlikely to be state of the art, but, IIRC, I used a combination of interval arithmetic with (binary?) subdivision and Newton-Rhapson.

The interval arithmetic + subdivision constructed (conservative) bounding boxes which could be used for intersection rejection. I think I may also have used interval arithmetic on the 1st derivative to help determine when it was safe to launch into Newton-Rhapson. Typically the Newton steps converged very rapidly.

You can also pre-dice your parametric surfaces and put them into an acceleration structure (Voxel grid/BVH) to speed up the process.

This work was based on/inspired by Daniel Toth's 1985 paper "On Ray Tracing Parametric Surfaces"


How to translate mouse movement to a rotation gizmo? I need to implement translation, scaling and rotation gizmos and I'm having trouble with the rotation gizmo. Basically I don't know how to translate the mouse movement to determine in which direction to rotate the object.

By rotation gizmo (widget) I mean something like the below:

For the translation and scaling gizmos, which are represented as 3 axes, I do the following: I make a vector of the difference between the previous and current mouse position in screen space. Then multiply this vector by the inverse of the view and projection matrices, which gives me a vector of the mouse movement in world space. Then I take the dot product of the vector in world space with the  selected axis. This allows me determine if the mouse movement is in line with the axis or away from the axis.

But I can't do the same thing for the rotation gizmo. The rotation gizmo is represented as 3 circles, each representing a rotation around an axis. Once a circle is clicked, I need to find some way to translate the mouse movement to determine which direction to rotate the object around the selected axis. I don't know how to do this. Please help.

It is important to note that I am using color picking and not raycasting to determine where the gizmo was clicked. This means that I don't have the point at which the ray hits the gizmo in world space to start from. I mention this because I've looked at examples that use raycasting like libgizmo and I don't think I can do the same thing.

Thanks in advance for any help.

Map the mouse position to the circle by drawing an imaginary line between the mouse and the center of the gizmo on the plane of the current circle. Do this for the start and current positions.

Then you can easily get the quaternion rotation around the center from the start to the end by doing normalize(quat(1+dot(v_1, v_2), cross(v_1, v_2))).

This works because the dot product is the cos of the angle between the vectors and the cross is the vector perpendicular to the 2 vectors (aka the rotation angle) and has length of the sin of the angle between the vectors. This means that you can create the double quaternion out of them. Then you can nlerp that double quaternion with the unit quaternion to get the rotation you need.

However with that information you can do an arcball by projecting the mouse onto the ball that is the gizmo and using the start and end vectors that way.


Overlaying graphics on particle simulation How can I overlay graphics on particle simulations? For instance if I am representing a liquid or gas with tons of little points how can I make those points look like a liquid or gas? Take Nvidia Flex for instance, they simulate liquids and gases with tons of small points but can render them as realistic liquids and gases instead of small balls.

For rendering of gases, I think the usual approach is to simply render each particle as a tiny disc. Gases don't really coalesce into surfaces like liquids do, so this should produce acceptable results. You could perhaps apply a light blur over the gas layer afterwards to soften it and hide the fact that it is made of discrete elements.

Liquids, on the other hand, tend to coalesce together to form droplets and smooth surfaces, so you need to derive a surface from the particles somehow. One way to do this is to use Metaballs, which also display this behavior and can be tweaked to suit different liquids and particle densities. By interpreting each particle as a metaball, you will have an implicit equation representing your liquid surface. To render this implicit surface you can then use an algorithm like Marching Tetrahedra to convert it to triangles, or utilize Ray Marching to directly render it. (Ray marching can be easily done in realtime in a fragment shader these days.) You can also use this approach for gases if you want a somewhat softer look.


Calculate vanishing point In perspective projection, group of parallel lines have the same vanishing point. I am interesting about the reverse calculation: Getting the group of parallel lines equations that their vanishing point specific point.
Say I know that the camera is perspective camera at $(0,0,0)$ and it's direction $(0,0,1)$, the view plane is $z = 1$ and I am interesting about the lines in plane $y= y_0$ that their vanishing point is $P = (p_x,p_y,p_z)$.
I have tried to calculate the projection point of some point $(x,y_0,z)$ and get the equations:
(i) $p_x = x(\frac1z)$
(ii) $p_y = y_0(\frac1z)$
(iii) $p_z = 1$
But it seems wrong because if the vanishing point is like $(x_0,0,*)$ then form (ii) we will get $z\rightarrow \infty$ but then (i) is wrong because $x(\frac1z)\rightarrow0$ but it need to be equals to $x_0$.
So how can I get the group of parallel lines have the same vanishing point in these conditions?

Here's a hint to get you started: Parallel lines include the line through the camera.

So really all you need is the direction from the camera to the vanishing point on the view plane. Then create lines parallel to that line.


Rendering 3-D point clouds obtained from depth maps as a 2.5D video Suppose I have code to generate a bunch of frames, each of which is a depth map(I have access to the matrices which represent depth) which in turn is used to generate a point cloud. One frame would look like this:

In short I have a point cloud corresponding to each frame. How would I generate a video where each point cloud in a frame is meshed together?

Are there frameworks which handle this kind of data? I am assuming OpenGL should help but I am not sure if it can do this.

Depending on your application, you could use image-based solutions that are much cheaper than meshing a uniformly sampled set of points. For example, you could use the depth map as a height map.

There are many techniques that allow you to render it in an efficient way, the more realistic the more difficult to code as a shader (but not too much). From easiest to harder:

height maps
normal maps
displacement mapping
parallax mapping
geometry shaders to recreate the actual mesh from a set of primitives and the depth map
and so on.


Probability density function while using spherical coordinates I'm following this tutorial where at somepoint the derived PDF for spherical coordinates for a Lambertian surface is

\begin{array}{l}
p(\theta, \phi) = \dfrac{\sin \theta}{2 \pi}.
\end{array}

But as soon as they compute a sample, the result is instead divided by $ \dfrac{1}{2\pi} $, which as they say is the "pdf of the integral"

Why isn't it divided by $ \frac{\sin \theta}{2 \pi} $ instead?

If we were using differential steradians over the unit hemisphere, the only possible probability density function integrating to 1 is infact $ \frac{1}{2\pi} $

But if we separate the integral over the hemisphere traced by spherical coordinates

$$
\int_{\phi = 0}^{2\pi} \int_{\theta = 0}^{\frac{\pi}{2}}  \,  \frac{\sin{\theta}}{2\pi}d\theta d\phi= 1.
$$

The PDF now becomes $ \frac{\sin \theta}{2 \pi} $ yet they still divide by $ \frac{1}{2\pi} $

EDIT: after carefully reviewing the concept of PDFs and integration over the hemisphere I'm starting to think the article I've linked is making a substantial error, mixing the idea of importance sampling with the pdf of choosing a direction of reflectance from a lambertian surface

Radiance is defined as
$$ L_{(x,\omega)} = \frac{\mathrm{d}^2\Phi}{\mathrm{d}\omega\ cos\theta\ \mathrm{d}x}  $$
Since it's defined over differential solid angles, we can interpret the result of one sample as if it was the flux density "over a unit steradian"

If we use monte carlo estimation and find "the average flux density over a single steradian" and multiply the result by $2\pi$, we get irradiance:

$$ (2\pi) \frac{1}{n}\sum^n L_{(x,\omega)} $$.
But in this particular case, $2\pi$ has nothing to do with pdfs! since it's the integral domain used for the monte carlo estimation!

Instead, the real pdf is computed for $\theta$ and $\phi$ because that is the probability density function of choosing one direction over the other, according to the particular properties of a lambertian surface. A mirror-like surface has a different probability of choosing one direction over the other, but this has nothing to do us with dividing the sample with $\frac{1}{2\pi}$. It would be different if we were using importance sampling, but in this case it seems like we're not

Is my reasoning correct? If not, what am I missing?

Normally $sin(\theta)$ isn't written as part of the pdf, because it's an artifact from using double integral for spherical integration. The double integral does the integration over a rectangular domain (in you case of size $2\pi$ x $\pi/2$), while you want integration over a sphere/hemisphere, so $sin(\theta)$ is added there as a weight function to transform rectangular to spherical integration.

A common notation is to use integration over solid angle with single integral and differential solid angle $d\omega$ denoting 2D integration over sphere, and you won't see the $sin(\theta)$ term in there.

To clarify, the notation with single integral has constant infinitesimally small solid angle $d\omega$ over the sphere. However, for the double integral over $\theta$ and $\phi$ the solid angle isn't constant (think of sphere tessellation) but a function of $sin(\theta)$. In case of Monte Carlo integration and even distribution of samples over the hemisphere, each sample represents solid angle of $2\pi/N$ steradians and there's no $sin(\theta)$ solid angle weighting required.


How different is the SPIR-V accepted by OpenGL (ARB_gl_spirv) from SPIR-V accepted by Vulkan? The format of SPIR-V bytecode is probably one of the least documented parts of both OpenGL and Vulkan that I am aware of. I have compiled a very basic fragment shaders into both the OpenGL and Vulkan SPIR-V formats. Looking at the files in a hex editor, I see no obvious differences, but when I hash the files, they give a different result. What are the differences between the OpenGL and Vulkan SPIR-V formats?

Vulkan and OpenGL use different models for their resource bindings. Vulkan uses descriptor sets + binding, while OpenGL uses binding indices. As such, the SPIR-V they consume use different decorations for their resources.

There are also a number of differences in terms of built-ins that the two APIs provide. For example, Vulkan accepts the InstanceIndex built-in, while OpenGL does not. OpenGL uses InstanceId, which has a different meaning from InstanceIndex.

The ARB_gl_spirv extension has a pretty exhaustive list of the differences between it and KHR_vulkan_glsl. Many of those are differences that the output SPIR-V will have to expose.


Determining input color space primaries from spectrum data If I have an output device, for which I have a known spectrum data for each of it's primaries, I should be able to calculate the CIEXYZ of its primaries using using the integral equations together with the CIE color matching functions. I can then calculate its respective CIExyY values, which for output device are always inside the boundary if the xyY spectral locus - the chromaticity diagram.
My question relates to calculating xyY of primaries of input device.
Suppose I have a known spectrum data for the primaries of some input device - be it a digital scanner or a digital camera or any other input device. It's color filters are not perfect, this means that even a pure spectrum color is represented as non-zero values in its own RGB color space. That means that (if I'm correct) that the whole chromaticity diagram should lie inside of the device RGB color space, and it's primaries lie outside of the diagram and be represented by imaginary colors.
How do I calculate those imaginary primaries for the input device given known spectrum for its primaries? If I'm not mistaken, I cannot simply use the integral formula, because that seems to be only for output devices and the resulting primaries would lie in the locus.

I vaguely understand your question, so this response should be taken more of a long comment rather than an answer.

If I have an output device, for which I have a known spectrum data for each of it's primaries, I should be able to calculate the CIEXYZ of its primaries using using the integral equations together with the CIE color matching functions. I can then calculate its respective CIExyY values, which for output device are always inside the boundary if the xyY spectral locus - the chromaticity diagram.

This is correct in general but note that reference CIE spectral power distribtions are defined for a standard observer.
The term input primaries is a little vague, are you refering to the spd of the light source inside the input device or the combined result produced by the object's spectral reflectance factor (also an spd) multiplied by the spd of the light source of the input device ?
If you know the spectral reflectance factor of the object and the color matching function and the resulting trichromatic value, you should be able to express the spd of the light source as a vector equation something like:
$x_1 \begin{matrix}a_1 \\ a_2 \\ \vdots \\ a_{\lambda}\end{matrix}+x_2 \begin{matrix}a_1 \\ a_2 \\ \vdots \\ a_{\lambda}\end{matrix}+...+x_{\lambda} \begin{matrix}a_1 \\ a_2 \\ \vdots \\ a_{\lambda}\end{matrix} = \begin{matrix}b_1 \\ b_2 \\ \vdots \\ b_{\lambda}\end{matrix}$ where $x$ is from your light source spd and the others are obtained from the remaining (you can for example distribute trichromatic value uniformly to $b$ vector, and fill $a$'s with zeros for non corresponding wave lengths and use the multiplied value for the corresponding wave length), but this is more or less an educated guess rather than a tested solution.
Notice that the expression is a linear system which might help you finding a solution.


Archimedean spiral in C++ Im trying to plot the x and y positions of an Archimedean spiral in C++.

So far I've been trying something like this, but no luck:

int dx = 0;
int dy = 0;
int x = 0;
int y = 0;

for (int i = 0; i < maxPoints; i++)
{
    dx = sin(i * PI / 2);
    dy = cos(-i * PI / 2);
    x += dx;
    y += dy;

    plot(x, y);
}

EDIT: More info

I'm developing a 3D game application that demonstrates the use of the Bullet physics engine by simulating dominos. Instead of placing the dominos in the scene manually I want to use some math to do it for me :)

For anyone who is interested here it is on GitHub.

Figured it out :) The dominos are now being placed along the X and Y coordinates generated by the function.

The original code in the question was plotting a wave of points outwards from the centre position or origin and was not what I wanted. What I needed was for each point to follow the Archimedean spiral with a certain space between the spirals.

Initially I used integer values to store the x and y coordinates but this was causing a precision error by truncating the floating point value in order to store it in the integer data type.

The below example generates points along the spiral continuously, relative to the maxPoints value.

float x = 0;
float y = 0;
float angle = 0.0f;

// Space between the spirals
int a = 2, b = 2;

for (int i = 0; i < maxPoints; i++)
{
    angle = 0.1 * i;
    x = (a + b * angle) * cos(angle);
    y = (a + b * angle) * sin(angle);

    plot(x, y);
}

Code for the project is on GitHub, you will need Bullet and freeglut


Mimic shadertoy in WebGL I was following this guide for drawing circles using shadertoy. Now I am trying to mimic the examples behaviour in WebGL.

Here is a working shadertoy example I try to mimic. The code is not mine, see link above:

float Circle(vec2 uv, vec2 p, float r, float blur) {
    float d = length(uv-p);
    float c = smoothstep(r, r - blur, d);
    return c;
}

void mainImage( out vec4 fragColor, in vec2 fragCoord )
{
    // Normalized pixel coordinates (from 0 to 1)
    vec2 uv = fragCoord.xy/iResolution.xy;
    uv -= .5;
    uv.x *= iResolution.x / iResolution.y;

    float c = Circle(uv, vec2(.2, -.1), .1, .01);

    // Output to screen
    fragColor = vec4(vec3(c), 1.0);
}

And this is the fragment shader I am using:

#version 300 es
precision mediump float;

uniform vec2 iResolution;

out vec4 outColor;

float Circle(vec2 uv, vec2 p, float r, float blur) {
    float d = length(uv-p);
    float c = smoothstep(r, r - blur, d);
    return c;
}

void main()
{
    // Normalized pixel coordinates (from 0 to 1)
    vec2 uv = gl_FragCoord.xy/iResolution.xy;
    uv -= .5;
    uv.x *= iResolution.x / iResolution.y;

    float c = Circle(uv, vec2(-.5, .2), .1, .01);

    // I am discarding all "blackish" values in order to discard "transparent" pixels
    if (c < .1 ){
        discard;
    }
    outColor = vec4(vec3(c), 1.0);
}

I am using another program before rendering this one for transparency I am discarding some pixels, see comment.

And this is the my drawing class, basically I am drawing two Triangles:

class Hud {
  constructor(gl, hudShader) {
    this.hudShader = hudShader;
    this.gl = gl;
    this.glCanvas = document.getElementById("glcanvas");

    const positionBuffer = this.gl.createBuffer();
    gl.bindBuffer(this.gl.ARRAY_BUFFER, positionBuffer);

    gl.bufferData(
      gl.ARRAY_BUFFER,
      new Float32Array([
        -1.0, -1.0,
        1.0, -1.0,
        -1.0, 1.0,
        -1.0, 1.0,
        1.0, -1.0,
        1.0, 1.0
      ]),
      this.gl.STATIC_DRAW
    );
  }

  Update() {}

  Draw() {
    this.gl.useProgram(this.hudShader.program);

    this.gl.uniform2f(
      this.hudShader.config.uniformLocations.iResolution,
      this.glCanvas.clientWidth,
      this.glCanvas.clientHeight
    );

    this.gl.enableVertexAttribArray(
      this.hudShader.config.attributeLocations.position
    );
    this.gl.vertexAttribPointer(
      this.hudShader.config.attributeLocations.position,
      2, this.gl.FLOAT, false, 0, 0);

    this.gl.drawArrays(gl.TRIANGLES, 0, 6);
  }
}

This produces weird behaviour. For example the position of the circle does not correspond to the example. Also if I scale the viewport the circle seems to change position and scale:

What am I doing wrong?

I did not go into much detail with your code but the first thing that comes to mind is that you are rendering with respect to the resolution passed into the shader.

vec2 uv = fragCoord.xy/iResolution.xy;

hope this helps.

Maybe calculate the aspect ratio?

float aspect = float(iResolution.x) / float(iResolution.y ? iResolution.y: 1);


map 16-bit integers to RGB colors with monotonic relative luminosity Red, Green and Blue apparently contribute differently to the luminosity perceived by humans:

https://en.wikipedia.org/wiki/Relative_luminance

I am trying to find an algorithm to map unsigned 16 bit integers into RGB values such that the relative luminance of the RGB values to which the integers map respects their order (with integer 0 being black, integer 2^16-1 white and all other integers mapped to some RGB color).

I don't really care about how the colors evolve as one walks up from 0 to 2^16-1 as long as luminosity always increases and a reasonably large swathe of the RGB space is used.

Apparently I am missing the right words to describe what I am trying to accomplish as I've failed to google anything pertinent. A couple of weeks ago I had stumbled upon a paper that described such an algorithm in the form of a helix that is arranged diagonally in the three dimensional RGB space such that when one starts at the base of the helix and works his way towards the other end, luminosity always increases and a large variety of color is used. Alas, I can't find it any more.

update

Found the algorithm I was looking for. It is known as the 'cubehelix' color scheme by Dave Green and is described here:
http://www.mrao.cam.ac.uk/~dag/CUBEHELIX/. The original paper is here: http://astron-soc.in/bulletin/11June/289392011.pdf

Since you want monotonically increasing luminance, it's probably best to work in a color space that separates luma from chroma, such as a Y'UV space. You can assign values in Y'UV and then convert it to RGB using the matrix at that link.

Start by setting $Y' = N / (2^{16} - 1)$, where N is your input 16-bit integer. Now you just need to make up some function to use for the U and V components.

It wasn't clear to me if you want the chroma part of the color to vary continuously with the input N, or be pseudo-random. For a continuous curve, you could just have it be a circle in UV space:
$$
U = 0.5 \cos(2\pi N/2^{16}) \\
V = 0.5 \sin(2\pi N/2^{16})
$$
or whatever other curve you feel like coming up with. For a pseudo-random chroma, you could use a hash function, or a Halton sequence or another low-discrepancy sequence.


Creating a mesh from a point cloud? I'd like to create a mesh from a point cloud generated by video tracking, ideally using python for some kind of prototype at least.

Initially I thought this is a fairly easy task, connecting the vertices, creating the faces, done ;) Then I've read that Screened Poisson Surface Reconstruction is currently the best approach. Although there is a nice github repository and code to play with, it's hard to understand the papers.

Q: Is there another clever implementation/approach you would recommend or is Screened Poisson Surface Reconstruction the way to go? Any suggestion how to implement that as simple as possible will be greatly appreciated.

There is algorithm called as delaunay triangulation which does triangulation or points it's comparably simple to understand . But it is very slow. If you don't want to implement your own algorithm from scratch please have look at pcl
http://pointclouds.org/documentation/tutorials/greedy_projection.php
Also have look at CGAL
http://www.cgal.org.


Test if a point is on a line segment I have a line segment defined by two end points. And I have another randomly chosen third point and I want to find out if the third point is on the line segment or not. By "On the line segment", I mean that exactly on the line segment, not intersecting the line segment and end up on the other side of the line segment. I hope I explained the issue. Some mathematical explanation along with algorithm to it  is earnestly requested.

Thanks

Let's say you have your two points that define the line segment: $A$ and $B$, and a point $P$ that you are testing to see if it is on the line segment.

Firstly, you get a normalized vector from $A$ to $B$, as well as the distance from $A$ to $B$.

direction = normalize(B - A);
length = length(B-A);

Next, you use dot product to project the point $P$ onto the line segment defined by $A$ and $B$, by first getting the vector from $A$ to $P$ and then dot producting that against the direction vector.

PRelative = P-A;
projection = dot(PRelative, direction);

This projection value represents how far down the line segment from $A$ to $B$ that the closest point to $P$ is.  It may be a negative distance, or it might be farther from $A$ than $B$ is, so you next clamp this value to the line segment.

projection = clamp(projection, 0.0, length);

Note that in your case, instead of clamping, you could just detect when the projection was out of bounds and return false at this point as well.

Continuing on, you now have the distance from $A$ to $B$ that the closest point to $P$ is, as the value in $projection$.  You calculate that actual point:

closestPoint = A + projection * direction;

If $closestPoint$ == $P$, you know that $P$ lies on the line segment defined by $A$ and $B$ because the closest point on that line segment is the point $P$ itself.

However, in practice, you might run into some numerical precision problems due to having limited accuracy in computer calculations.

To get around this, instead of checking if $closestPoint$ and $P$ are the same, you should probably instead consider them the same if the distance between them is smaller than some threshold value.  This way, if precision issues come up, you'll still get the answers you want in the cases you want them.

Too large of a threshold value, and you will get points being considered on the line that you really wouldn't consider on the line (if this happens, make your threshold smaller).

Too small of a threshold value, and some points that you would consider being on the line will not register as being on the line (if this happens, make your threshold larger).


Are textures automatically Morton-coded when loaded to VRAM? I know that texture caching on GPU works to optimize locality in 2D (or 3D if using 3D textures). I believe this is done using space-filling curves like Morton or Hilbert curves.

When passing an array of texture data to the GPU, though, for example using glTextureSubImage2D, the data is specified in scan-line order.

So how and when does the conversion take place? I can think of a couple of possibilites:

The texture is laid out linearly in VRAM, and the texture unit
merges transactions on different areas when loading to cache.
The texture gets swizzled when passed to the GPU, and is stored in VRAM in Morton-coded order (or another space-filling curve).

So which is it, or is there another possibility I've missed? If it's option 2, is there an option to pass textures 'verbatim' if you already have a Morton-coded version on CPU-side? Or would you have to de-swizzle it before loading to GPU?

In vulkan, like a lot of other implicit operations, this transition is made explicit. Images can be in one of 2 tiling layouts: optimal and linear. Linear images have a fixed layout while optimal is with implementation-defined space-filling curve. (There is no way currently to figure out which exact curve is used).

To fill a optimally tiled image you need to copy from another image or from a vkBuffer (which has the data in linear format). The swizzle is done as part of the copy. But with the requirement that the data source must not be changed during the operation. Because other operations are allowed be read from the copy operation's source.


PBR : generating dynamic HDR maps Most of the PBR tutorials out there use HDR maps loaded from disk. But in games these maps need to be generated by taking snapshots of the scene from the objects location which is where problems arise.
Let's say there are just 2 objects with PBR material a sphere and a cube.
Here are 2 problems I come across
1)Each of these 2 objects need an full 360 degree view of the scene around them stored in an cubemap for proper IBL implementation which means for just 2 objects we have to render our scene 12 times!!![6 times to fill the cube map of each object] which is ridiculously expensive and this is just for 2 objects.
Is there an alternative to this especially in dynamic scenes where objects move around all the time?
2)If the sphere and the cube are very close to each other then the color of one object will depend on the light reflected from the other and this light can bounce an infinite number of times between the objects. This can be solved using recursive ray tracing where we control the bounces using an depth variable. Is there an similar recursive approach for the raserization pipeline ?[Similar to 2 mirrors facing each other and reflecting each other problem]
Some theory, ideas or maybe  some linked articles would be of great help

There are no easy answers here, if you are serious then it is possible to replace cubemaps with hemispherical maps, which result in just 2 offscreen renders each instead of 6. They are not popular because the math and setup are nontrivial. RTR4 has references to papers. If you get that working do a writeup here please! Look for Dual-paraboloid shadow mapping.
Another solution is to only update your maps every other frame, or every third frame. So if you have say 12 maps, then update 4 of them each frame round robin style. Also frugal culling can greatly reduce the number of faces that have to be rendered but quality is reduced. I see so many implementations that render all the geometry in the scene to every face when much of that geometry is outside the frustum of that face. This alone can increase performance but scene complexity will play a major role in overall performance.
As far as infinite reflections is concerned, think about that for a second, infinite rays is going to take infinity to render no matter what your technique is. If you manage to trap photon's in a perfect reflector in real life then you can't observe it because observing it destroys the perfect reflector. I'm not trying to be sarcastic, I'm just giving a reality check.
But simple techniques can get good reflections in real time. Screen Space reflections are tricky little buggers to get right but when done well can yield very nice results. You don't see many implementations around because there are a lot of corner cases to consider. Again if you get a good implementation of this working, please share it here!
What you are implying in your post sounds like you would, at the very least, have to do reflections against all the cube faces, this too is not realistic, even on the best hardware, your computes are better spent someplace else, like ray tracing...


Help with wireframe artifacts on mesh procedural UV texturing My goal in my project is to render a procedural texture for a given mesh based on a predefined camera projection.
The way I choose to accomplish it is by creating tex coords based on only the visible triangles for a given projection.

Its important to note that the mesh is unindexed and each vertex is unique to each triangle.
After calculating the UV I pass the tex coords, vertices, MVP matrix to a shader program.

In the vertex shader I define the gl_position as the generated tex coords so in the end of the pass i will get a texture that is defined by the tex coords.

layout(location = 0) in vec3 aPos;
layout(location = 1) in vec2 aTexCoord;

out VS_OUT {
vec4 projector;
vec2 uv;
vec2 old_uv;
} vs_out;
void main()
{
gl_Position = vec4(aTexCoord*2.0f - 1.0f, 0.0f, 1.0f);
vs_out.projector = vec4(aPos, 1.0f);
vs_out.uv = aTexCoord;
}

AN example of the output of the vertex shader

Then in the fragment shader, I color the mesh with a simple red color (in the future I hope to project images instead of the red color) :

#version 450
layout(location = 0) out vec4 FragColor;
in VS_OUT{
vec4 projector;
vec2 uv;
} vs_out;
uniform sampler2D shadowMap;
uniform mat4 model;
uniform mat4 projProj;
uniform mat4 projBias;
uniform mat4 projView;
float ShadowCalc(vec4 proj)
{
float closestDepth = texture2DProj(shadowMap, proj).r;
float currentDepth = proj.z / proj.w;
float shadow = currentDepth - 0.00005 > closestDepth ? 0.0 : 1.0;
return shadow;
}
void main()
{
vec4 ndcProj = (projBias * projProj * projView * model * vs_out.projector);
vec2 projUV = (ndcProj.xy / ndcProj.w);
float fDepth = ndcProj.z / ndcProj.w;
vec4 col = vec4(0.0, 0.0, 0.0, 1.0);
if (0.0 < projUV.x && projUV.x < 1.0 && 0.0 < projUV.y && projUV.y < 1.0 && ndcProj.w > 0.0 && ShadowCalc(ndcProj) > 0.5)
{
col = vec4(1.0,0.0,0.0,1.0);
}

FragColor = col;
}

The result after the fragment shader (the texture is cleared with a green color before)

I then display the mesh with the texture using the same tex coords with the following program:
Vertex shader:

#version 450
layout(location = 0) in vec3 aPos;
layout(location = 1) in vec2 aTexCoord;
uniform mat4 model;
uniform mat4 projProj;
uniform mat4 projView;
out vec2 texCoordOut;
void main()
{
gl_Position = projProj * projView * model * vec4(aPos, 1.0f);
texCoordOut = aTexCoord;
}

Fragment shader:

#version 450
in vec2 texCoordOut;
out vec4 out_color;
uniform sampler2D unwarp_image;
void main()
{
vec4 color = texture(unwarp_image, texCoordOut.st);
out_color = color;
}

The result:

The mesh rendered with the texture on screen

As you can see, the mesh has these strange wireframe artifacts.
I've thought of several solutions such as enabling conservative conservative rasterization but none seem to work.
Any ideas on how to get a clean rendering of the mesh with the procedural texture?

This looks like a side effect of bilinear filtering where it can sample texels neighbouring those that are at the edges of the used areas of the texture - the red/green boundary in your case.

One solution for this is to perform a "colour bleed" pass on the texture where for every 'empty'(Green) texel with none-empty (Red) neighbours you can output the average of all the none-empty neighbours, so what this will do is add a 1 pixel boundary to the red part of your image that matches its neighbours (In this case the average should be red)

This is a solution to a similar problem seen with alpha blended textures with sharp edges where colour bleeding can sometimes appear at the boundaries between semi-transparent and completely transparent pixels when magnifying the image. Most modern art packages will automatically colour bleed the RGB channels when saving RGBA images for this reason.


Gaussian blurring a function defined over a surface mesh I have a scalar function defined over the vertices of a surface mesh. I want to compute an approximate (and generalized, I suppose) "Gaussian blur/convolution" of this function over the surface.

I can imagine for each vertex, taking an average of the function at vertices in it's (multi-hop) neighborhood weighted according to the Gaussian of their Euclidean distance (in R3) from the current vertex multiplied by the local area they represent. This approximation would be good if the surface isn't too curved at the scale of the Gaussian kernel. However it would still be computationally prohibitive.

Is there a more efficient convolving algorithm, perhaps based on some kind of iterated "flow" or partial exchange through edges of the mesh?

The mesh can have either triangular or quadrilateral faces - whichever is more convenient in answering this question. But the edge lengths are not constant, so local neighborhood geometry differs between vertices.

A few different approaches

I'll consider a few variations on your specific request, since you mention efficiency and I suspect your specific request may be the least efficient. I'll also suggest ways of improving the efficiency without varying from your intended approach, so you can weigh up the alternatives.

Blurring the volume instead of the surface

If you want the distance metric to be 3D Euclidean distance instead of 2D Euclidean distance within the surface, then you could perform the blur on a regular 3D grid to which the scalar function you have in mind has been applied. Then you can use the final result of your Difference of Gaussians to calculate the scalar values at the vertices of your irregular mesh. This avoids having to take into account the mesh shape for the bulk of the calculation.

The 3D grid is likely to have a much larger number of vertices than the 2D mesh, but they will all be equally spaced and the large kernel blur can be achieved by repeated application of a small kernel blur taking into account only the 6 nearest neighbours, which will always be at a constant distance away. This approach involves potentially more calculation, but the ease with which the regular grid could be GPU accelerated may appeal.

This will give a different result from performing the blur on the mesh vertices using 3D Euclidean distance. For example, the 3D grid approach will be affected by distinctive regions of the 3D scalar function that are near but not on the mesh. This may be desirable or not depending on your specific purpose.

Using 2D distance instead of 3D distance

If you find that you need the distance metric to be 2D Euclidean within the surface, then you can get a good approximation to a larger kernel Gaussian blur by repeatedly applying a smaller kernel Gaussian blur. If there is not too much variation in the edge lengths within your mesh you may be able to choose a kernel size which allows for only including vertices one edge away at each iteration. This allows for only using single edge lengths to calculate the size of the contribution of a vertex, rather than calculating a 2D multi-edge distance.

3D distance using the surface without the volume

If you need the calculation to be precisely as described in your question, being calculated within the mesh rather than within the surrounding volume, but also using the 3D Euclidean distance, then using nearest neighbours and several iterations will not work. Unless the mesh is near to flat, the repeated application of a nearest neighbour blur will result in an approximation to the 2D Euclidean distance case, since the values will only be able to bleed from vertex to vertex, not directly along the shortest path as they would in a single pass. This will give less spread than would be achieved by a single pass that calculates the 3D distance to a vertex 10 edges away. (I have used 10 edges since you mention a 10-hop in your comment on the question.)

Implementing the blur in a single pass will mean calculating the 3D Euclidean distance between every vertex and every other vertex within a 10 edge radius. This will be expensive, but perfectly possible. Since you mention efficiency, consider that there are some redundancies you can eliminate provided you have sufficient available memory.

The two blurs that you produce prior to taking the Difference of Gaussians will use the same set of 3D distances up to the edge radius of the smaller kernel blur. If you can save these then you only need to calculate them once, rather than once per blur.

Also, each distance will be used twice per blur - once in each direction as the length from vertex A to vertex B is the same as the length from B to A. Caching/memoising these distances will avoid calculating them twice.

Effects from arbitrarily many edges away

If the surface curves such that some vertices which are many edges away are still near enough to affect each other in 3D distance, then rather than considering vertices within a certain number of edges away, you may need to consider all vertices within a certain 3D radius, regardless of how long the path via edges. In this case you can consider fewer vertices by using space partitioning, choosing a specific method which suits the mesh.

If you don't want parts of the surface which approach each other to influence each other, then you probably want a 2D distance metric rather than the 3D one.

If you have a wide range of different edge lengths then you may find the same problem of not being able to define a set number of edges to traverse, even if the mesh is fairly flat. Again you may need to define a 3D distance instead of a number of edges, and consider all vertices that lie within that radius.


Two-dimensional bounded area defined parametrically How do I go about defining this area without using a piecewise function?

I think it has something to do with Bilinear Surfaces but I'm not sure how to get started.

One solution is bilinear mapping, which works for every convex quadrilateral.

Let $A, B, C, D$ be the vertices of a quadrilateral $Q$ in the plane, in this order.

Then $f(u,v)=(1-u)(1-v)A + u(1-v)B + uvC + (1-u)vD$ maps the square $[0,1] \times [0,1]$ to $Q$.


How are the colors for each pixel from a Ray Tracer stored in an Image? I have begun learning how to create a Ray Tracer and 1 thing I am confused about is how the pixel color from a Ray Tracer is stored into an image. Do we use SetPixel for a Bitmap? Do we use a third party library like libpng?

Most tutorials don't really explain this well online, so if anyone can explain what the most common method for this is that would be great.

It doesn't matter. All that matters is that you can store information per pixel.

In most cases this means allocating a x*y array of ints (for 3 byte-sized channels of RGB) to store the pixel values and then feed it into your image compression library of choice to save it out to disk.


Inverse Kinematics with more than 2 segments I'm building a 2D inverse kinematics system and can't figure out how to find all the joint positions. I have the math to make a typical IK set up with a single bend (e.g. arm, leg), but I get stuck when adding another bend.

How do I find the position of all joints for an inverse kinematic chain that is longer than 2 segments?

Here's an example of the planar solution I'm looking for: https://youtu.be/3sqd_CZNq7s?t=6s

I'm building this in Adobe After Effects so using an existing IK solver isn't going to work.

Here's my visual setup for reference:

What I'm aiming for is that when the blue box is moved (target) joints 2 and 3 bend appropriately to maintain their length. Joint 1 is static and doesn't move.

To make this work, I need x,y coordinates to be calculated in real time. No need to render anything on screen—just the position values for each joint.

Here's my javascript so far:

var angle1, angle2, angle3, theta1, theta2, theta3, targetSqrDist;

// joints
var joint1 = thisComp.layer("joint 1");
var joint2 = thisComp.layer("joint 2");
var joint3 = thisComp.layer("joint 3");
var joint4 = thisComp.layer("joint 4");

// segment lengths
var segment1Len = length(joint2.transform.position, joint1.transform.position);
var segment2Len = length(joint3.transform.position, joint2.transform.position);
var segment3Len = length(joint4.transform.position, joint3.transform.position);

// target position
var target = thisComp.layer("target");
var ix = target.transform.position[0] - joint1.transform.position[0];
var iy = target.transform.position[1] - joint1.transform.position[1];

// target square distance
targetSqrDist = ix * ix + iy * iy;

// Segment 1
angle1 = Math.max(-1, Math.min( 1,
    (targetSqrDist + (segment1Len*segment1Len) - (segment2Len*segment2Len)) / (2 * segment1Len * Math.sqrt(targetSqrDist))
));

theta1 = Math.atan2(iy, ix) - Math.acos(angle1);

var s1X = joint1.transform.position[0] + segment1Len * Math.cos(theta1);
var s1Y = joint1.transform.position[1] + segment1Len * Math.sin(theta1);

// Segment 2
angle2 = Math.max(-1, Math.min( 1,
    (targetSqrDist - (segment1Len*segment1Len) - (segment2Len*segment2Len)) / (2 * segment1Len * segment2Len)
));

theta2 = Math.acos(angle2);

var s2X = s1X + segment2Len * Math.cos(theta2 + theta1);
var s2Y = s1Y + segment2Len * Math.sin(theta2 + theta1);

// Segment 3
angle3 = Math.max(-1, Math.min( 1,
    (targetSqrDist - (segment2Len*segment2Len) - (segment3Len*segment3Len)) / (2 * segment2Len * segment3Len)
));

theta3 = Math.acos(angle3);

var s3X = s2X + segment3Len * Math.cos(theta3 + theta2);
var s3Y = s2Y + segment3Len * Math.sin(theta3 + theta2);

This code would be applied to each joint with a unique position value assigned to it, depending on the joint's location.

What you have here is an underconstrained problem*. To solve this problem you do what you do to any other underconstrained system. This is also why you mostly only see the 2 joint version as it is not underconstrained in the planar solution, and in any case the preregistration is trivial.

Basically you have 2 options:

Add more constraints
Deal with the freedom

Adding more constraints to the system reduces the degrees of freedom. For example if you only permit one of the joints to rotate about its z axis and the another around its x axis then y then you have reduced the problem to the same 2 member problem. Another way to constrain it is to force the orientation of the last link. This also reduces it to the same case as 2 links. But there are other meaningful ways to reduce complexity that depend on your application.

Image 1: Constraining one member reduces the problem to a 2 member problem. In the planar case that means 2 solutions (other in blue).

Dealing with the freedom just means that your answer is going to be a family of answers. Then you develop a way to choose the solution from that family. Letting the user adjust the parameter is equivalent to adding one more constraint. But there are other methods such as least squares, or energy methods that yield different results. Sky is the limit here. For example I have worked with robot controllers that decided the parameter based on what was the least amount of rotation on joints. The problem was that this wasn't entirely deterministic so over time the robot ended up doing things differently...

* This is more suited to Engineering.SE but since IK is a common graphics problem lets deal with it here.


Comprehensive explanation of conservative rasterization algorithm I am trying to implement conservative voxelization as part of real time GI in my hobby rendering engine. I find this article by NVIDIA. I am stuck at understanding the second algorithm. The problem that I have is that I don't understand this section of the article

  We describe both algorithms in window space, for clarity, but in
  practice it is impossible to work in window space, because the vertex
  program is executed before the clipping and perspective projection.
  Fortunately, our reasoning maps very simply to clip space. For the
  moment, let us ignore the z component of the vertices (which is used
  only to interpolate a depth-buffer value). Doing so allows us to
  describe a line through each edge of the input triangle as a plane in
  homogeneous (x c , y c , w c )-space. The plane is defined by the two
  vertices on the edge of the input triangle, as well as the position of
  the viewer, which is the origin, (0, 0, 0). Because all of the planes
  pass through the origin, we get plane equations of the form

  ax c + by c + cw c = 0  a(xw c ) + b (yw c ) + cw c ) + cw c = 0  ax +
  by + c = 0

  The planes are equivalent to lines in two dimensions. In many of our
  computations, we use the normal of an edge, which is defined by (a, b)
  from the plane equation.

First, I don't understand how to visualize a plane that use w value as one of this coordinates and what is the meaning of this plane. And then later in the article, they calculate this plane by doing a cross product like this

// Compute equations of the planes through the two edges

float3 plane[2];

plane[0] = cross(currentPos.xyw - prevPos.xyw, prevPos.xyw);

plane[1] = cross(nextPos.xyw - currentPos.xyw, currentPos.xyw);

Again, I still don't understand why this is true.

Is there any relation between the plane normal with the line normal because in the next part, the algorithm use this normal xy value to move the line outwards by substracting the z value of this plane.

// Move the planes by the appropriate semidiagonal

plane[0].z -= dot(hPixel.xy, abs(plane[0].xy));

plane[1].z -= dot(hPixel.xy, abs(plane[1].xy));

// Compute the intersection point of the planes.

float4 finalPos;

finalPos.xyw = cross(plane[0], plane[1]);

I can understand it if the xy component of the plane's normal is actually the normal of the line as well and the z value of the plane's normal represent the line distance from point (0, 0)

Can someone please explain in more detail or provide me some visualization of this algorithm ?

I'm not sure what exactly you mean by

  how to visualize a plane that use w value as one of this coordinates

but here's a sketch that will, hopefully, clarify this sentence:

  a line through each edge of the input triangle as a plane in homogeneous (x c , y c , w c )-space

I've used uppercase letters for clip-space, and lowercase letters for window space.

Let's say AB is an edge of your original triangle.
It projects to an edge ab in window coordinates.
Point A is defined using [XA, YA, ZA, WA] coordinates. After projection it will have coordinates [xA, yA] = [XA/WA, YA/WA] and the value ZA/WA will be used to determine which object is closer to the camera at this point.
So your [X, Y, Z, W] always turns into a [X/W, Y/W] in the end. You can see that Z doesn't play any role in where the point is going to end up on the screen. So we just throw it away for now and interpret the remaining [X, Y, W] as 3D coordinates of the points.

Now we have point A = [XA, YA, WA] and B = [XB, YB, WB].

The projected point a will lie on the line that connects [0, 0, 0] and A.
Point b will lie on the line [0, 0, 0] and B. Three points define a plane, and a normal to the plane defined by 3 points can be found as a cross product of two vectors in this plane (OA and OB, for example).
"For a convex polygon (such as a triangle), a surface normal can be calculated as the vector cross product of two (non-parallel) edges of the polygon."
More on why in Wikipedia here and here.

Generally this moves your problem from 2D domain in the projected coordinates into a 3D domain of not-yet-projected points. The 2D and 3D are "dual" to each other. For example, if a point lies in the plane of OAB, it for sure lies on the line through ab (and vice versa).

The thing you need to remember is that W is your 3'rd coordinate, not Z! I know, it's very counter-intuitive, but it is what it is, Z only carries depth information, it does not affect position of objects on the screen.

I hope I cleared it up for you a bit.


Checking the correct offset direction for Bezier curve offset I am implementing Bezier curve offset.

My approach is simple:

Given the handles, calculate the vectors normal to them
Move the handle along the normal vector.
If the new curve intersects the old one, reverse the offset direction.
etc.

I would like to find an alternative to step 3. I believe the problem comes from my careless handling of vector maths (mainly calculations involving the cross product). Are there other ways to check the offset direction other than the one I just described?

For a two dimensional spline it is possible to define a left and a right side, or let us call the directions counterclockwise and clockwise side (or maybe port and starboard would be appropriate). The side is easily identified as the cross product of the tangent in direction of point order crossed by the paper plane.

Image 1: Offset side in 2D.

The higher dimensional cases are more problematic, I will be examining the 3D space only. A free space curve has no inherent good second direction. Mathematicians have defined, what is essentially a hack, that the second direction is in the direction of curvature which has a mathematical definition. The only problem is that curvature is subject to change and can flip direction along the curve. This is not good.

So how to counter this? Well simplest is if a curve on a surface of a 2D plane or surface. In this case you can simply use the underlying primitives directions as reference and do a 2D case. This has the added benefit of not doing a somewhat random offset direction.

For the free curve case there are several approaches you can use, they all have the downside of needing to keep some extra data with you.

One is to calculate the location where the curvature is 0 and "cut" the curve at that point and continue with a flipped curvature direction. In case of a straight line just continue with last good direction. A variation of this is not to use curvature but a user defined vector and use that instead.

This can also be done numerically by sampling multiple points along and detect the flip by comparing nearby result and not letting it turn more than a certain degree untill you declare a flip (what your doing sound like a variation of this). One other is to propagate a vector along the curve. One way to do this is to specify a vector at each point and let the user flip them if he feels the need. Or interpolate vectors only at certain points on the curve.

Most applications simply just use the will not do it or bug out option. This has the benefit of being rather sane, as the curvature based bi-normal is a bit erratic to say the least. This way the user has to come up with a way to make the directional control with for example a ruled surface.

*  Bézier, NURBS or something other like a Catmul-Rom spline doesn't matter


Test if polygon edge is facing another polygon How do I test to see if a polygon edge is facing another polygon? For the edge being tested to return true, the edge needs to be facing a polygon which has a different attribute than itself.

In this example, I am iterating through the edges on Polygon A. Segment 0 and segment 1 would return true for facing the polygon, where segment 2 would return false, because, while it is still facing a polygon, the polygon has the same 'green' attribute.

You can take the normal of the line segment you're testing and find the dot product of that normal and the normal of each segment of the polygon you're testing against. If it is positive, then they are facing the same way. If the dot product is negative, then the line segments are facing in opposite directions.

If they are facing in opposite directions, then they're either pointing at each other, or away from each other. You can figure out which by calculating a vector from a point on the first segment to a point on the other segment and seeing if it's facing the same direction as its normal (by again using the dot product). If so, they are facing each other, otherwise they're facing away.

You can skip all the tests above by first testing if the polygons are the same color and only doing the tests if they're not.


What is the difference between shading and shadowing? Shading and shadowing are often mentioned in how to create realistic 3D graphics, but the distinction is not clear. Are both responsible for the generation of shadows, or do they refer to the same concept?

In simplified terms, shading controls how object's surface's brightness changes when the angle between its normal and light vector changes. Shadows are areas where the light cannot reach because it's occluded by other objects. So no, they both are not responsible for creating shadows.


What is the physical interpretation of Normal, Roughness, Albedo and Metalness map? In PBS context what is the metalness map? Are my interpretation of Normal, Roughness and Albedo map correct?

I know almost for sure that:

Normal map : Models the normals per differential region in our material (kind of).

Roughness map : Together with the normal map allow to define a microfacet distribution in a differential region of our material.

Albedo map : This models the percentage of light that is diffusely reflected.

Normal + Roughness allow to define the amount of energy that is specularly reflected.

I struggle to understand the physical meaning of the metalness though, I understand it's related to have a material look like metal of course, but physically speaking I don't get what it models.

Those parameters have no direct physical equivalent. They were chosen to be intuitive and to range from 0-1. See the Burley's Physically-Based Shading at Disney for the full rationale and derivation of what is now commonly referred to as "PBR".

The "metalness" parameter is a simple blend between two models - one dielectric BSDF with a coloured diffuse and white specular, and a metallic BSDF without diffuse and a coloured specular component.


Screen to World coordinates glm::unProject I'm having some trouble trying to figure out how to go from a coordinate on the screen, to 3D world coordinates on a plane.

My case is as seen on the picture further down. World origin is in the center with the grid representing the X-Y plane.
I want to be able to click/touch on the screen, and transform the screen coordinates to the correct 3D world coordinates on the X-Y plane.

I first tried to figure out the maths myself by simple geometry, but got stuck...
I then tried the function glm::unProject() after checking out this post.

The screen has dimensions mWindowWidth and mWindowHeight.
x, y are the screen coordinates where origin is in the top left corner. Positive x to the right, positive y down.

projM is the projection matrix.

Third parameter is the model matrix, which I believe in my case only needs to be an identity matrix.

I get the world coordinates X and Y (Z is already known as it is on the plane) by using these parameters in glm::unProject:

glm::vec3 posVec = glm::unProject(
   glm::vec3(x, float(mWindowHeight) - y, 1.0f),
   glm::mat4(1.0f),
   projM,
   glm::vec4(0.0f, 0.0f, float(mWindowWidth), float(mWindowHeight))
);

What happens is that if I have screen coordinates near world origin, the world coordinates seems correct. But If I use screen coordinates near the edges and corners of the screen, I get a certain offset (see image below).

Where is this offset coming from, and what am I misunderstanding?

EDIT:

As mentioned in the comments, the second parameter should be the "model-view" matrix. I tried to switch glm::mat4(1.0f) with viewM (view matrix). The effect is still the same, if not with even more offset.

I also tried to get the z buffer depth value by using:
(Got it from this tutorial)

float z;
glReadPixels(int(x), int(mWindowHeight) - int(y), 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &z);

This results in very small values of X and Y world coordinates, and essentially where ever I touch, the resulting coordinates are near the origin.

You can calculate the world position of the pixel on near plane quite easily by first defining Normalized Device Coordinates (NDC) for the point and then transforming the NDC back to the world space. You can calculate NDC for your point as follows:
$$v=[2*(x+0.5)/width-1, 1-2*(y+0.5)/height, 0, 1]$$
I'm using 0 here for the z-component assuming near-plane in NDC is 0, but it could be something else as well, so check with your projection matrix what values comes out at the near-plane. This vector must be then transformed to clip-space by multiplying by near-plane distance:
$$v'=np*v$$
Now that you have clip-space coordinates, you just need to transform this back to world space with the standard clip$\rightarrow$world transformation matrix:
$$v''=M*v'$$


B-Spline patches on the tessellation shader. How to get the control points I have a polygonal mesh with ngons in it. I want to make a tessellation shader to which I send only the regular quads in such a way that it can construct a B-Spline surfaces across it. I came across the useful tutorial, but I do not completely understand it, and my problem is slightly more general in that it contains irregular polygons and vertices. In the mesh I am looking for regular quads, i.e., quads for which all vertices have valency 4. They look like this

     |    |
  ___|____|___
     |    |
  ___|____|___
     |    |
     |    |

It is the inner face that I want the tessellation shader to divide up and then smooth using B-spline method. However, for the B-spline method I need  16 points, So I must subdivide the inner face like this

 ______         .  .  .  .
|      |        .  .  .  .
|      |   -->  .  .  .  .
|______|        .  .  .  .

I am lost on how to find these points, I can of course just use averaging to get these points but then there will be no smoothing happening. The subdivision rules I normally use are all binary, that is, they split edges into two edges, but this can not produce such a set of control points.

How do I proceed from here? Should I just resort to using quadratic B-splines, and use the top diagram to find approximations of the half way points and send those to the shader? Another issue is that adjacent faces of regular quads may very well be triangles or pentagons etc. So it is difficult to find the points I need for the tessellation shader to perform B-spline smoothing.

Thanks for any tips,
if anyone would like some more clarification about the problem please ask, I realize that the question might be vague.

For regular quad faces, what you want to do is send the vertices for the quad you're evaluating and the eight neighboring quads around it: a total of 4x4 vertices. The B-spline needs to know where the neighboring vertices are in order to generate a surface patch that connects smoothly to the neighboring patches.

In the case that the adjacent faces aren't quads, I don't think you can use the straightforward B-spline evaluation. In Catmull-Clark subdivision, everything is quads after the first level of subdivision, though you'll still have extraordinary vertices to deal with.


issue with importance sampling I was trying to do importance sampling on lambertian surface.At first I uniformly choose direction from unit sphere.
vec3 direction = camera->genDirection();
...
direction = random_in_unit_sphere();
float cosine = dotp(direction,surfaceNormal);
/*
float dotp(float val){
val = dot(val);
if(val>0.0001f) return val;
else return 0.0001f;
}
*/
vec3 brdf_result = material->baseColor/Pi;//lambertian
vec3 pdf = 1.0f/(2.0f*Pi);
throughput = throughput * brdf_result * cosine / pdf;

With 10 samples per pixel,yields:
then I choose random direction from the unit hemisphere above the surface
direction = random_in_unit_hemisphere(surfaceNormal);
float cosine = dotp(direction,surfaceNormal);
vec3 brdf_result = material->baseColor/Pi;
vec3 pdf = 1.0f/(1.0f*Pi);
throughput = throughput * brdf_result * cosine / pdf;

the result is very similar,except for less noise

and then I use the importance sampling method from (http://in1weekend.blogspot.com/)
    class onb {
public:
    vec3 operator[](int i)const { return axis[i]; }
    vec3 u()const { return axis[0]; }
    vec3 v()const { return axis[1]; }
    vec3 w()const { return axis[2]; }
    vec3 local(float a, float b, float c) { return a * u() + b * v() + c * w(); }
    vec3 local(const vec3& a) { return a.x * u() + a.y * v() + a.z * w(); }
    void buildFromNormal(const vec3& n) {
        axis[2] = normalize(n);
        vec3 a;
        if (std::abs(w().x) > 0.9f)
            a = vec3(0.0f, 1.0f, 0.0f);
        else
            a = vec3(1.0f, 0.0f, 0.0f);
        axis[1] = normalize(cross(w(), a));
        axis[0] = cross(w(), v());
    }

private:
    vec3 axis[3];
};
vec3 randCosDir() {
    float r1 = randFloat01();
    float r2 = randFloat01();
    float z = sqrt(1.0f - r2);
    float phi = 2.0f * Pi * r1;
    float x = cos(phi) * 2.0f * sqrt(r2);
    float y = sin(phi) * 2.0f * sqrt(r2);
    return vec3(x, y, z);
}

,
onb uvw;
uvw.buildFromNormal(surfaceNormal);
direction = normalize(uvw.local(randCosDir()));
float cosine = dotp(direction,surfaceNormal);
vec3 brdf_result = material->baseColor/Pi;
vec3 pdf = dotp(uvw.w(), direction)/Pi;
throughput = throughput * brdf_result * cosine / pdf;

however the result is different:

The baseColor of the wall is vec3(0.8f,0.8f,0.8f),and the color of the dome light is vec3(1.0f,1.0f,1.0f).
In some tutorial the cosine item is inside the lambertian brdf,and some are in the render equation,and in
http://in1weekend.blogspot.com/ "weekend one" there is no cosine item at all.I really get messed with those concepts.Is there any one help me?thanks so much.
another rendering with baseColor = vec3(1.0f,1.0f,1.0f) and dome color = vec3(0.5f,0.5f,0.5f)  (importance sampling)
the average color of the final image over all pixels is vec3(0.470884f,0.470884f,0.470884f).

10,000 samples per pixel with uniform hemisphere sampling:

I just found that if I use pdf = 1.0f/(4.0f * Pi) in unit sphere sampling or pdf = 1.0f/(2.0f * Pi) in unit hemisphere sampling,the result are almost the same as the importance sampling one(I also get the same result when set the baseColor to vec3(0.4f,0.4f,0.4f),the half of vec3(0.8f,0.8f,0.8f).
And the surface area of a unit sphere is just 4.0f * Pi(I forget why I use 2.0f * Pi before).I HAVE to come to two conclusions:
1: the importance sampling one is correct!
2: I'm too stupid!!!!!!


How to raytrace triangular faces when camera is not at origin? I've watched the Pixar In A Box, Maths of Rendering Tutorial on Khan Academy. I've started writing a raytracing program and have come to the point where I must solve for t (the parameter for the parametric ray through the camera and a point on the view plane).
In the tutorial, this is done by simply substituting the parametric formula for each x, y, and z into the equation of the plane that the triangle lies on. This leaves a single unknown (t).
However, in all the examples and exercises that are given by the tutorial the camera is located at [0,0,0]. As I'm sure you'll understand, this is in many cases unrealistic. Because the camera is at 0, they simplify as follows:
$$i=\frac{1}{t}C+tP$$
becomes simply:
$$tP$$
Which makes for an easy solve. My question is when the camera is not at [0,0,0] should I:

Calculate the coordinates of the plane relative to the camera. (So the camera acts as if it is [0,0,0]
Solve the equation as a quadratic. I'm still confused a bit about why this is quadratic in the first place. Could someone explain this? And which solution do I use?

Of the two options, which is more acceptable for use in this community? Why? Is one more computationally efficient?
Edit:
I think I have found a solution as follows:
The parametric equation (which is effectively just a weighted average) is represented like this in the tutorial:
$$x(t) = \frac{1}{t}C_{x}+tP_{x}$$
But I realize now that is the same as:
$$x(t)=C_{x}+t(P_{x}-C_{x})$$
Which is dramatically easier to solve.
So my final question is, which of the three is the most acceptable way to solve for t? I'm assuming the third one.

Wow I'm stupid, the whole problem was I thought it was 1/t and it is actually 1-t in parametric form. Hope this helps someone anyway.