Why is recursion forbidden in OpenCL? I'd like to use OpenCL to accelerate rendering of raytraced images, but I notice that the Wikipedia page claims that recursion is forbidden in Open CL. Is this true? As I make extensive use of recursion when raytracing, this will require a considerable amount of redesign in order to benefit from the speed up. What is the underlying restriction that prevents recursion? Is there any way around it?

It's essentially because not all GPUs can support function calls—and even if they can, function calls may be quite slow or have limitations such as a very small stack depth.

Shader code and GPU compute code may appear to have function calls all over the place, but under normal circumstances they're all 100% inlined by the compiler. The machine code executed by the GPU contains branches and loops, but no function calls. However, recursive function calls cannot be inlined for obvious reasons. (Unless some of the arguments are compile-time constants, in such a way that the compiler can fold them and inline the entire tree of calls.)

In order to implement true function calls, you need a stack.  Most of the time, shader code doesn't use a stack at all—GPUs have large register files and shaders can keep all their data in registers the whole time.  It's difficult to make a stack work because (a) you would need a lot of stack space to provide for all the many warps that can be in flight at a time, and (b) the GPU memory system is optimized for batching together a lot of memory transactions to achieve high throughput, but this comes at the expense of latency, so my guess is stack operations like saving/restoring local variables would be awfully slow.

Historically, hardware-level function calls haven't been too useful on the GPU, as it has made more sense to inline everything in the compiler.  So GPU architects haven't focused on making them fast.  Probably some different tradeoffs could be made, if there is a demand for efficient hardware-level calls in the future, but (as with everything in engineering) it will incur a cost somewhere else.

As far as raytracing is concerned, the way people usually handle this sort of thing is by creating queues of rays that are in the process of being traced.  Instead of recursing, you add a ray to a queue, and at the high level somewhere you have a loop that keeps processing until all the queues are empty.  It does require significant reorganization of your rendering code if you're starting from a classic recursive raytracer, though.  For more info, a good paper to read on this is Wavefront Path Tracing.


How to assign a texture to a single entry of a texture array? The C++ psuedo code to transmit the idea of what I want to do would be:

Texture_array tarr = Texture_array(6);
tarr[0] = texture;

Which I am attempting to do as follows:

    glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
    glTexStorage3D(GL_TEXTURE_2D_ARRAY, 1, GL_RGBA8, 894, 894, 6);
    glTexSubImage3D(GL_TEXTURE_2D_ARRAY, 0, 0, 0, 0, 894, 894, 0, color_format, GL_UNSIGNED_BYTE, data);
    //Set the texture parameters of the image
    glTexParameteri(target, GL_TEXTURE_WRAP_S, GL_REPEAT);
    glTexParameteri(target, GL_TEXTURE_WRAP_T, GL_REPEAT);
    glTexParameteri(target, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
    glTexParameteri(target, GL_TEXTURE_MAG_FILTER, GL_NEAREST);

On the fragment shader I attempt to read the value as:

vec3 c = vec3(texture(text, vec3(f_uv.xy,0)));

However the end result is completely blak, despite data being an array of a valid texture (I have tested it and know data has been properly initialized, it works with normal textures, just not texture arrays)

glTexSubImage3D(GL_TEXTURE_2D_ARRAY, 0, 0, 0, 0, 894, 894, 0, color_format, GL_UNSIGNED_BYTE, data);

You're transmitting 3D data. But you told OpenGL that the 3D block of data has a depth of 0. So the number of pixels you're transmitting is... 0.

I have no idea why OpenGL doesn't provide an error when passing 0 for any of the sizing components (since that means you're not passing data). But that's your problem. You need to pass 1 for the depth if you're uploading a single 2D slice to the array.


Proper way to handle diffuse + refraction + reflection rays in path tracing? Let's say we are path tracing and that there is an object which has some amount of diffuse reflection, and some amount of specular reflection.

What is the best or correct way to handle that in the path tracer?

Looking at smallpt (http://www.kevinbeason.com/smallpt/), it doesn't allow for objects to have both, but instead forces materials to be either diffuse or specular.

I've seen other path tracing code which rolls a random number to choose which way to go, using the luminance of the diffuse and specular reflectance as weightings in the random roll.

I've also seen code which follows both paths and diverges when this happens and sums the results, where each recursive ray was multiplied by it's corresponding reflectance.

I've seen refraction dealt with similarly to all the above.

Is there a physically correct way to handle these situations?

Let us take one step back.
When you do path tracing, you are doing a Monte Carlo integration.
What does this mean? You try to solve $\int f(x)\mathrm{d}x$ by sampling.
The Monte Carlo integration says for a sufficiently big $n$: $\frac{1}{n}\sum_{i=0}^n f(x_i) \rightarrow \int f(x)p(x) \mathrm{d}x$

If we instead sum $\frac{f(x_i)}{p(x_i)}$ we get $\frac{1}{n}\sum_{i=0}^n \frac{f(x_i)}{p(x_i)} \rightarrow \int f(x) \mathrm{d}x$.
Now your $f(x)$ is the rendering equation and you got yourself a way to approximate the rendering equation.

Now that we are on the same page about the basics, let's consider what you do.
You generate a path of light, that will become one of the samples $\frac{f(x_i)}{p(x_i)}$. In order to do this you model the interaction of the light with the surface at each step along the path. And you always update your current path probability $p(x_i)$.

What is now the correct way to handle a material that has multiple different ways that a ray could travel? There is no one correct way. The only important part is, that your values $n$, $f(x_i)$ and $p(x_i)$ stay correct.

When you diverge the path and spawn two rays, you have to increase the number $n$ of total samples accordingly because now you have done two samples. Then handle both pathes as if they were separate from the beginning.

If you want to only continue one path, you can do it with the russian roulette way.
Define three intervals (one common way is to chose the intervals to be a partition of the numbers between 0 and 1). The relative sizes of these intervals have to be proportional to the material. The tricky part is how do you design this. In my class we defined a number reflectivity $\rho$.
We draw a random number between 0 and 1, if it is lower than $\rho$ the path ends. Else we reflect. The BRDF will take care to weight the specular vs diffuse part if you do enough samples.

Of course you can improve this via importance sampling, and that is probably where you have problems. For a very specular surface design a random distribution that just draws more directions from the specular reflection direction. Rest will be diffuse. I don't have enough experience with actually bulding path tracers to help you how to design one, but your sources will probably have something here.

Important here is again: Design it so that you know how probable a specific direction is, and then keep your $p(x_i)$ up to it, so you can in the end divide by this and get a correct Monte Carlo estimate.

TL;DR There is no correct way. It is ONLY important to keep the probabilities of samples in mind.

I hope I haven't missed the point of your question entirely.

Addition:
Monte Carlo samling, as long as you correct weighting by probability, will always with infinite samples converge against the correct value. This means even if you use a complete stupid way to generate samples, if you do it long enough, that will work. Maybe not in your lifetime or that of your great grandchildren, but it will.


How to triangulate from a Voronoï diagram? I computed a Voronoï diagram from a set of points (with Boost.polygon).

I try to find a Delaunay triangulation, connecting each cell center for each Voronoï edge, but I miss some edges.

In the following image, the red dots are my initial points, the blue lines are the Voronoï edges (I ignored infinite edges), and the green lines are the triangulation edges (one green edge for each blue edge, connecting two cell origins).

We can see that diagonal edges are missing. What am I missing?

The centre point in your diagram is a degenerate edge of the Voronoi diagram. If you generate a Voronoi diagram for an irregular point cloud, every vertex will have degree 3. A vertex with degree 4 (or more) can only happen when two (or more) vertices coincide. That means there is a zero-length edge between them. But that edge should still have a corresponding edge in the Delaunay triangulation. The issue is that it's arbitrary which of the two possible edges you pick, because the zero-length edge has no associated direction.

To visualise what I'm talking about, consider starting with four less regularly spaced points (such that we start off with only degree-3 vertices) and gradually translating them into their regular positions.

We can do this in two different ways, which both lead to the degenerate case in your diagram. You will see that you end up with two different Delaunay triangulations, which are both valid limits for the degenerate case:

I assume that your code is missing this degenerate case for one reason or another, but without actually seeing how you compute the Delaunay triangulation from the Voronoi diagram it's impossible to point you any further than this.

Also note that having even higher degeneracies (by more than four points distributed at equal angles around a circle) would probably require additional attention:

These animations also show that (even in a non-degenerate case), the corresponding Voronoi and Delaunay edges do not necessarily actually cross within their finite extent. That might make it harder to see that the 2 (or 3) edges that triangulate the regular polygon at the end actually correspond to several degenerate edges which are all at the centre. Also note that in total there are 5 different triangulations of the pentagon and 14 triangulations of the hexagon (although I don't know whether all 14 can be obtained by deforming a non-degenerate triangulation).

Edit (by OP)

Voronoi diagrams computed with Boost.polygon enable to walk through each Voronoi vertex, and each edge linked to those vertices (clockwise or counter-clockwise). In this way, it is possible to create one triangle for each pair of edges (two connected edges will link to 3 cells).


How many Direct/Compute/Copy Queues are meaningful? DirectX 12 exposes command queues for either graphics (called "Direct"), compute or copy tasks. In terms of provided functionality, each one is a super-set of the following one. The specification states that command queues can be executed concurrently by the device.  However, the API does not limit the number of command queues in any way (at least I am not aware any limitation).

Apparently, different vendors handle this very different:

Intel states in a recent presentation (slide 23) that currently their GPUs are not able to handle Graphics & Compute in parallel and that the copy engine has a weak throughput. They advise against the use of multiple graphics/compute queues.
AMD started long time ago to advertise the use of queues / "asynchronous shaders" starting with Mantle and the current gen consoles. There are also some developers (example) that confirm significant performance gains by executing compute and graphics tasks in parallel.
There has been recently some fuss about Nvidia not supporting asynchronous shader in the hardware: Using separate Graphics and Compute queue at once seems to make things slower which indicates driver emulation. Parallel copy operations, on the other hand, have been supported by CUDA for a very long time, which makes it clear that the DMA engine can work independently.

Is there any way to decide at runtime if it is meaningful to commit CommandLists to multiple CommandQueues instead of a single one? (given that former case does not involve much engineering-overhead)

While I can easily see how it is useful to perform memory operations parallel to compute/graphics operations, it strikes me as unnecessarily complicated to run multiple compute and graphics processes in parallel (unless there is no major perf. benefit).
It is also not clear to me, how this can lead to significantly better performance anyways; except for pathological cases where many small sequential tasks are not able to generate enough GPU load.

Ship your application with a benchmarking sequence testing the actual platform. (Possible answer for many questions I guess...)

I suspect the performance is highly dependent on how you use the hardware. Since the hardware is unlikely to somehow instrument your application backwards, telling you what to do, I´d go with whatever looks good in your design.

  "...command queues can be executed concurrently by the device..."

Keyword is CAN. I see no reason why any vendor would screw this up. In the end it is the platform provider (Intel/AMD/Nvidia) who is responsible of making you a good enough driver for you not to consider switching vendor. If they do have a "know issue" with this functionality (which by the way has no functional meaning, only performance) then they should also solve it using what they know. I mean for crying out loud, the fallback is something they have already implemented; synchrounous execution.

Hardware is enough voodoo as it is for us developers.


Estimating real world object distance from photo I'm not really sure if this is possible but it should be.

I currently have a photo of a sphere, whose radius I've measures physically. Knowing the apparent size of the sphere from the photo and all internal camera parameters is it possible to estimate the distance from the camera to the sphere?

There is an equation, more used in lens imaging that is useful:

object_image_size = (real_object_size * lens_focal_length)/object distance

Problem is my object_image_size is in pixels and I need to convert it to a real world measure, preferable the size that actually showed up on the sensor.

Yes, theoretically, for the sphere no for other things in image. If it is possible to augment the situation then it is possible to measure anything you can track much more reliably. First, knowing the scale of a object that you can successfully track is a requirement to getting a scale for the image as the cameras lost all scaling info. But this requirement alone is not enough.

In practice you do not know the other settings of a camera (unless calibrated with stereo geometry first), all kinds of problems arise when you do real camera tracking. Such as real cameras lens not being ideal. By tracking more than 2 features on the scene you can automatically estimate the actual focal distances and camera distortions. More importantly you can get an error estimate on your data.

One way is to have two photos from different angles in a scene with enough depth variation (for a stationary object this is quite easy). You can then use stereo geometry to solve the depth many such tracker apps exist as part special effects packages and making your own is not entirely out of the question (fun link, fundamental matrix song). The second way is to have a known plane, the plane can be solved by knowing 4 points and their relative locations. In reality it is better use more points for error and lens correction estimates. A third way is to use structured light (a cloud of laser light points, or a line for example) to triangulate the point distances. Or you could use a light field camera.


What is missing in this image that stops it from looking like a photograph? Look at this image:

This is CG, Saya, a project meant to make as real CG as possible to the point of becoming indistinguishable from reality. I would say that it is pretty believable. But, it still doesn't seem real enough, something's missing. And I don't know what it is. And it drives me nuts.

I am not saying that CG has to do reality, I am personaly fond of stylization, but I really want to know why we can't get a 3d image just right yet.

Is it the light? The clothing? The lines above the lip, too defined?

Any insights would be appreciated.

The hair seems to have sampling noise or aliasing in it. This might be poor jpeg compression but it does not look like it.
I've never seen hair part that way (could just be me).
The neck looks a little too regular (I'm no modeler though).
The skin's subsurface scattering falls a bit short. Perhaps it's not that but there is something slightly off about the skin.
The shirt has a slight plastic feel to it.

Most of this is entirely subjective of course. I think it's fairly good but falls short of the state of the art in many areas.


What is the benefit of using Half Edge over Winged Edge? For mesh representation, what is the benefit of using Half Edge over Winged Edge data structure?

I understand both mesh representations, the only difference is that half edge uses directional edge and winged edge uses undirectional edge. So far, I cannot think of what is the usefulness of using directional edge, yet it only gives more memory consumption.

As far as I can tell, the main advantage of half-edge is that traversal can be a bit simpler due to a guarantee of edges having a consistent orientation within each face.

Consider the problem of iterating over all the vertices or edges of a given face, in counterclockwise order. In the half-edge structure, this can be done by starting with an arbitrary half-edge of that face and simply following the "next" pointers until you get back where you started.

In contrast, doing this in a winged-edge structure is a bit annoying, because the edges are oriented arbitrarily; any given edge that you encounter might point either clockwise or counterclockwise relative to the face you're trying to iterate around, so you have to do an extra conditional check at each step to see if you should traverse the current edge forward or backward.

Other kinds of connectivity queries behave similarly: the half-edge version lets you follow links in a consistent sequence while the winged-edge version requires orientation checks at each step.

Whether the conditionals are actually a performance problem for winged-edge will probably depend on other factors. For a "naive" implementation with pointers every which way and data scattered across memory, I'd expect cache miss overhead to swamp any effect of the conditionals. On the other hand, if you have a tightly packed data structure with everything hot in the cache, you might see some effects from the conditionals due to incorrect branch predictions, etc. It's hard to say.

Leaving performance alone, I'd prefer half-edge just because it seems easier to reason about and write correct code for, even if it results in a slight memory overhead.

By the way, there are a couple of other design choices with mesh data structures that often seem to get confused with this one. A commenter mentioned single vs double linking, but naturally you can do either single or double linking with either half-edge or winged-edge. (Although I don't see how single linking with winged-edge would even work, since as mentioned above, you might have to traverse edges either backward or forward in the course of a query.)

Also, there's the question of whether the vertex and face structures store a list of all their edges, or just one (and require you to traverse the edges to find the rest). Having a variable-length list of edges per vertex/face considerably complicates the logic if you want to do it efficiently (i.e. not having a separate heap allocation per vertex/face), but again this is independent of whether the edges are half-edge or winged-edge.


Is long term continuous use of GPGPU safe for my GPU? I'm looking to use my GPU for non-graphical calculations (artificial life simulations) but ideally I would like to leave this running for weeks at a time, 24 hours a day.

Is there anything I should take into account before trying this? Is a GPU subject to overheating or shortening of its lifespan by continuous use?

What about the computer in general (it's a laptop in my case, but I'm interested in the differences in case it's worth me getting a desktop computer for this)? Will running the GPU non-stop place any strain on connected parts of the computer? Are there known problems that result from this?

Running a GPU at full capacity will reduce its lifespan through electromigration; the speed at which the chips are damaged depends on how hot it is.

A desktop computer has enough room that the designer can put in a cooling system to handle the worst-case thermal situation.  Running your GPU at continuous full load won't overheat it too badly, and you can expect it to last several years.

A laptop, on the other hand, is strongly space-constrained.  Typically, the cooling system is designed to handle brief bursts of heat interspersed with long periods of near-idle operation.  You may not be able to run it at full load 24/7: the BIOS or operating system will slow things down to keep from overheating.  In any case, running at full capacity will likely cause things to burn out in a year or less.


can half of a picture be processed and viewed? Lets say i have a picture. I send it to a server. While sending it to the server half way  the upload gets canceled. Now the server has half or some of the picture. Now is there a way in which this can be  processed and viewed. Are there any formats which support this? And are there any other factors which account as to why the picture can or cannot be processed?

Yes, most formats work this way! For example JPEG can be decoded up to the point where the chunks are available:

Image 1 Scaled version of original JPEG picture at (1 048 198 bytes) and corrupted one (610 922 bytes) side by side. Original image by Maruf Mostafa image available here

This is easy for you to test take an image and just simply delete some bytes out of it (like with a text editor). Some files are easy just open them, like the JPEG in question that just opens up in many editors without a hitch.

PNG required some slight special handling to uncompress. It did't open in Photoshop or windows image preview. But was still relatively straightforward to open as the web browser happily opened even partial PNG files to the point it was defined to.


Is sub-texel shading possible? I suspect the answer is no, but thought I'd ask anyway.

It seems that other than using nearest neighbour and bilinear interpolations, there is nothing you can do to determine how two adjacent pixels are interpolated ... such as applying an edge preserving interpolation, or adding some procedural variation.

I'd love to be wrong! If indeed there is no way to do this, might there be a practical solution which involves creating a higher resolution (maybe much higher) texture (like a "mipmap level -2") on the fly?

If done naively I guess it would be very memory-hungry ... perhaps there is some way it could be done to a reusable buffer, or some way ensuring only part of the image would be processed at a time (like an OpenCL/CUDA/Compute Shader workgoup) thus limiting the buffer size. (Perhaps Vulkan could help somehow?)

Obviously it would be done on a LOD basis ... no need to do it on far away textures.

Thanks

The GPU hardware only supports nearest-neighbor, bilinear, trilinear, and anisotropic texture filtering. However, nothing stops you from implementing your own texture filtering in the pixel shader.

To do this, you'd use textureGather to grab a 2×2 block of texels around your sample point. Then you'd manually calculate the position of the sample point relative to the texel centers. Armed with the raw texels and the interpolation position, you can then apply whatever interpolation function you like.

The position calculation would be something like this:

vec2 textureSize; // texture's width and height
vec2 offset = fract(texCoord * textureSize + (-0.5 + 1.0/512.0));

The 1/512 offset in there is needed as a rounding correction on certain GPUs, to ensure that fract()'s output steps from 1 to 0 at the exact same point that textureGather switches texels.

As an example, to re-implement bilinear filtering, you'd do this (assuming a single-channel texture for simplicity):

vec4 texels = textureGather(...);
float result = mix(mix(texels.w, texels.z, offset.x),
                   mix(texels.x, texels.y, offset.x),
                   offset.y);

But in practice, you'd be replacing the above with your own interpolation logic. This will for sure be slower than using hardware bilinear interpolation, but it may be an acceptable cost for your app.

If you need more than 2×2 block of texels, you can also use multiple calls to textureGatherOffset to grab more of the surrounding region, 2×2 at a time.

As for generating a higher-resolution version of the texture on the fly, that can certainly also be done. It would be an extension of texture streaming techniques that load larger mip levels from disk only when needed. Here, you'd be procedurally generating the larger mip levels instead of loading them, but it's a similar idea. Unfortunately I don't have a good reference handy for how to do this. (It's somewhat related to virtual texturing, but less complicated—for just plain texture streaming, you don't need sparse textures or a page table, etc.)

However, there is unfortunately no GPU hardware support for doing the thing you suggested of avoiding the extra storage, by processing the texture only a piece at a time. Doing that in hardware (“texel shaders” that would generate texel values on-demand as they are sampled by fragment shaders) is an idea that has been proposed many times but so far has not happened.

It might be possible to do it in software by having fragment shaders append sample requests into a queue, then switching to a different shader to service those requests, then re-running the fragment shaders with the results...but that would take a lot of work and might not be very fast. It would for sure be a research project.

Unless you have a good way of determining in advance which texels are likely to be needed, it would be a lot more practical to just generate the entire texture, assuming memory and performance budgets allow for it.


Exponential shadow maps sampling with PCF for sampler2DShadow instead of sampler2D I am trying to make use of free hardware PCF sampling with sampler2DShadow and extend it to more samples.

So far I am sampling shadow map (sampler2D) in ESM way and extending this to PCF and it works.
So I have no acne (thanks to ESM) and soft result, but I need a lot of PCF samples:

float shadowMapSize = 512.0;
float c = 5000.0;
float receiver = lightCoords.z;
float shadow = 0.0;
for(int i = 0; i < SAMPLE_NUM; ++i)
{
    vec2 off = (POISSON_SAMPLES[i])/shadowMapSize;
    float occluder = texture(shadowmap, lightCoords.xy+off).r;
    shadow += exp(c*(occluder-receiver));
}
shadow /= SAMPLE_NUM;

shadow = clamp(shadow, 0.0, 1.0);

But I wanted to use sampler2DShadow and get 4 PCF tabs for free and extend this to 16 taps for almost free with only 4 poisson samples like that:

float shadowMapSize = 512.0;
float c = 5000.0;
float receiver = lightCoords.z;
float shadow = 0.0;
for(int i = 0; i < SAMPLE_NUM; ++i)
{
    vec2 off = (POISSON_SAMPLES[i])/shadowMapSize;
    float occluder = texture(shadowmap, vec3(lightCoords.xy+off, receiver)).r;
    shadow += exp(c*(occluder-receiver));
}
shadow /= SAMPLE_NUM;
shadow = clamp(shadow, 0.0, 1.0);

But unfortunately it doesnt work as expected:

Is there a way to make it work (in ESM way, not traditional due to acne) or due to sampler2DShadow specification it is rather impossible? In traditional way it is possible.

PCF does N shadow tests, where each test returns 1 (in light) or 0 (in shadow), and returns the average of the results. sampler2DShadow is a special sampler where PCF is applied to the nearest texels to a sample, based on the depth coordinate provided. So you can't use it with ESM.


Why did tessellation come to be a prominent feature? Tessellation has been touted as one of the major features in newer graphics APIs like DirecX 11, and it is shown as a standalone stage in a modern graphics pipeline.

Compared to the amount of hardware and software attention given to this feature, it doesn't seem to be heavily used in real-time graphics. So this doesn't seem to be a feature that arose out of graphical demand.

Then why did tessellation become such a prominent feature? To cater to the demands of non-realtime rendering? As a side-effect of the increasing shift in GPU architecture as generalized parallel processors in heterogeneous computing? Or is this a forward-thinking feature that will be used in graphics as tessellation-capable GPUs become increasingly common?

The main purpose of tesselation is to increase the resolution of the mesh, while only transferring a small amount of triangle data around. In addition, tessellation allows you to dynamically change the LOD of the mesh, so you can optimize your shader calls.

So, we can pass the GPU, say, 3000 triangles, and have it tesselate it to 300000 triangles. We are essentially trading storage space/bandwidth for compute power. Since GPUs have lots and lots of compute, and memory is limited/slow, this is a pretty good tradeoff.

As for "not being heavily used in real-time graphics". I somewhat disagree. Many AAA games have been using tessellation for a long time. That said, tessellation is hard to get right. Done wrong, tesselation can lead to lots of problems, such as cracking, or over tessellating to sub-pixel triangles, which destroys your fill rate.

Combined, these problems make it difficult to implement good and fast tessellation for your everyday person. Thus, most of the uses of tessellation you see nowadays are in AA or AAA games, game engines, and offline rendering tools.

That said, there is active research going on that is trying to better utilize the tessellation hardware, and make it easier to use. For example: Efficient GPU Rendering of Subdivision Surfaces using Adaptive Quadtrees

In the end, tessellation is a great feature, but is difficult to get right


Drawing a cube a million times I need to build terrain using one cube with dimensions 1x1x1, the coordinates are provided in a .txt file and there are about 11M triplets.

The problem is that using my current code I can only draw about 60k of them, then the browser tab is resetted and a prompt to stop an unresponsive script comes out, so I use too much memory and time to generate them.

Here is the chunk of code I use to draw them:

function generateCubes(data) {
    var cubeGeometry = new THREE.CubeGeometry(cubeSize, cubeSize, cubeSize);
    var material = new THREE.MeshLambertMaterial({color: 0x587058});
    var mesh = new THREE.Mesh(cubeGeometry, material);
    var mergedGeo = new THREE.Geometry();
    var instance;
    var line = data[0].split(';');
    var translateX = line[0], translateY = line[1], translateZ = line[2];
    //var group = new THREE.Object3d();
    for(var i = 0; i < 100000; i++) { // should go to data.length
        line = data[i].split(';');
        //instance = mesh.clone();
        //instance.position.set(line[0] - translateX, line[2] - translateZ, line[1] - translateY);
        //group.add(instance);
        mesh.position.x = Number(line[0]) - translateX;
        mesh.position.y = Math.round(Number(line[2]) - translateZ);
        mesh.position.z = Number(line[1]) - translateY;
        mesh.updateMatrix();
        mergedGeo.merge(instance.geometry, instance.matrix);
    }
    group = new THREE.Mesh(mergedGeo, material);
    scene.add(group);
}

The function is called from a success in an $.ajax call.

The commented parts are used without merged geometry, that way I can draw around 100k of the data.

As suggested in the comments, there's a few ways you might be able to reduce the geometry before merging it into the big mesh and sending it out for drawing.

First of all, you might not want to draw cubes that are hidden behind other cubes. While this is not a trivial thing to check, it becomes easier when we at least try to limit it to cubes that are on the outer surface of the terrain and the cubes are laid out in a regular grid. In this case you first of all make a regular grid storing a boolean flag for each coordinate where there's a cube. Then all you need to do is walk over that array and for each active cell check its neighbours' flags and only add it to the geometry when it doesn't have all 6 face-neighbours active too. You can even optimize this by checking only the 3 neighbours in the direction of the camera.
Another thing you can do is frustum culling, basically ignoring all cubes that are not inside the actual view volume, which amounts to just checking the cubes against the 6 planes of the viewing frustum.
What you also could do is perform basic backface culling yourself, and only adding the geometry of the 3 cube faces that actually face the camera (or better yet, combining it with the first point, the ones facing the camera and not having an active neighbour).

But all this only reduces the amount of geometry that you are generating and sending over for drawing. It does not do anything for reducing the actual computation you do to generate that geometry (rather on the contrary actually). It is not clear what exactly it is that's causing your script to take too long or acquire too many resources but I also don't know anything about Javascript or Ajax or whatever specific web-stuff you're doing.

Maybe it's possible to delegate all this to some kind of background task that doesn't block the main process. This way you might also be able to process the whole file in smaller batches, generating parts of the whole geometry and updating the display whenever you generated another batch.
The above method might help to reduce overall memory consumption if the amount of actually generated geometry is much less than those 11M cubes from the file and if the regular grid isn't too large either. But you actually only need 1 bit for every grid cell, making just 2MB for 256^3 grid and 128 MB for a 1024^3 grid. You might further reduce this by tiling the grid space into smaller sub grids and processing these each. However, this might require you to traverse the terrain file multiple times if that isn't already sorted appropriately.
Which brings us to the possibility of simply restructuring your input file format to better facilitate any of the above methods, like already sorting your cubes into spatial subgrids, or even storing a full boolean grid rather than a compact list of coordinates.

This all really depends on what the bottleneck is here. But it should give you some general ideas of how you could approach this for optimization.


Difference between BVH and Octree/K-d trees So I'm currently reading up on acceleration data structures and so far I've heard there are 2 main schemes. Object Subdivision and Spatial Subdivision.

The Bounding Volume Hierarchy belongs to the Object Subdivision approach and K-d/Octree belong to the latter. However I'm having trouble distinguishing the two of them as both of them seem almost the same.

I had implemented a data structure before for my GPU raytracer which I thought was an Octree but I am not sure anymore. This was according to the paper "Ray tracing Complex Scenes" by Kajiya. I'm gonna quote him directly.

  We could spend additional computer time to construct a better
  hierarchy. A median-cut scheme will help group near objects together
  even if they aren't modeled coherently.

  The previous algorithm attempts to group objects based on their
  nearness to each other, but it does so along one dimension at a time.
  It might do better to group objects in along three dimensions at once
  by employing an octree. This preprocessing step would be very similar
  to the one described in [Glassner 84] or [Kaplan 85]. The difference
  is that, if an object straddles two or more octree cells, it is
  arbitrarily placed in exactly one of them. Furthermore, after the
  preprocessing is completed, the octree is used solely to define the
  hierarchy; each object must still be assigned a distinct bounding
  volume.

So I constructed this structure the same way. First I made an Octree, I inserted spheres into the cells if their center was in that cell. Every time I checked if the leaf nodes consisted of more than 2 spheres. If it did, I had to subdivide the cells again. After the octree was completed, I looped over all the compartments and resized them w.r.t the minimum x,y,z and the maximum x,y,z of the objects it had. This was done to avoid the case where some portion of a sphere would be outside the cell it's currently in (due to it's center lying in that cell).

Now according to the paper the above structure is a BVH? But I'm thinking its an octree? What's the difference?

2) Secondly, so far I have only inserted spheres. What's the case with triangles? What should be the maximum threshold for leaf nodes after which I should subdivide i.e. The number of traingles?

Well I researched quite a lot after that and this paper helped a lot.

"Space Subdivision algorithms" by Macdonald 1988.

So just summing what I understood. Some of them are obvious reasons but after reading the paper make much more sense and I'll try to do the same here.

1) In BVH we are subdividing objects into smaller pieces. So for example a model of dragon consisting of thousands of triangles, We first start with a Bounding Volume covering the whole dragon. Then we subdivide again and again until our conditions are met. In BVHs, the leafnodes usually have 1 object in contrast to octrees which can have many. However this isn't clear whether it's referring to an object (sphere) or a single triangle since spheres can contain hundreds of triangles if triangulated. Here is a picture explaining BVH.

In contrast, Octrees/K-d trees and other space subdivision, divide the space recursively. A simple picture of an octree given in the paper.

2) The above pictures show another difference. That is, in Space subdivisions we have disjoint sets of space regions or voxels or cells whatever you wanna call them. In contrast, BVHs have disjoint sets of objects.

This means in octrees one object may be assigned to more than 1 voxel. And thus we may have to check for that object again and again though this can be avoided using techniques like mailboxing etc.  In BVHs, however, this isn't the case as an object may be inside only 1 bounding volume at a time, thus it may be checked for intersection only once which is when the ray intersects the bounding volume it's currently in.

3) The other difference the paper mentions is that in Space subdivision traversal algorithms usually calculate the exit point of the ray from the current intersected child node and calculate a new point just past this exit point and from this information the next child node is found. The benefit of this is that traversal may stop as soon as it hits an object.

On the other hand, BVHs can't do this since in BVHs we may have object overlapping in nodes. In this case we will have to maintain a list of the intersected objects and find which is the closest one. A good example of this is given in "Raytracing Complex Scenes by Kajiya"

As we can see although the ray first intersects the upper BV, the closest object hit is actually inside the lower BV which the upper one has no information of unlike space subdivision structures.

And now on to the part of how I implemented the structure, it seems what Kajiya suggested was to use the octree scheme to define a traversing algorithm for the BVH. In the end when I recalculated the bounding volumes of each cells, this relaxed many voxels boundaries causing several to overlap. After this operation the structure didn't resemble "Disjoint sets of spaces" rather it resembled "disjoint sets of objects" hence it's essentially a BVH.

Do note that this operation may sound like somewhat an efficient optimization to the space subdivision structure, that is relaxing the bounds of voxels to avoid having all the voxels keep track of the object that overlaps them. This way we assign 1 object to 1 voxel and relax it bounds.

But doing so we lose the traversing efficiency of space subdivision structure where we could have stopped after intersecting only 1 object. Other pros may also have been lost. That's what I have understood so far.

If anybody would like to add some more feel free to do so.


Can I determine the pixel geometry programmatically? Different screens can have different pixel geometry, so that the red, green and blue components are arranged in different patterns. Using sub-pixel rendering to give a higher apparent resolution is only possible if the pixel geometry is known (what will give an improvement in clarity on one type of monitor will make things worse on another).

This is particularly relevant if an application needs to run on both a desktop/laptop and a mobile screen, as different pixel geometry is quite common in mobile screens.

Is there a way to determine which geometry the screen uses at runtime, without having to ask the user? I'm interested in whether this is possible in general, but ideally I'd like to know whether this is possible when using JavaScript with WebGL.

It appears that Microsoft has punted on this in Windows 7:

This is the method available in the control panel for selecting what layout ClearType uses.

Additionally, it seems that iOS and the Windows modern UI style de-emphasize subpixel antialiasing heavily, due to the prevalence of animations and screen rotations. As a result I expect the OS vendors to not spend a lot of effort trying to figure out the subpixel layout of every screen.


Parametric line intersection test I have the following question in an exam review and I am looking for feedback about how to approach the solution. Any advice will be highly appreciated.

  "Using a parametric line intersection test, give the parameter values of the endpoints of a line from (3,3,4) to (11,8,7) clipped to the cube extending from 0 to 5 in x, y, and z"

Hint:

The three coordinates can be handled independently.

For instance in $x$, you need to solve

$$x_0 + t (x_1-x_0) = 0$$
 and
$$x_0 + t (x_1-x_0) = x_{max}$$


Texture rendering in OpenGL I know there are tons of questions about it, but I found nothing useful so far so, here I am.

I'm trying to render a texture which is attached to a wavefront .obj mesh. I continued to fail (somehow, it was like textures were not correctly loaded), so I wrote a simple example code to see where I was wrong. The sample code follows, and the texture is "correctly" loaded and rendered. However, there are some problems in the render itself: the image is incomplete / blurry / repeated / distorted. I will also post the expected result and the actual result.

Working on Linux Ubuntu 14.04, OpenGL 3.0 Mesa 10.5.9.
Any idea of what is happening?

Texture (copyright Angryfly @ Turbosquid) and result on the right:

Sample code:

#include <math.h>
#include <GL/gl.h>
#include <GL/glu.h>
#include <SDL2/SDL.h>
#include <SDL2/SDL_image.h>
#include<iostream>

int main(int argc, char* argv[]) {
    SDL_Init( SDL_INIT_VIDEO );
    SDL_GL_SetAttribute( SDL_GL_DEPTH_SIZE, 16 );
    SDL_GL_SetAttribute( SDL_GL_DOUBLEBUFFER, 1 );
    // prepare the window
    SDL_Window *window = SDL_CreateWindow(argv[0], 0, 0,
            800, 600,
            SDL_WINDOW_OPENGL|SDL_WINDOW_RESIZABLE);
    SDL_GLContext context = SDL_GL_CreateContext(window);
    // enable textures and bind a name (say 50)
    glEnable(GL_TEXTURE_2D);
    glBindTexture(GL_TEXTURE_2D, 50);

    glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,
                     GL_LINEAR_MIPMAP_NEAREST );
    glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

    glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT );
    glTexParameterf( GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT );
    // load image with SDL
    SDL_Surface *s = IMG_Load("texture.png");
    // the following doesn't work (white space)
    //glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, s->w, s->h, 0, GL_RGB, GL_UNSIGNED_BYTE, s->pixels);
    gluBuild2DMipmaps(GL_TEXTURE_2D, GL_RGB, s->w, s->h, GL_RGB, GL_UNSIGNED_BYTE, s->pixels);

    glViewport(0, 0, 800, 600);
    glClearColor(1,1,1,1);
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    gluPerspective(70, 0.75, 0.2, 1000);

    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();
    glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT );
    gluLookAt(0, 10, 0, 0, 0, 0, 1, 0, 0);

    glActiveTexture(50);
    // display texture as 2 triangles.
    glBegin(GL_TRIANGLES);
    glTexCoord2d(0,0); glVertex3d(-5, 0, -5);
    glTexCoord2d(0,1); glVertex3d(-5, 0, 5);
    glTexCoord2d(1,0); glVertex3d(5, 0, -5);

    glTexCoord2d(0,1); glVertex3d(-5, 0, 5);
    glTexCoord2d(1,0); glVertex3d(5, 0, -5);
    glTexCoord2d(1,1); glVertex3d(5, 0, 5);

    glEnd();
    glFinish();
    SDL_GL_SwapWindow(window);
    // wait for quit procedure
    SDL_Quit();
    return 0;

}

This type of artifact is a tell-tale sign that you've messed up your texture image format at the byte level. It's hard to tell what exactly is wrong, but it's something along the lines of bad pitch or a component layout mismatch.

Have a read of this wiki page and try again. glTexImage2D should work, keep trying until it does.


Methods for grid traversal in a glsl pixel shader? I'm working on a shadertoy "snake" game, using the new multi pass rendering abilities to save game state between frames.

I'm using raytracing to render the board (an AABB), and am planning on using spheres to render sections of the snake's body.

The game board is a 16x16 grid and each grid can either have a sphere there (a segment of the snake's body) or not.  Snake body segments don't move, they are just either there on the grid or not.  When the snake moves, a new sphere appears in the front and an old sphere disappears from the back.

The problem I'm trying to solve is how to render the snake body spheres.

For instance, a naive approach would be to store a 16x16 grid in pixels specifying whether there was a snake body in that grid cell or not.

I would then do a ray vs sphere check for up to 256 different spheres within my pixel shader, which seems like a no go.

Another method might be to figure out where the ray begins and ends on the game board (when it's between the high and low height values of where the spheres are) and then use something like bressenham line algorithm to go from the start to the end of the line the ray takes on the board, and check only the grid cells that the ray hits.

The problem there is that it requires a dynamic loop.

Maybe a more practical solution would be to make the camera have a nearly top down view and where the ray enters the playable game world, test any sphere in the cell it hits as well as the 8 neighboring cells.

I'm betting there are some much better solutions that I'm not thinking of.

Does anyone know of any interesting techniques or creative solutions?

Thanks!

Edit: here is an older version of this game i made, which was CPU / software rendered, to give an idea of what I'm planning.

why not building a bounding box (or spheres) hierarchy ?

(but for a shadertoy implementation, the lack of dynamic loop length might spoil the gain ).


How to triangulate a simple 2d surface to generate a constrained surface trimesh I want to split a couple of simple 2d forms into tri meshes. The meshes should conserve the original geometry and ideally I'd like a possibility to influence refinement vs. tris count.

I have the 2d forms available as images (svg, png) and am happy to convert to something else if necessary (STL, ...?). Output should ideally be an IBO (index buffer containing points and triangle index numbers), but I will settle for something I can convert into that. I don't need a library, a manual process is fine, but I'm not above using one, if it gets the job done.

I found a number of possible solutions to this (gmsh, netgen, ... and more) but many of those seem too much (would require high time investment to learn) for my relatively simple problem.

I am moderately proficient in python, blender and common linux software and am happy to come up with a small script to solve this, but don't want to dive deep into e.g. GL in order to solve this - hence my question: What is the easiest workflow to generate a trimesh from a svg or png?

This feels like a "common" issue, but I'm having a bit of hard time to find a solution with good effort/result ratio (maybe my google-fu is broken).

The fastest and easiest way to do this is using blender. Simply import the svg (supported out of the box) into blender. The svg will be imported as a "curve"-type geometry. Optionally do some clean up/refinement. The curves can then be converted to a mesh. The mesh defaults to a trimesh. Voila - svg to trimesh in a couple of seconds.

Mesh refinement is not very fine-grained, but there is some measure of control via the "beauty-fill", "subdivide" and "triangulate" as well as "merge vertices by distance" tools from within blender.

The IBO (vertices + triangle vertex indices) can then be extracted using a short python script:

import bpy
obdata = bpy.context.object.data # selected object
for v in obdata.vertices:
    fh.write('({}, {}),\n'.format(v.co.x, v.co.y))
for f in obdata.polygons:
    for v in f.vertices:
        print('{} '.format(v))


Where is the best place for Tangent-bitangent calculation, in shader or in C/CPP code? I have two methods of calculating tangent and cotengent (needed for normalMap lighting calculation).

The one is doing it from CPP code (with assimp library for example)
The second is doing it directly in shader (in Vertex shader/in fragment shader)

What is the best optimized method? (optimized for speed, with vsync activated)

I use Opengl 3.3+

Doing it in the CPU side during initialization is what I'd go for, this is assuming you are initialising that data once and passing it to the GPU.

On the GPU side, doing the calculations per fragment would be too costly and should be out of the question, unless it's really needed for a special effect then you shouldn't add unneeded ALU work to the fragment shader, this will only cause your application to slow down. On the vertex shader is more acceptable but again if you have highly tessellated geometry, it'll be expensive.

Whenever there is data that you can compute just once, then go for that instead of doing it every frame. Even if you were planning on doing it just once on the GPU and using transform feedback or an FBO to store the data, I think this approach would be more costly. As always though, profile your application, careful tradeoffs are specific to architectures.


Confused about z-axis behaviour I'm a newcomer to OpenGL and I was playing around with drawing triangles with different z-coordinates. From what I understand, the z axis point out of the screen, and the -z axis points into the screen.

When I draw a square with 3 corners at a 0.0 z-coordinate, and the last corner at, say, 3.0 z-coordinate, I get this:

I don't understand how it's making this shape... I thought it would be something like this, since the 4th vertex is just 'far away'.

Can someone explain?

Edit: This is my vertex data

// vertex array
float vertices[] = {
    -0.5f,  0.5f,  0.0f, 1.0f, 0.0f, 0.0f, // top left, first 3 are location, last 3 are color
     0.5f,  0.5f,  0.0f, 0.0f, 1.0f, 0.0f, // top right
    -0.5f, -0.5f,  -2.0f, 0.0f, 0.0f, 1.0f, // bottom left
     0.5f, -0.5f,  0.0f, 1.0f, 1.0f, 1.0f // bottom right
};

// element buffer array
GLuint elements[] = {
    0, 1, 2,
    2, 1, 3
};

And I am calling the draw like:

glDrawElements(GL_TRIANGLES, 6,GL_UNSIGNED_INT,0);

The post-projective Z range in OpenGL extends between −1 and 1. So, a Z-value of 2 is out of range and will be clipped by the far plane.

The reason the square appears with a quadrant missing is that it's split into triangles from bottom left to top right, and so each triangle has two vertices with Z = 0 and one vertex with Z = 2. So, interpolating along the triangle, it hits Z = 1 (the far plane) halfway across, and everything beyond that is clipped. This results in half of each triangle disappearing.

BTW, you'll need a perspective projection matrix in order for far-away objects to appear smaller on screen. With no projection matrix, you're effectively using a parallel projection.


Why would I specify image format in a shader layout qualifier? The OpenGL wiki entry for layout qualifiers says:

  Image uniform variables have qualifiers that define the format that all reading operations will convert the data into...

When is it a good idea to declare the image format in a shader?

i.e. layout (binding = 0, r32f) readonly uniform image2D input_image

Wouldn't this shader work just as well if I did not specify the r32f?

From the same wiki

  Image variables can be declared with a format qualifier; this
  specifies the format for any read operations done on the image.
  Therefore, a format qualifier is required if you do not declare the
  variable with the writeonly memory qualifier. Write-only variables
  cannot be used as in any reading operations; this includes calling
  load and atomic (read/modify/write) functions. So if you want to read
  from an image, you must declare the format.

  The format defines how the shader interprets the bits of data that it
  reads from the image. It also defines how it converts the data passed
  for write operations when it writes it into the image. This allows the
  actual Image Format of the image to differ between what the shader
  sees and what is stored in the image, sometimes substantially.


How to choose which reflection probe to use? I have multiple reflection cube map that's prebaked before a scene.

However I am being confused as to which one to choose.

I am told I need to choose "closest cube map" then use it.

However it is unclear to me that whether I should do that based off of per pixel or per object(object's position).

I looked at Unity to get some ideas as well.

Unity chooses a reflection probe based off of distance and area probe affects the object.

However there is no clear "do this". So I am tentative and left unsure.

Are you trying to make your own rendering engine?

Here is a solution that prioritizes visual quality ; it is slightly more complex than the object-based solution you are referring to.
For better visual quality your shading should be on a per-pixel basis.
(If you highly prioritize performance, or do not require detailed reflections, you should probably stick to the object-based solution though).

If you are doing deferred rendering, instead of selecting only a single cubemap for each pixel, you can use a formula that combines all the cubemaps that are close enough to a pixel.

A trick sometimes used to achieve this is to add a pass that computes the number of cubemaps that affect a single pixel (so you can do averages in the final shading pass).
If you are doing forward rendering for transparent objects for example, I think it might be a bit trickier to compute the average of cubemaps.
That's because you need to have pixel positions already available before shading ; you might think it could be possible to do it if you had a kind of depth pre-pass buffer for that purpose, however for transparent objects you usually do not want to write their depth in that buffer, because this leads to artifacts when transparent objects are overlapping.
You would therefore need to have a rather complex logic to handle those cases...

However even with forward rendering you can still select the closest cubemap on a per-pixel basis, although without the "averaging" formula the transitions between the boundaries of two cubemap influence zones will be very visible and reflections will not look realistic.


GLSL function parameters In GLSL: Common Mistakes it states:

  Functions parameters must be declared with the in, out, or inout qualifiers.

but I have never declared function parameters with in, out, or inout qualifiers and I have never run into any problems. Could this be because of my specific graphics card? What are the side effects of not using the in, out, or inout qualifiers?

Info: I am using GLSL 3.3 and I have an Intel graphics card

The GLSL Specification in section 6.1.1 "Function Calling Conventions" states:

The keyword in is used as a qualifier to denote a parameter is to be copied in, but not copied out.

The keyword out is used as a qualifier to denote a parameter is to be copied out, but not copied in. This should be used whenever possible to avoid unnecessarily copying parameters in.

The keyword inout is used as a qualifier to denote the parameter is to be both copied in and copied out. It means the same thing as specifying both in and out.

A function parameter declared with no such qualifier means the same thing as specifying in.


The constructor of hyperboloid in pbrt-v3 The book omitted the explanation of the constructor of hyperboloid.

Could someone give some explanations about:

Why the arguments are two points?
What are the meanings of ah and ch?
Why is there a while loop?

They must define it as a surface of revolution of a line segment.
From the comments, they are probably the a and c coefficients of the canonical hyperboloid equation. With a surface of revolution, a == b so no need to store b separately.
It's clearly related to figuring out ah and ch from the line segment. I can't say more precisely without working out the math involved. I'll leave that to someone with more free time.


Best practices for VRAM management in dynamic environments (e.g. games) While making a GPU Marching Cubes implementation a while back, I ran into the issue of OpenGL holding on to memory I had asked it to release. There are a lot of threads on this over at Stack Overflow, and the gist seems to be that you simply can't rely on the API to release VRAM  in a timely manner. In my app, I was trying to be smart, releasing all temp buffers used in mesh creation once I was done with them, and allocating an exact-sized vertex buffer each time the mesh changed. This was counterproductive, and rapidly led to out-of-memory crashes.

In my case, since it was just a toy app for a uni project, I solved it by allocating worst-case sized buffers and leaving them allocated, recycling buffer space wherever possible for different passes during the mesh extraction process. This was also a bit of an edge-case, since buffers were pretty huge (sometimes hundreds of MB) so out of memory errors appeared quickly.

This is fine for a strictly limited use case like mine, but my question is, how is VRAM handled in environments like games, where large numbers of meshes and textures need to be loaded and unloaded dynamically? I'm thinking particularly of open-world games like GTA where the world is constantly being streamed-in as the player moves around, but even in more linear games, loading a new level would need a lot of unloading and reloading assets.

Are assets generally loaded to sections of large fixed buffers and space usage tracked somehow? Or if using a large number of smallish buffers, can the APIs keep up with a few of them being created and destroyed regularly?
Can anyone with experience in engine design shed some light on how this is done in real world usage?

For consoles (and D3D12/Metal/Vulkan), you know everything you need to know to develop an allocation strategy. You know:

How much memory, of each kind, is available.
How big each kind of resource will be. Textures of a particular format and size take up X bytes, require Y alignment, etc.

Given such information, you can devise an allocation strategy for data uploading. Open-world games use streaming of fixed-sized blocks of data. Each sector of the game contains resources that take up X number of bytes, and therefore, artists have X number of bytes of storage to work with. It's their job to fit all of the stuff needed for that region of the game into that much space. There will also be stream blocks for larger collections of regions, or other data that gets shared between all game regions.

For other games, some of them load everything at level-load time. Some do hard-loads in the middle of "levels" (see many Source-engine games). Others stream things more dynamically, but even there, artists are restricted to using resources that total up to X number of bytes.

On consoles, such sizes can be known a priori, but with Vulkan/Metal/D3D12, these have to be queried for a particular piece of hardware. So in the latter case, you have to be more reactive to what's available, but all of the information is available to the user.

For other APIs/platforms (OpenGL/D3D pre-12), things are rather more complicated. Consoles and the lower-level APIs give you direct access to memory allocations, which gives you more flexibility. You can use a piece of memory for vertex data at one point, then put an image into it later on. Memory is divorced from its current use, so it's much easier for an application to choose how to recycle memory.

For GL/D3D-pre-12, things are difficult. A buffer object is a buffer object, and the memory currently associated with it cannot later be used as a texture. Well, it can, but you cannot explicitly ask that to be done. The most you can do is deallocate the buffer, allocate the image, and hope that the buffer's memory was recycled.

For cases where you're doing dynamic loading in fixed-sized blocks, you basically have to give your artists more limitations. Instead of limiting the artists by memory, you have to spell out the limitations by object So you'd say that a stream block can have "X numbers of textures of Y formats and Z size, and W bytes of vertex data". Artists can't sacrifice 4 textures to allow one texture to be 4 times the size, or sacrifice a texture to get more vertex data, or sacrifice some vertex data to get more texture space.

That is what you have to do if you don't trust the implementation's memory allocator to effectively recycle resources. For hard-load cases, it's generally OK to just delete all of the objects and expect the implementation to recycle effectively. But even here, it's a good idea to make sure that you don't try to allocate all of the available storage; leave the implementation a healthy percentage for fragmentation and so forth.

Of course, these kinds of things are a big part of the reason why IHVs have application-specific optimizations: they know up-front which resources were allocated for which purpose, and the driver makers can design allocation schemes that work with these assumptions.

Which is why Vulkan et. al exist: so that application developers don't have to rely on IHVs to work around problems in the API.


What Are The Default Values of `glReadBuffer/glDrawBuffer(s)` for an FBO? glReadBuffer(...)/glDrawBuffer(...)/glDrawBuffers(...) access part of a framebuffer's state.

For reading the default framebuffer, this is GL_FRONT or GL_BACK.  However, the documentation doesn't say what it is for FBOs.  The only thing I found was an offhand comment here stating that the default is GL_COLOR_ATTACHMENT0 (which is what I would guess).  Can I rely on that?

For writing, I didn't immediately find any claims what what color buffers are enabled.

You can usually track down the answer for such questions in the OpenGL specification.  Assuming the value you care about is consistent across OpenGL versions (often the case for initial values, seldom the case for minimum values).

In this case, in the GL 4.5 specification, check out the following:

glReadBuffer(...) initial value: (default framebuffer: GL_FRONT iff single-buffered, GL_BACK iff double-buffered) (FBO: GL_COLOR_ATTACHMENT0) (see page 506)
glDrawBuffer(...) initial value: (see glDrawBuffers(...))
glDrawBuffers(...) initial value: (default framebuffer: fragment color 0 goes to GL_FRONT iff single-buffered, GL_BACK iff double-buffered) (FBO: fragment color 0 goes to GL_COLOR_ATTACHMENT0) (both: fragment colors other than 0 go to GL_NONE) (see page 497)


Choosing corner values for trilinear interpolation I'm caching distances for SDF functions in volume textures for faster rendering, and I'd like to use trilinear interpolation for smoother edges. I know that I'll need distance values at each corner; can I blend adjacent voxels, or will I need to store eight separate distances in each sample?

As a rule, SDFs do much better under interpolation than regular images do—it’s why you can mostly get away with using tiny textures for e.g. font data. You should be fine interpolating neighboring voxels.


Trouble making path tracer multithreaded, read access violation I'm trying to make the path tracer shown in the Ray Tracing in One Weekend series multithreaded.

Here's the starting of my main function:

//scene setup
    hitable* world = cornell_box();
    auto start = std::chrono::high_resolution_clock::now();
    light_hitable_list = hitable_list(lightsVector);
    string filePath = "img.png";

    Image image(nx, ny);
    TilePool tilePool(nx, ny, 50);

    //maxThreads = std::thread::hardware_concurrency();
    maxThreads = 1;
    std::vector<std::thread> renderThreads;
    renderThreads.resize(maxThreads);

    //RenderSettings renderSettings{world, &image, &tilePool, ns, BLACK_SKY, false, "img.png"};

    for (unsigned int i = 0; i < maxThreads; i++)
    {
        renderThreads[i] = std::thread(&renderScene, world, &image, &tilePool);
    }
    for (unsigned int i = 0; i < maxThreads; i++)
    {
        if (renderThreads[i].joinable())
        {
            renderThreads[i].join();
        }
    }

And here's the renderScene function:

void renderScene(hitable* world, Image* image, TilePool* tilePool) {

    while (tilePool->getPoolSize() > 0)
    {
        Tile tile = tilePool->getNextTile();

        for (int j = tile.yMax - 1; j >= tile.yMin; j--) {

            for (int i = tile.xMin; i < tile.xMax; i++) {
                vec3 col(0, 0, 0);
                for (int s = 0; s < ns; s++) {
                    float u = (float(i) + random_number()) / float(nx);
                    float v = (float(j) + random_number()) / float(ny);

                    ray r = cam.get_ray(u, v);
                    vec3 p = r.point_at_parameter(2.0);
                    col += de_nan(color(r, world, &light_hitable_list, 0));
                }
                col /= float(ns);
                col = vec3(sqrt(col[0]), sqrt(col[1]), sqrt(col[2]));

                (*image)[3 * i + 3 * nx * (ny - 1 - j)] = col[0];
                (*image)[3 * i + 3 * nx * (ny - 1 - j) + 1] = col[1];
                (*image)[3 * i + 3 * nx * (ny - 1 - j) + 2] = col[2];

            }

        }
    }
}

If I run the code above as it is I get this error:

  Exception thrown at 0x00007FF70F601F2E in raytracer.exe: 0xC0000005:
  Access violation reading location 0x000001C200000190.

here: https://github.com/KarthikRIyer/RayTracer/blob/ec51999070218839e6aef2a65868e57fc6bb728d/src/objects/hitables/hitable_list.h#L29

And instead of using just 1 thread if I use maxThreads = std::thread::hardware_concurrency();

I get this error:

  Exception thrown at 0x00007FF79DE01FCE in raytracer.exe: 0xC0000005:
  Access violation reading location 0x0000015E0000012C.

here: https://github.com/KarthikRIyer/RayTracer/blob/ec51999070218839e6aef2a65868e57fc6bb728d/src/objects/hitables/hitable_list.h#L38

These errors are not for the parameters passed into the function. I have checked if they are NULL. It seems to be for the function call.
I'm completely new to multithreaded programming. But it doesn't feel like the error might be due to multithreading because I get such an error even while using maxThreads = 1.

Instead of the cornell box scene if I use the model_scene defined here: https://github.com/KarthikRIyer/RayTracer/blob/ec51999070218839e6aef2a65868e57fc6bb728d/src/Source.cpp#L135 sometimes I get the same error as above and sometimes it randomly starts working even with more than one threads.

Can someone please shed light as to why this might be occuring and what points should I keep in mind while writing a multithreaded path tracer?

I found out what was wrong. It wasn't related to multithreading at all.
I was taking in a vector in a constructor hitable_list(std::vector<hitable*> listVector) and using a pointer to the first element of the vector elsewhere. So as soon as the vector went out of scope the pointer became invalid.

To solve this issue I just removed this constructor and whenever I need to pass in a vector I make sure that the vector stays in scope always and directly pass in the pointer to the first element of the vector, and it's size.


How do you apply a normal map to a 3d mesh? I'm writing my own mesh renderer, and I previously was able to apply a normal map to a sphere, so I understand the basic process of applying a normal map. But my understanding is that in order to apply it, you need a consistent tangent space at every point on the mesh. It's possible on a sphere because you can calculate the spherical coordinates and then pick orthogonal tangents parallel to lines of longitude/latitude, but as far as I can tell, there's no clear way to do that to a mesh.
To give a specific example, I found this free .obj file online, and I can render it with textures, just without normal mapping. But I can open up the model in the generic Windows "3D Viewer", and in that program, it has normal mapping applied, so clearly it's possible. I couldn't find anything online, is there a way to do this?

There are numerous approaches to setting up tangent bases on a mesh, and unfortunately, no totally universal standard for how they are calculated.
Tangents are based on the mesh's UV mapping, so that the tangent vector points (at least roughly) along the U axis in texture space, and the bitangent along the V axis. This means each triangle in a mesh has its own tangent basis (as long as the UV mapping is nondegenerate). This often gets converted into a smoothed tangent basis per vertex, much like you can create smoothed normals by setting each vertex normal to an average over the normals of the surrounding triangles. Different tools and engines might do the smoothing in different ways, might handle UV seams and mirroring differently, might orthonormalize the basis per vertex or per pixel or not at all, etc. Here is a nice article to read if you want to know more. And as noted there, there is a method for tangent spaces called mikktspace that is gradually gaining widespread support and might become a de facto standard.
Object/character models like the one you linked usually have normal maps baked out from a content creation tool (e.g. based on an ultra-high-detail source mesh); ideally, you need the tangent basis you use in your renderer to match the tangent basis used in the tool to do the bake. Sometimes there will be vertex tangents stored in the mesh, but the .obj format doesn't support tangents so you would need to see if the model has tangents in one of the other formats, like .fbx.


Wrapping normals around a sphere I'm using a vertex shader to effectively wrap vertices on a sphere. Starting with world_position already after multiplying the vertex by the model transform matrix to get the coordinates in world space, I calculate the new position via an inverse Mercator projection:

float theta = 2.0 * PI * world_position.x / world_radius;
float phi = 2.0 * atan(exp(world_position.y / world_radius)) - PI / 2.0;
float radius = world_position.z

gl_Position = vec4(radius * vec3(sin(theta)*cos(phi), sin(phi), cos(theta)*cos(phi)), 1.0);

What I now need to do is take these angles, $\theta$ and $\varphi$, and use them to transform the normals as well. I've tried creating a rotation matrix to represent rotation first around the Y axis (for $\theta$) and then around the X axis (for $\varphi$), and multiplying that by the vertex's normal (after multiplying the normal by the model matrix's inverse transpose), but that didn't give the results I expected.

How do I use the angles to rotate the normals so they match the rotations of the vertices?

Turns out I needed to be looking at the former normal as a unit vector $(\hat{r}, \hat{\theta},\hat{\varphi})$, and then use a rotation matrix as outlined here to find $(\hat{x},\hat{y},\hat{z})$.


Volume Rendering - Dataset not rendered correctly So Im trying to implement a Volume Renderer mainly for medical images like CT-scans and MRI images using GPU raymarching. I'm using the book "Real Time Volume Graphics" and this tutorial as a refernce.

I'm trying to render only grayscale images as a start. I understand that the pvm2raw part gives us an 8-bit raw file so each voxel has the isovalue in the range [0-255]. So I decided map this single value on all color channels rgba when taking a sample.

I am taking the datasets from here. You can see my Renderer's result in this video (check the end of my post and sorry for not providing a direct link as I already uploaded on my reddit post)

As you can see the lobster dataset stretches when viewed from the side. My first guess was that I wasn't accounting for the aspect ratio correctly. I have tried scaling the Bounding box to account for the aspect ratio, then adjusting the stepsize according to that. I've also tried to ignore aspect ratio altogether. Both give similar images.

I tried rendering more datasets like "Knee", "Bruce's head", "DTI Scan", etc but those didn't even gave an image. There was just random noise everywhere. Here's an example of "Bruce's head"

This happens with both uniform and non-uniform datasets. If some datasets are getting rendered correctly like the "Foot" in my video, this should mean it's more a with that specific dataset itself rather the core algorithm. Nonetheless the link to the code is provided If anybody can point out some mistakes I'd be grateful.

The link to the Compute shader is here. Main functions are rayMarchVolume. I didn't scale the root bounding box to the dimensions of the volume, it's normalized (0-1), accounted for the aspect ratio (0, ar) and adjusted so it's in the center (-ar/2, ar/2). The function cartesianToTextureCoord(vec4 point) accounts for this fact and returns a texture coord in the range 0-1

EDIT:- If you are hesitating to post an answer here since it's only a suggestion, you can post on Reddit, then when the problem is solved you can post your answer here

UPDATE:- I've provided a release to this project on github. If anybody is willing to help me with this, he can download the release and run the program. Since the algorithm is based entirely inside the compute shader you can change the file easily and test your findings on the spot. Thanks.

Sorry to post such a silly solution but apparently the problem was reading 16 bit uint data into 8bit uint array. This was the reason for inexplainable noise, interleaving and other artifacts.

After I explored more about what the medical datasets actually represent and what flavors they come in (8,16,32bit) I managed to solve the problem.


Software to compose separate models into an articulated model? I've inherited a system which animates a humanoid avatar by loading various models and manually calculating transformations for each one. I'm in the process of porting part of this system to Three.js, and rather than port the manual calculations across I'd prefer to pre-compose an articulated model and let the engine handle the transformations.

Is there any software which would let me import the existing models, define the bone hierarchy and the model which corresponds to each bone, and export a single articulated model in T-pose?

For what it's worth, I have access to both Windows and Linux, and if really necessary I can probably obtain access to OS X. The original models are in FBX format, but I have managed to export them to Three.js files, so I could import from either format.

Is there any software which would let me import the existing models,
  define the bone hierarchy and the model which corresponds to each
  bone, and export a single articulated model in T-pose?

I found that it is relatively easy to create articulations using Blender.
This tutorial might be helpful: Blender 3D: Noob to Pro/Bones | Wikibooks


How are obj files textured? I am a new learner in 3d graphics. So forgive me if I seem to ask something very basic. I am confused how obj files are textured. So far I have seen:

obj + jpg
obj + mtl + jpg

When I look online, most say that they are textured by UV mapping. But when I look into the files, I find no specification of the UV mappings, i.e. how to map a pixel in the jpg file to a vertex (or maybe face?) in the obj file? I suppose there is some rule that is used by default. But I really need to know what it is.
Any reference, such as official documents, blogs, textbooks, online learning materials would be of great help. Thanks!

Update
I came across an example here, called "texture mapped square"
# A 2 x 2 square mapped with a 1 x 1 square
# texture stretched to fit the square exactly.
mtllib master.mtl
v  0.000000 2.000000 0.000000
v  0.000000 0.000000 0.000000
v  2.000000 0.000000 0.000000
v  2.000000 2.000000 0.000000
vt 0.000000 1.000000 0.000000
vt 0.000000 0.000000 0.000000
vt 1.000000 0.000000 0.000000
vt 1.000000 1.000000 0.000000
# 4 vertices
usemtl wood
# The first number is the point,
# then the slash,
# and the second is the texture point
f 1/1 2/2 3/3 4/4
# 1 element

I assume that the coordinates following vt commands are the UV coordinates of the corresponding pixels in the texture image. Exactly what do the values mean? Are they normalized coordinates, that is, is the image normalized to have a coordinate system [0,1]x[0,1]?

OBJ files typically contain vertex positions, texture coordinates (also known as UVs), and normals, as well as face data. There are other kinds of things they can contain but those are the most common (see here for a fuller specification).
Vertex positions are specified with the v command, texture coordinates with vt, and normals with vn. The faces then reference those values by index using something like f 1/2/3 4/5/6 7/8/9, where each grouping of values represents a vertex on the face, and the numbers separated by slashes are the position/texture/normal indices.
The texture coordinates are the part that tell you what point in the texture image (jpg) goes with that vertex. If you've downloaded or exported a model that has a texture mapping, you should be able to see those vt commands in the obj file.
If a model doesn't have texture coordinates, though, then it most likely needs the help of a human 3D modeler. It is not easy to create usable texture coordinates for a model automatically (this is a big research topic). 3D modeling applications have various tools to help the modeler set up a texture layout, but it still usually needs some hand-tweaking to work well.


Why can't I utilize multiple CPU cores with OpenGL like Vulkan? Can I pass the data to the GPU through multiple cores with OpenGL?

Because the fundamental foundation of OpenGL makes multi-CPU-core submission impossible.

OpenGL, at its core, is a synchronous API. Every OpenGL command behaves "as if" it were executed and completed immediately. Sure, glFlush and glFinish are provided, so that implementations can have some asynchronous execution of commands. But these are essentially fig leaves over the definition of OpenGL as synchronous.

You can attach a texture to an FBO, bind that FBO, render to it, unbind that FBO, bind the texture as a source, and read from it. You can do all of this without any thought as to the fact that you've issued two sets of commands where there can be zero overlap between them. This is because OpenGL implementations are required to issue whatever synchronization mechanisms are needed to make this work. You as the user can be completely oblivious to this.1

Because of that, consider what would happen if you could issue that render-to-texture in one thread and the read-from-texture in another. How exactly would that even work? You would need some way to say "don't execute the read until the write is done". But since the two sets of commands are on different threads, how do you say that?

OpenGL has no mechanism for doing so; the closest it gets would be some form of fence sync. And even that would require that the render-to-texture thread has already finished sending those commands, since glFenceSync issues the fence actual fence (you cannot create a fence that has not yet been submitted). Which means that the render-to-texture thread may as well send the read-from-texture commands too.

To make OpenGL work asynchronously means making lots of adjustments to its memory model. You'll have to add lots of commands for making things visible to other threads. For allowing thread A to not have its commands execute until thread B is finished. You'd need to make command buffers and queues first-class constructs of the API. And so forth.

And you have to do it in a way that's backwards compatible with the existing memory model. Unless you want to go through another core/compatibility split again.

NV_command_list shows the challenges of trying to build this sort of thing into OpenGL. They force you to use ARB_bindless_texture. They had to invent bindless buffers, so that you can provide vertex arrays, UBOs, and SSBOs.

And even then, it's not something that AMD could even implement. Why? Because of image layouts. NVIDIA's hardware has fewer layouts and handles transitions automatically. AMD's hardware requires explicit control of layouts. Once you break the synchronous memory model, that implementation detail of what layout a texture is in becomes really important. You have to have control over it. You have to explicitly state it in various API, and you have to know to change it when you use a texture in a different way.

This is also almost certainly why AMD doesn't implement D3D11 deferred contexts.

Or you can do what Khronos did: make an API that naturally handles all of this stuff. A clean break is ultimately best for everyone. It allows us to jettison other garbage that OpenGL had, so that we can have a nice, clean, lower-level API.

1: While you can be oblivious, obviously you shouldn't be oblivious if you care about things like performance. You should put some work between the render-to-texture and the read from that texture, if at all possible. That's one of the biggest problems with OpenGL's synchronous-by-default API: it makes it easy to do things the slow way, since the API itself doesn't force you to issue that synchronization. In Vulkan, you cannot be ignorant of that synchronization. Or at least, not without your code invoking UB.


How can I remove aliasing in an image without introducing blur? I am trying to reduce aliasing in an image using some filters. I tried the Gaussian variant of filters to remove the said high frequency patterns in the image, but I feel it's a bit too much of a blur.

I need a filter that can help me do some kind of anti-aliasing without doing too much blur.

I have searched a lot and unfortunately the results are not fruitful.

Depending on the specific content of your input images, you could either use the Gaussian filter with kernel size varying with distance from the camera ( Link ) or use a slightly more complex bilateral filter, which is widely used to eliminate noise while keeping edges mostly intact.


Gpu derivatives. How it's done across 2x2 boundary? This question bothers me since the first time I got to know about derivatives in pixel shader. Suppose we have a 2d texture:

5   10  20  30
6   11  50  100

If I correctly understand, the derivatives for the pixel [0, 0] will be:

dx = pixel[1, 0] - pixel[0, 0] = 10 - 5 = 5
dy = pixel[0, 1] - pixel[0, 0] = 6 - 5 = 1

And this will be 100% guaranteed because gpu executes pixels in 2x2 block. But what is the dx-derivative for the pixel[1, 0]? The pixel right to it is from another block which maybe not executed yet? And what about pixel[3, 0]? It doesn't have a right neighbor at all!

In Vulkan the shader only looks at each 2x2 and won't attempt to look beyond the neighbourhood: http://vulkan-spec-chunked.ahcox.com/ch15s05.html

$$dPdx_{0,0}=dPdx_{1, 0} = P_{1,0}−P_{0,0}\\
dPdx_{2,0}=dPdx_{3, 0} = P_{3,0}−P_{2,0}$$

For times when a pixel would fall off the geometry the shader is invoked for the values it would have if the triangle extended a pixel further.


OpenGL - Detection of edges I'd like to load arbitrary meshes and draw thick black lines along the edges to get a toon-shading like look.
I managed to draw a black silhouette around the objects by using the stencil buffer. You can see the result here:

But what's missing are the black lines in the object itself. I thought about checking normal discontinuities: Checking if a neighbouring pixel has a different normal vector than the current one. If yes, an edge has been found. Unfortunately, I've no idea how I could implement this approach, neither in OpenGL nor in GLSL vertex/fragment shader.

I'd be very happy for some help regarding this approach or any other regarding edge detection.

Edit: I do not use any textures for my meshes.

Too be more precise, I would like to create a CAD/CAM solution that looks as much as possible like this (taken from Top Solid https://www.youtube.com/watch?v=-qTJZtYUDB4):

Generally edge detection boils down to detect areas of the image with high gradient value.

In our case we can crudely see the gradient as the derivative of the image function, therefore the magnitude of the gradient gives you an information on how much your image changes locally (in regards of neighbouring pixels/texels). Now, an edge is as you say an indication of discontinuity, so now that we defined the gradient is clear that this info is all we need. Once we find the gradient of an image, it's just a matter of applying a threshold to it to obtain a binary value edge/non-edge.

How do you find this gradient is really what you are asking and I am yet to answer :)

Lots of ways! Here a couple :)

Built in shader functions

Both hlsl and glsl offer derivative functions. In GLSL you have dFdx and dFdy that give you respectively gradient information in x and y direction. Typically these functions are evaluated in a block of 2x2 fragments.
Unless you are interested in a single direction, a good way to have a compact result that indicates how strong is the gradient in the region is fwidth that gives you nothing else but the sum of the absolute value of dFdy and dFdy. You are likely to be interested in an edge on the overall image rather than on a specific channel, so you might want to transform your image function to luma. With this in mind,when it comes to edge detection your shader could include something to the like of:

  float luminance = dot(yourFinalColour,vec3(0.2126, 0.7152, 0.0722));
  float gradient = fwidth(luminance );
  float isEdge = gradient > threshold;

With an high threshold you will find coarser edges and you might miss some, conversely, with a low threshold you might detect false edges. You have to experiment to find the threshold that better suits your needs.

The reason why these functions work is worth mentioning but I don't have time for it now, I am likely to update this answer later on :)

Screen space post-process

You could go fancier than this, now the field of Edge detection in image processing is immense. I could cite you tens of good ways to detect edge detection according to your needs, but let's keep it simple for now, if you are interested I can cite you more options!

So the idea would be similar to the one above, with the difference that you could look at a wider neighbourhood and use a set of weights on sorrounding samples if you want. Typically, you run a convolution over your image with a kernel that gives you as a result a good gradient info.  A very common choice is the Sobel kernel

Which respectively give you gradients in x and y directions:

You can get the single value out of the gradient as $ GradientMagnitude = \sqrt{ (Gradient_x) ^ 2 + (Gradient_y) ^ 2 } $

Then you can threshold as the same way I mentioned above.

This kernel as you can see give more weight to the central pixel, so effectively is computing the gradient + a bit of smoothing which traditionally helps (often the image is gaussian blurred to eliminate small edges).

The above works quite well, but if you don't like the smoothing you can use the Prewitt kernels:

(Note I am in a rush, will write proper formatted text instead of images soon! )

Really there are plenty more kernels and techniques to find edge detection in an image process-y way rather than real time graphics, so I have excluded more convoluted (pun not intended) methods as probably you'd be just fine with dFdx/y functions.


Do normals in obj files have to be changed into eye coordinate before being used? Obj files give pre-calculated normals which are in model coordinates. So I wonder if they have to be transformed into eye coordinate (where I use them to calculate lighting)? I'm thinking yes because it makes sense, but only rotation and scale because translation doesnt really make sense for free vectors.

Yes, all vectors used for lighting need to be in the same coordinates. If you do lighting in eye space, the normals would need to be transformed to eye space as well. Lighting calculations can also be done in world space, or any other space that's convenient.

You're right that normal vectors shouldn't receive translation; likewise other vectors that represent directions or displacements rather than points in space. If you represent transformations as 4×4 matrices, it's simple to skip the translation by just leaving off the last column.

Also, one subtlety that comes up is that if your transformations contain non-uniform scaling (different scale factors along different axes) or shear, then you need to use the inverse transpose matrix for normals. This is to ensure that the normals stay perpendicular to the polygons. (The inverse transpose will be the same as the original matrix if neither scaling nor shearing is present.)


Spine. What is the name of the process? https://www.youtube.com/watch?v=twUz99ek5do&list=WL&index=5

There is a process from 1:51, where the guy start making polygons around the bodyparts. I think this is this part is where (with later transformations) you can make a single draw like a 3D animation (with transforming the triangles). What is the name of the process, and is this effect available in other animation softver?
I'm thinking about mostly maya, spriter or unity.
Ty

This appears to be simply skeletal animation, which is a standard technique that is available in all modern animation packages. Whether applied to 3D meshes or (as here) 2D ones, the principle is the same.

The pseudo-3D effect on the sprites is created by using a mesh where vertices are placed along the contours of the sprite texture, then animated to squash and stretch parts the image in such a way that it creates the illusion of 3D motion, even though the sprite images are authored in 2D. There's a description of the technique in the Spine docs.


Can I write histogram equalization, blurring, sharpening grayscale image etc.. as shader programs with openGL? unsigned int histogram[256]={0};
for(int i=0;i<width*height;i++)  //to iterate over pixels
  ++histogram[image[i]];

const float cutoffpercentage = 0.05;  //To cut off lower and upper values
unsigned char lowerbound,upperbound;
unsigned int histAccu =0;  //pixel entries < 5% =lower bound
const unsigned int lowerpercentile = cutoffpercentage * width*height;
const unsigned int upperpercentile = (1-cutoffpercentage)*width*height;

for(int h =0;h <256;h++){
histAccu += histogram[h];

if(histAccu <= lowerpercentile){
lowerbound = h;
continue;
}

if(histAccu >= upperpercentile){
upperbound = h;
break;
}
}

Then assign new gray values. Can I write this program to equalize the histogram of an image on a gpu shader memory and programs for image segmentation, blurring, sharpening etc...as a shader program using openGL ?? Can we write shader programs just like a normal application program in C ?? Kindly help me on this, as I'm new to shader programming.

Yes you can. There are basically two approaches:

You draw everything to a framebuffer object. Then you apply image processing fragment shaders to resulting texture of the previous draw call, where the shaders take the pervious texture with a sampler.

or if you have a particular image and you would like to have more control on the numeric precision, the number of neighbours available to you while working on pixel etc:

You apply compute shaders directly to image.

For the first approach, see Learn OpenGL, for compute shaders, see Anton's tutorial


Is it possible to store the plain buffer data in gltf files? The GL Transmission Format comes along with a JSON styled main file which basicly describes the scene and binary files which contain the buffers.

I'm currently writing a WebGL library and I need to work alot with the vertex and index buffers. So, my question now is:

Would it be possible to store plain text array buffers in the gltf (e.g., as JSON) instead of generating binary blobs always when the buffers are adjusted?

Embedding human readable data is not supported. However you can put the data into a data uri to store the data inline in base64 format.

"buffers": {
    "a-buffer-id": {
        "byteLength": 1024,
        "type": "arraybuffer",
        "uri"="data:application/octet-stream;base64,..."
    }
},


Can I use the same vertices to render multiple things? I'm passing my vertex shader a bunch of vertices and color data. I would like to first render the triangles and then render a point at each vertex. The triangles render fine, but I can't think of a way to render the triangles and the points without making and calling a whole new method for rendering points, which I guess I could do, but I'd like to know if there is some shader magic that accomplish it for me.

Thanks in advance!

This is my vs:

#version 120

uniform mat4 projection;

void main() {

gl_Position = projection * ftransform();
gl_PointSize = 1.0;
gl_FrontColor = gl_Color;

}

This is my fs:

#version 120

#define point_color vec3(1,1,1)

void main() {

vec4 color = gl_Color;
gl_FragColor = vec4(color);

}

This is the code behind it. I'm using lwjgl.

public void render() {

    glEnableClientState(GL_VERTEX_ARRAY);
    glEnableClientState(GL_COLOR_ARRAY);
    glEnable(GL_PROGRAM_POINT_SIZE);

    glBindBuffer(GL_ARRAY_BUFFER, VBO_id);
    glVertexPointer(3, GL_FLOAT, 0, 0);

    glBindBuffer(GL_ARRAY_BUFFER, c_id);
    glColorPointer(4, GL_FLOAT, 0, 0);

    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, i_id);
    glDrawElements(GL_TRIANGLES, draw_count, GL_UNSIGNED_INT, 0);

    glDrawArrays(GL_POINTS, 0, draw_count_v);

    glBindBuffer(GL_ARRAY_BUFFER, 0);
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0);

    glDisableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_COLOR_ARRAY);
    glDisable(GL_PROGRAM_POINT_SIZE);
}

If your intention is to render some textured or colored triangles and points colored diferently at the same time at vertex postions with one draw call:

glDrawElements(GL_TRIANGLES, draw_count, GL_UNSIGNED_INT, 0);

In order to access vertex data in vertex shader, you can create vertex array object which references to vertex buffer object with data you provided (vertices) for example at index 0.

then in vertex shader you have:

layout (location = 0) in vec3 vPosition; //positions for each vertex

You might then do something like that (vertex shader):

flat out vec3 fpos; //not interpolated vertices
out vec3 pos; //interpolated vertices (for default)
void main(void)
{
  pos = vPosition;
  fpos = vPosition;
}

And then in fragment shader:

if(length(fpos-pos)<threshold) //small threshold
 fragColor = pointColor;
else
 fragColor = triangleColor;


Does the original Perlin noise ever have any advantage over Simplex noise? Perlin noise is one of the most popular procedural noise functions. Perlin later developed Simplex noise which improves on some of the shortcomings of Perlin noise, notably its inefficiency in higher dimensions and directional artefacts (Wikipedia lists five advantages of Simplex noise). Still, Perlin noise appears to be widely used. I can imagine that the main reason is that Simplex noise is conceptually much more difficult to understand, but by now there should be enough implementations so that you don't have to re-implement it yourself.

Does Perlin noise have any advantages over Simplex noise? When picking a noise function, would I ever choose Perlin instead of Simplex?

To directly answer the question: Simplex noise is patented, whereas Perlin noise is not. Other than that, Simplex noise has many advantages that are already mentioned in your question, and apart from the slightly increased implementation difficulty, it is the better algorithm of the two.

I believe the reason why many people still pick Perlin noise is simply because it's more widely known. It's also worth noting that Perlin noise is very frequently confused with a combination of value noise and Fractal Brownian Motion (FBM).

Perlin noise, Simplex noise and value noise are all methods for synthesizing coherent noise. On the other hand, FBM (sometimes called "FBM noise"), is what is used when adding multiple layers of noise on top of each other at different scales to obtain more complex functions. The combination of FBM and value noise is simple to implement and can be very useful for terrain synthesis, procedural clouds and friends, and it is quite popular. However, it tends to be mistakenly labelled Perlin noise, misleadingly adding to its popularity.


What algorithm to reassemble a broken image? What algorithm can I use to reassemble a broken image?

Is there any technique? algorithm? Can I use graphs? machine learning? computer graphics?

If the pieces are fairly close to where they were, ICP could be of help (under the assumption that the pieces are planar polygons).