Anti-aliasing

Anti-Aliasing

Aliasing generally refers to jagged edges on geometry caused by a limited amount of pixels to fully shade a continuous primitive (like a triangle, for example). Anti-aliasing combats this effect by smoothening out those edges.

MSAA

MSAA = Multisample anti-aliasing. It’s pretty simple: just increase the number of sampling points per pixel. Pixels that would normally be considered outside of a primitive (and not shaded) will be shaded (and the color will be interpolated accordingly). Since my renderer is a forward renderer, I can use MSAA pretty easily. If I ever write a deferred renderer, though, which is quite popular, then MSAA would be off the table.

Enabling MSAA in OpenGL is pretty easy. The way I do it is I create a multisampled texture used as a color attachment to a framebuffer, and a corresponding multisampled depth attachment. The one thing that tripped me up was forgetting to use the 2d multisample texture enum instead of the regular 2d texture enum (so maybe not that easy for me). Then, I render everything to that framebuffer (you can specify the number of times to sample: 2x, 4x, 8x, etc.) and blit to the default framebuffer. Higher sampling means better anti-aliasing quality. This workflow fits pretty nicely with my HDR pipeline too. I just set the multisample framebuffer to store floating point values, then blit the color to my HDR framebuffer (also stores floating point), and finally, I can handle HDR + tonemapping with anti-aliasing (gamma correction is also part of the pipeline).

Specular Anti-Aliasing

Now, MSAA sure helps a lot. But, after implementing PBR with really nice specular lighting, both from direct and indirect lighting sources, I noticed some weird shimmering on my wooden floors. Now, those wooden floors have a normal map, and a metallic-roughness map, and the artist that authored it clearly wanted a really glossy floor. When you normal map surfaces that have very low roughness (very smooth), a type of problem known as specular aliasing manifests. It’s pretty annoying, but a price to pay for high quality details and physically correct smoothness. Specular aliasing causes shimmers, sparkles, and jagged lighting. MSAA can’t fix that. So, I did some research and found some methods to smoothen this out. In particular, I use a paper from Tokuyoshi and Kaplanyan on improving geometric specular antialiasing with some ideas from Toksvig (mipmapping normal maps). In the end, I converged to a decent shader-only solution that helps to somewhat smoothen out the specular aliasing. I’ll tell you roughly how it works from my own understanding. So specular aliasing is caused by two things: a narrow specular lobe from low roughness, and high variance at a rate faster than can be captured by a pixel. In games, we typically can only afford one shading sample per pixel which means it’s much harder to capture these highlights. The shimmering is caused by missing a highlight one frame, and then capturing it in the next (a flicker). The solution is to widen the lobe my making the surface rougher, and/or reduce the variance. Basically do the opposite of what causes the problem. Michael Toksvig, the guy who released a paper tackling this shimmering, noticed that the length of a mipmapped normal represented variance. And when I talk about variance, I mean how much do normals change (with respect to some quantity) given that a single pixel could be representative of multiple normals. Again, higher variance increases the aliasing issue. Typically, you normalize the vector you get from the normal map, and then use it as the per-fragment normal that it is. But, in his paper, he added an extra step where you could use the normal (from before it’s normalized, so not a normal really, but it’s from the normal map) to tweak the specular shininess (in our PBR workflow, it’s the specular lobe). The length of the averaged normal vector is the variance. The smaller the length, the higher the variance. The closer you are to a unit vector (length 1), the lower the measured variance. He did this for Blinn-Phong lighting but the idea remains important: if you measure higher variance, widen the specular lobe as a response (make it rougher). The variance changes with different mip levels, and those mip levels change with different viewing angles. This idea had an interesting side effect in that artists could create areas on the normal map with high variance leading to a rougher appearance. This led to a free gloss-map. Back to the main point, Tokuyoshi and Kaplanyan refined this idea of variance more than a decade later. We want to increase the roughness if we detect more variance right? The paper I read assumed a Beckmann NDF which modeled microfacets as a gaussian distribution. Guassians have an interesting property in that they can be convolved with simple addition. So, roughness, or how much the surface tilts, and the size of the specular lobe is governed by a gaussian distribution. The width of this gaussian is the variance. So if a surface is rougher, the gaussian is wider, the variance is higher, and more surfaces are tilting whichever way they want. If a surface is smoother, the surfaces align more, the gaussian is thinner, and that means less variance. So to get a new filtered roughness, we convolve the roughness gaussian and the squared screen space derivative of the normal with respect to the pixels. α𝑓𝑖𝑙𝑡𝑒𝑟𝑒𝑑2=α2+min⁡(2*σ2(Δnu2+Δnv2),κ)\alpha_{\textit{filtered}}^2 = \alpha^{2} + \min(2*\sigma^2({\Delta{n_u}^2 + \Delta{n_v}^2}), \kappa) where we use Disney’s α\alpha: α=𝑟𝑜𝑢𝑔ℎ𝑛𝑒𝑠𝑠2\alpha = \textit{roughness}^2 and σ=0.5\sigma = 0.5, and κ=0.17\kappa = 0.17. These values are tweakable if you find yourself overfiltering or underfiltering. The squared screen space derivative (summed along the x and y direction) is multiplied by σ\sigma to produce the variance. This produces a wider lobe the higher the pixel-normal variance is. It’s quite nice because we’re measuring how fast the normal (also it doesn’t need the halfway direction vector) changes in the x and y screen space at our current pixel, and using that as our variance. One problem is that it computes a derivative based on its other neighbors (other pixels) which means that it misses details at a sub-pixel level. It’ll estimate zero variance if a normal map detail is finer than a pixel because it means it got sampled at a higher mip level where the averaged normal would lose that detail. This is where I slot in a modification to the proposed algorithm. I take the averaged normal from the mipped normal map (the same one Toksvig uses for his mipmapping technique) and calculate its variance. Now to be honest, adding this step, I can’t quantitatively say it offers much of an improvement on top of the specular mapping from Tokuyoshi/Kaplanyan but I feel like it does. So let’s say we take the variance of the mipped normals: 1K∑iK(Ni−Navg)2\frac{1}{K} \sum_{i}^{K} (N_i - N_{avg})^2 where NiN_i are the unit length normals and NavgN_{avg} is the averaged normal we sampled from the mipped normal map. Other than that this is just the standard formula for variance. Let’s expand that square. 1K∑iK|Ni|2−2K∑iKNi⋅Navg+1K∑iK|Navg|2\frac{1}{K} \sum_{i}^{K} \left|N_i\right|^2 - \frac{2}{K} \sum_{i}^{K} N_i \cdot N_{avg} + \frac{1}{K} \sum_{i}^{K} \left|N_{avg}\right|^2 1K∑iK|Ni|2−2K(∑iKNi)⋅Navg+1K∑iK|Navg|2\frac{1}{K} \sum_{i}^{K} \left|N_i\right|^2 - \frac{2}{K} (\sum_{i}^{K} N_i) \cdot N_{avg} + \frac{1}{K} \sum_{i}^{K} \left|N_{avg}\right|^2 NiN_i is just a unit vector, the average of NiN_i for all ii to kk is NavgN_{avg}, and NavgN_{avg} is constant. Using that let’s evaluate further: 1−2(Navg⋅Navg)+Navg21 - 2(N_{avg} \cdot N_{avg}) + N_{avg}^2 and the final variance is: 1−Navg21 - N_{avg}^2 We can just add this to the screen space variance we added earlier, and now it should take into account sub-pixel variance because it’s encoded in the normal map. The screen space variance is per pixel while the mipmap normal variance is per texel. And adding these variances together really only works if they don’t double count variance (there could be overlap), but since this is an approximation and we clamp everything by κ\kappa anyways it doesn’t matter too much. Applying this technique is pretty cheap in the shader and it does smooth out a lot of the specular shimmering I’ve been getting. That said there is still noticeable aliasing sometimes, but it’s a lot smoother than without. It is also a combination of a really bright HDR environment map and a glossy wooden floor texture that makes this shimmering stand out a lot. With other environment maps specular aliasing isn’t noticeable with or without the specular anti-aliasing. For now, I’m just going to accept this is the worst case, and keep developing on!