Skip to main content

Image based lighting notes

These are some of my notes from implementing image based lighting, a.k.a. IBL.

I thought I understood it pretty well until I started implementing it. Now, after a lot of reading, discussing and trying things out, I'm finally getting back to the stage where I tentatively believe I understand it. Hopefully these notes will save future me, or anyone else reading them, from going through the same difficulty. I'll update them if I find out anything further.

Some helpful references




The radiance integral

$$L_o(v) = L_e(v) + \int_\Omega L_i(l) f(l, v) (n \circ l) dl$$
  • The amount of light reflected at some point on a surface is the sum, over all incoming directions, of the light from that direction multiplied by the BRDF for the surface material, multiplied by the cosine of the angle between the light and the relevant normal vector, plus any light emitted by the surface itself.
  • We usually calculate this separately for the diffuse and specular lighting at the point, then sum the results.
  • For diffuse lighting, the relevant normal vector is the surface normal.
  • For specular lighting, the relevant normal vector is the half-vector: the vector halfway between the light direction and the view direction (where both vectors are pointing away from the surface point rather than towards it).

BRDF

  • A BRDF is a function which, given a view vector and a light vector, returns the amount of light which will be reflected along the view vector.
  • The view vector is the vector from the point being shaded to the view position, not the other way around.
  • The light vector is the vector from the point being shaded to the light position, not the other way around.
  • BRDF is rotation invariant, only depends on the angle between the two vectors.
  • BRDFs usually have other parameters which are constant for any given material.

Diffuse BRDF

  • Diffuse contribution comes from light emerging from the surface after bouncing around inside it a bit.
  • This means the light direction can be fairly random.
  • So a fairly common diffuse BRDF is simply a constant value: the diffuse color for the mtaerial divided by pi (this is the Lambertian model).
  • Materials with lots of subsurface scattering will need a more complicated diffuse BRDF such as Oren-Nayar.
  • Oren-Nayar is based on the microfacet model described below.

Microfacet BRDF

$$f(l, v, h) = \frac{D(h) F(v, h) G(l, v, h)}{4 (n \circ l) (n \circ v)}$$
  • Usually used for specular BRDFs, but Oren-Nayar (diffuse) is also based on microfacets.
  • Assumes that a surface is composed of lots of tiny planar fragments whose normals vary according to some statistical distribution. This is the Cook-Torrance model.
  • Breaks the BRDF into 3 main components:
    • D(h), the normal distribution function.
    • F(v, h), the Fresnel function.
    • G(l, v, h), the geometric shadowing function.
  • D(h) tells us the fraction of microfacet normals which point in a given direction.
    • This describes how rough or smooth a material is.
    • Rougher materials will have a more even spread of normals across whereas smoother ones will tend to have them concentrated around a single direction.
    • Lots of options for this: Blinn-Phong, GGX, GTR, etc.
  • F(v, h) gives us the fraction of light that gets reflected in each channel.
    • This tells us the material's "colour".
    • Pretty much everyone uses Schlick's approximation for this.
  • G(l, v, h) describes how much of the reflected light is blocked by other microfacets.
    • Tells us how "dark" a material is.
    • Is (or should be) affected by how rough the material is.
    • There are lots of options for this too.

IBL in general

  • The idea of IBL is to bake out components of the radiance integral into textures ahead of time, so that our shaders can rapidly approximate it given just a direction, a surface colour and a roughness value.
  • We can bake out the irradiance (incoming light) for any given direction:
    • The irradiance will be from all pixels within a cone around the given direction.
    • The solid angle of the cone is determined by the normal distribution function for your BRDF (which in turn is shaped by the material's roughness).
    • This will be stored in a texture where we can look up values using a direction vector, i.e. a cube map or a lat-long 2D texture.
    • You can use a mip-mapped texture to store the results for different roughness levels. mip-level 0 will be the smoothest level, higher levels will be rougher.
    • Mip-level N+1 should represent a cone covering twice as many pixels as Mip-level N, so that the sampling works out correctly.
    • Because the mip-level contents depend on the mapping from the roughness parameter to a cone width, the irradiance map will be specific to a particular normal distribution function.
  • You can precalculate the rest of the BRDF separately as well:
    • This is the split sum approximation described in the SIGGRAPH 2013 course on physically based rendering.
    • Generate a 2D texture where the u axis corresponds to dot(N, V) values >= 0.0 and the v axis corresponds to roughness values between 0.0 and 1.0.
    • The texture contains a scale and bias to the material color.
    • Apply the scale and bias, then multiply the result by the value from the irradiance map to get the final color for the pixel.

IBL vs. Reflections

  • What's the difference between IBL and reflections?
    • Theoretically: nothing. They're the same thing.
    • I think there may be an accident of terminology here, where "IBL" is often used to mean diffuse IBL; and reflections is used to mean specular IBL.
  • It's common to provide a low-res HDR image with a matching high-res LDR image. In this case:
    • The low-res HDR image is for diffuse IBL.
    • The high-res LDR image is for reflections.
  • This implies that reflections are expected to sample from an LDR input whereas IBL (of any kind, specular or diffuse) is better off sampling from a HDR input.

Comments

Popular posts from this blog

How to outperform std::vector in 1 easy step

Everyone who's familiar with C++ knows that you should avoid resizing a std::vector inside a loop wherever possible. The reasoning's pretty obvious: the memory allocated for the vector doubles in size each time it fills up and that doubling is a costly operation. Have you ever wondered why it's so costly though?

It's tempting to assume that because implementations of the STL have been around for so long that they must be pretty efficient. It turns out that's a bad assumption because the problem, in this case, is the standard itself: specifically, the allocator interface.

The allocator interface provides two methods that obtain and release memory:

allocate allocates uninitialized storage
(public member function)deallocate deallocates storage
(public member function)

(taken from this page).

What's missing is away of growing an existing memory allocation in place. In C this is provided by the realloc function, but there's no equivalent in the std::allocator interfa…

OpenGL ES and occlusion queries

This is a follow-up to my earlier post "WebGL doesn't have query objects".

Since I wrote that post, the situation has changed a bit. It's still true to say that WebGL doesn't have query objects, but the underlying reason - that OpenGL ES doesn't - is no longer true.

For OpenGL ES 2.0, there's an extension which provides basic query functionality: EXT_occlusion_query_boolean (which seems to have been based on ARB_occlusion_query2 from regular OpenGL). For OpenGL ES 3.0, the functionality from that extension appears to have been adopted into the standard. The extension provides two query types, both of which set a boolean value to indicate whether any pixels passed the depth and stencil tests.

While this is progress, unfortunately it's still not sufficient to implement the pixel accurate collision detection method I described in an earlier post. For that purpose it's not enough to know whether any pixels passed the tests; you want to know whether al…

Octree node identifiers

Let's say we have an octree and we want to come up with a unique integer that can identify any node in the tree - including interior nodes, not just leaf nodes. Let's also say that the octree has a maximum depth no greater than 9 levels, i.e. the level containing the leaf nodes divides space into 512 parts along each axis.

The encoding The morton encoding of a node's i,j,k coordinates within the tree lets us identify a node uniquely if we already know it's depth. Without knowing the depth, there's no way to differentiate between cells at different depths in the tree. For example, the node at depth 1 with coords 0,0,0 has exactly the same morton encoding as the node at depth 2 with coords 0,0,0.

We can fix this by appending the depth of the node to the morton encoding. If we have an octree of depth 9 then we need up to 27 bits for the morton encoding and 4 bits for the depth, which still fits nicely into a 32-bit integer. We'll shift the morton code up so that i…