A frame analysis of Hitman World of Assassination

Today, I want to take a look at the rendering of another game, or rather a series of games. Readers from my previous graphics analysis might have gathered that I really enjoy stealth games. The recent Hitman trilogy, released through the last decade by the danish studio IO Interactive, have been some of my all-time favorites. As a mysterious professional assassin, you are tasked with eliminating one or multiple targets in each level ; manipulative, corrupt or dangerous individuals evolving in the upper layers of society while serving an overarching machiavelian plot. I have found these games' mix of stealth and simulation in large sandbox levels to veer towards the puzzle genre. You achieve your contracts by learning how to navigate the maps and leverage all tools put at your disposal.

The three games have been released in 2016, 2018 and 2021 respectively and are all based on the same in-house engine, Glacier 2. The studio took care of upgrading the engine for each release, while ensuring it remained optimized and retro-compatible. Notably, all levels of the trilogy have now been ported to the most recent game. Each map is a sprawling and labyrinthic environment, with highly detailed realistic spaces often filled with glitz and mirrors (literally), and populated by hundreds of non-playable characters. After spending dozens of hours in these levels I was really interested in investigating how the game handles the rendering of its worlds, crowds and materials, but also its approach to rendering many dynamic reflections, even on console.

The view that started it all ; look at these beautifully sharp reflections

In this analysis, we will try to understand how the game renders a frame from start to finish. I have found there are always interesting things to learn through these explorations: which techniques were used, how were they implemented and adjusted for the game specific needs,... We will first look at the general layout of a frame, before diving into each step. At the end, a few additional frames will provide us with additional details on specific parts of the engine. Before we dive in, a few notes:

In this analysis, we will only care about rendering: how does the game display the levels and characters on screen, i.e. how are the data authored by the artists and designers combined with the current state of the game (including player interactions) to generate a final image. More specifically, we will describe what the game does on the graphics card at each frame, ideally 60 or even 120 times per second. On Windows, the game controls the GPU through the DirectX 12 programming interface[1].

Being more recent than my previous frame analyses, the engine fully leverages modern techniques[2] : compute shaders are used abundantly, as is the ability to distribute GPU work on multiple processing queues. Different parts of the frame are assigned to these queues depending on the type of workload, maximizing GPU use. Dependencies between queues are expressed using fences, and I will try to point them out when used. The game also relies heavily on memory aliasing, which allows different sections of the frame to reuse a region of memory as storage for multiple resources ; this helps minimize memory consumption of intermediate results in the frame.

Fortunately, the released game inserts debug markers in each frame[3], which were very helpful in understanding the overall frame structure. I have slightly reordered specfici steps to improve clarity in the explanation. The engine renders most objects to a geometry buffer (G-buffer), a set of textures storing the material parameters of the visible surfaces at each pixel. Lighting is applied as a postprocess on top, using a tiled deferred approach. Transparent objects and atmospheric effects are added afterwards, before a stack of post-processing filters helps bind the frame elements together.

The main steps of a Hitman frame are as follow:

In the next section we will focus on this view to examine each rendering step.

The view is in Mendoza, one of the later levels in the game, a large argentinian estate and vineyard where a fancy reception is taking place (along more discreet and less palatable activities).

Despite my efforts, all interesting effects are not necessarily visible in that view ; I will introduce a few similar views along the way to illustrate certain intermediate steps. A specific section at the end will examine different points of view that exemplify water rendering, depth of field, instinct vision, scope view, mirror reflections and picture-in-picture overlays. Let's dive!

Before any rendering for the player view takes place, the engine prepares data that will be used later during the frame. This mainly includes texture streaming and handling of character interactions with the environment. Extra steps that only involve static level data are also performed at this stage when loading the map.

While playing, the engine keeps loading and unloading textures from the CPU memory to the video memory, depending on which objects are in the vicinity of the camera. This is done because all textures used in the level would not fit in memory at once. To avoid crowding the main GPU pipeline with these transfers, they are performed on a dedicated copy queue. For performance reasons, the transfer destination texture is not the same as the one used during rendering, and final mip-per-mip copies between the two are performed on the main graphics queue. As they are on different queues, synchronization between the upload and finalization steps is achieved using a shared fence which is signaled for each completed upload using an increasing value.

Many levels feature screens and projectors that display images or videos. When dynamic, their appearance are copied at each frame to a shared texture array ; its mips are then updated with a blurred copy of the texture. This will later be used to apply the projectors lighting on rough surfaces. Here is an example taken from another view.

The more complex video feeds are decoded from YUV textures, where chrominance channels are stored at half-resolution compared to the luma. This representation is commonly used for image and video codecs as it maps well to the human eye behavior. A shader converts the YUV frame to a regular RGB texture for use at the next frame.

Character interactions with the environment are simulated by marking a spot at the character location in a render texture, seen from a bird's eye view above the level. This texture will later be sampled by elements that need to be aware of the player location in their vicinity, mainly grass tufts that are walked on and water surfaces. The engine stores the corresponding displacement effects in a 512x512 map with two layers: one where the character is drawn as a soft circle leaving a trail behind, and one where 2D water ripples are drawn. Particle systems simulated on the CPU are used to create these shapes.

A more complex example will be shown when examining water rendering in a later section.

Each map can contain hundred of light sources, and one would expect a good portion of them to cast shadows. While some lights will update their shadows dynamically at each frame, many other are generated during level loading by rendering only static objects into a dedicated render texture. The results are cached in a shared shadow map atlas, and won't be ever updated afterwards.

The static shadow map atlas generated at loading time ; each small square corresponds to a light

There are also mentions of a Box reflection update pass which is never active in game: its role is probably to capture all the environment map probes placed in the level[4]. The Rendering 'Hitman' with DirectX 12 presentation mentions shooting these when loading the map, but I've never managed to capture it ; I suppose they are now baked ahead of time and shipped with the game (especially on consoles).

Once preparation is complete, the bulk of the geometric rendering work is performed. The vast majority of level objects are opaque and are going to be rendered in a geometry buffer before being lit in a subsequent step.

But first, all transparent objects are rendered to a separate depth buffer, and output their normals in a R16G16 render texture. This pass treats them as opaque objects with depth testing, so for a given pixel only the closest transparent object is kept. The result appear to be unused in my captures (but might be used when raytracing is enabled, which would explain the early location in the frame).

Still before drawing any object, whenever small vegetation such as grass is present, a compute shader pass takes care of placing and culling tufts around the camera. It outputs in a large buffer a set of transformation matrices for each instance of grass, and indirect draw call argument for each type.

The renderer uses a G-buffer with six render textures along with a depth and stencil texture. Instead of rendering lit objects directly, the engine outputs intermediate material parameters to these layers, which will be sampled back during the lighting stage. This minimizes wasted work on surfaces that would be overwritten by other objects in front of them, and allow for other optimizations.

To minimize the overdraw even more, the engine renders extremely simplified geometry to the depth buffer[5] in a depth prepass. These occluder meshes are composed of simple quads and boxes representing the large bits of geometry in the level (buildings, terrain), and are rendered without a pixel shader for maximum performance. Reverse-Z is used to minimize depth fighting artifacts even for distant scenery. Some standard objects are also rendered in the Z-buffer, mostly vegetation. This last case requires a pixel shader to alpha test the leaves properly, but is worth it as trees can occlude large regions of the image.

Then, the G-buffer is filled. The engine re-renders the standard objects drawn in the prepass, using a Z-equal depth test. This way there is no need to perform alpha testing again, lowering the cost of rendering vegetation twice[6]. Other regular objects are then rendered. Most of them appear to have their input geometry normalized to a unit bounding box in order to store vertices in a more compressed format. The usual diffuse/normal/roughness textures are present; additionally the engine supports specular tint to create more complex shading effects. All are compressed using appropriate BCn formats: BC1 for diffuse, BC7 for packed specular tint and roughness, BC5 for normal maps.

Each rasterized object outputs information to the G-buffer for lighting later on. The amount of per-pixel data is quite large and is stored with the following layout:

• Albedo is stored as an 8-bit sRGB triplet in a texture. Its alpha channel stores a material type identifier.
• Normals are stored on two channels in a 16-bit unorm texture.
• Motion vectors are stored in a 16-bit float RG texture.
• Another 8-bit sRGBA texture stores the specular tint in its RGB channels, and what appears to be a subsurface scattering intensity in the alpha channel.
• Roughness is stored in the red channel of a third 8-bit RGBA texture, while the other channels are mostly unused. Green seems dedicated to baked ambient occlusion (AO), but in practice I never saw it used as the game relies on screen space AO instead. I could find no use for the blue and alpha channels in all my captures.
• An optional 8-bit RGBA texture stores complex material parameters, which are only writen to while rendering hair.

On top of regular objects, many decals are rendered to add details to most surfaces. Decals are simple bits of geometry that are placed close to existing geometry and update the G-buffer without modifying the depth. They often use blending to mix their material parameters with the ones of the underlying object. In this example view captured nearby, water puddles[7] are applied at this stage, by darke