<cat>Dev</cat> <title>Render Reference - Stencils</title> # Render Library References These pages seek to provide helpful insight into the groups of functions provided by the <page text="Render Library">render</page>. The major function groups are: * <page text="Beams">render_beams</page> * <page text="Minification and Magnification Texture Filters">render_min_mag_filters</page> * <page text="Render Targets">render_rendertargets</page> * **Stencils** # A Brief Preamble It's going to be very helpful to have a grasp on <page text="Render Targets">render_rendertargets</page> before learning about Stencils. It's not necessarily a hard requirement and you don't need to be an expert, but you will have a much easier time mentally "connecting the dots" if you're familiar with the terms defined there. To avoid repetition, the shorthand term `byte` is used throughout this page to mean "A <page>Number</page> with an integer value between `0` and `255`, representing 8 bits of data." While this is not a term used in Lua, it is commmon in programming. # What are Stencils? **Stencils are a system to constrain (or "mask") drawing operations on <page text="Render Targets">render_rendertargets</page> on a per-pixel basis.** Whenever anything is drawn to a Render Target, including the main game view, the Stencil performs logical checks (More on these later.) for each pixel that is about to be affected by the draw operation and determines whether or not that pixel should be allowed to change from its current values. It's important to understand that while Stencils are often used in 3D effects, they are fundamentally a 2D concept operating exclusively on 2D pixels. They are only able to interact with 3D objects because those 3D objects are being drawn (or "rasterized") onto a 2D Render Target. # What are Stencils Used For? Here are just a few of the things Stencils have been used for: * Impossible rooms that appear larger on the inside than the outside can contain * Bullet holes that the player can see through * Adding windows to walls that don't have them * Portals as seen in the game Portal # What are the Parts of the Stencil System? ## How to Read This Section The following section defines and explains each of the available Stencil functions, operations, and concepts in the order they are executed during a draw operation. Each pixel affected by a draw operation has the exact same logic applied to it and is independent of the other pixels. You can think of this section as a `for` loop iterating over each affected pixel. There are three (3) possible paths through this logic that a pixel can take. At the end of each path, the loop continues onto the next pixel. Notes on where we are in the `for` loop will be shown `[In this Format]` ## Parts of the Stencil System `[Prior to the draw operation]` * The **Stencil Buffer** holds a `byte` (Here called a "Stencil value" or "Stencil Buffer value") for every individual pixel on the Render Target it belongs to. * By default, every pixel's Stencil value is `0`. * The <page text="Reference Value">render.SetStencilReferenceValue</page> is a `byte` that Stencil Buffer values are compared against. When a draw operation is performed, each of the affected pixels has its Stencil Buffer value compared to the Reference Value. `[A draw operation has just been executed]` `[At this point, no Stencil configuration can be changed.]` `[This is the start of the loop over the pixels affected by the draw operation]` * The <page text="Compare Function">render.SetStencilCompareFunction</page> determines which of the possible relationships (Greater than, less than, equal to, etc.) between the Stencil Buffer value for a pixel and the current Reference Value should be considered a "**Pass**" and which should be considered a "**Fail**". <page text="The full list of possible Compare Functions can be found here">Enums/STENCILCOMPARISONFUNCTION</page> * <page text="The Test Mask">render.SetStencilTestMask</page> is a `byte` that will be <page text="bitwise ANDed">bit.band</page> with all values as they are read from the Stencil Buffer. * By default, the Read Mask is set to make no change to read values. Depending on if a pixel Passed or Failed the Compare Function, its value is then potentially modified, depending on the current Stencil configuration. * **Operations** modify the Stencil Buffer value for a given pixel depending on if it has Passed or Failed. Operations include incrementing the current value by one (1), replacing the current value with the Reference Value, making no change, and others. <page text="The full list of possible Operations can be found here">Enums/STENCILOPERATION</page> * <page text="The Write Mask">render.SetStencilWriteMask</page> is a `byte` that will be <page text="bitwise ANDed">bit.band</page> with all values before they are written to the Stencil Buffer. * By default, the Write Mask is set to make no change to written values. * The <page text="Fail Operation">render.SetStencilFailOperation</page> is performed on pixels that Fail the Compare Function. `[This pixel is done and the next one can be processed.]` For the pixels that Pass the Compare Function, there is one additional check they must Pass before they are allowed to be modified by the draw operation. * The **Depth Test** (or "Z test") compares the pixel's current Depth Buffer value against the Depth value of the draw operation. Unlike the Compare Function, this comparison is fixed and cannot be configured. * The <page text="Z Fail Operation">render.SetStencilZFailOperation</page> is performed on the Stencil Buffer for pixels where the current Depth Buffer value is less than the draw operation's proposed depth value. This indicates that the draw operation should not be visible on this pixel because it is drawing behind the current pixel. `[The next pixel can be processed.]` If a pixel has passed both the Compare Function as well as the Depth Test, it is considered Passed and will have its contents replaced/modified by the draw operation. This will affect the Color, Depth, and Stencil Buffer values for the pixel. * The <page text="Pass Operation">render.SetStencilPassOperation</page> is performed on the Stencil Buffer for pixels that have passed both the Compare Function and the Depth Test and are being updated by the draw operation. `[The next pixel can be processed.]` The following flowchart outlines the same information as the text above to aid in understanding. <image src="b2b4c/8dc4830dfd0658f.png" size="67100" name="StencilSystemFlow.png" /> # How Can Those Parts Be Used to Make Things? ## Common Usage Pattern In practical applications, Stencils are used in the following pattern: 1. Configure the Stencil system 2. Perform draw operations to set Stencil Buffer values 3. Re-configure the Stencil system 4. Perform draw operations that are masked by the Stencil Buffer values from steps 1-2 It's quite common for steps 1-2 and steps 3-4 to be repeated multiple times (Called "passes" and not to be confused with the term used to denote a pixel's pass/fail state) in order to configure the Stencil Buffer in ways that it cannot be with only a single pass. ## 2D Masking ## Simple 2D Masking <example> <description> This example shows a simple masking operation on 2D content following the Common Usage Pattern outlined above. * First, we'll configure the Stencil system, including parts we aren't using, because we can't trust other addons to clean up after themselves when they use the Stencil system and we don't want their configuration to influence our results. * Next, we'll create our mask by performing a draw operation that will change the Stencil Buffer values for the pixels we want to draw onto without actually drawing anything onto the screen. * Then, we'll re-configure the Stencil system to stop modifying the Stencil Buffer during draw operations because we don't want to mess up the mask we just created. * Finally, we'll draw the content we want to display on the screen, masked by the values in the Stencil Buffer. </description> <code> local COLOR_BLACK = Color( 0, 0, 0 ) local COLOR_BLUE = Color( 34, 19, 233 ) local MATERIAL_COLOR = Material( "color" ) -- Performs a draw operation that will be used to create our mask -- This is a separate function only to make the hook's code easier to read. local function DrawMask() -- We only need to set this color or material is so we can be sure we're not drawing a fully -- transparent shape, which would prevent any pixels from being affected and thus prevent -- the mask from being created surface.SetDrawColor( COLOR_BLACK ) surface.SetMaterial( MATERIAL_COLOR ) -- Draw the mask surface.DrawTexturedRectRotated( -- Place it in the center of the screen ScrW()/2, ScrH()/2, -- Set its size to be based on the screen size ScrW()/4, ScrH()/4, -- Make it spin based on the current time CurTime() * 100 ) end -- Performs a draw operation that we'll be masking with Stencils -- This is a separate function only to make the hook's code easier to read. local function DrawContent() -- This time, the color we choose matters visually surface.SetDrawColor( COLOR_BLUE ) surface.SetMaterial( MATERIAL_COLOR ) -- Draw the content -- This is a rectangle that moves in a circle and rotates counter/anti-clockwise surface.DrawTexturedRectRotated( -- Bounce around near the center of the screen so we can see the masking more obviously ScrW()/2 + math.sin( CurTime() * 4 ) * ScrH()/8, ScrH()/2 + math.cos( CurTime() * 4 ) * ScrH()/8, -- Set its size to be based on the screen size ScrW()/4, ScrH()/4, -- Make it spin the opposite direction of the mask CurTime() * 100 * -1 ) end hook.Add( "HUDPaint", "StencilExampleHud", function() -- If you'd like to see the mask layer, you can comment this line out render.SetStencilEnable( true ) -- First, let's configure the parts of the Stencil system we aren't using right now so we know they -- won't affect what we're doing ------ Make sure we're starting with a Stencil Buffer where all pixels have a Stencil value of 0 render.ClearStencil() ------ 255 corresponds to 8 bits (1 byte) where all bits are 1 (11111111), which is a bitmask that won't ------ change anything render.SetStencilTestMask( 255 ) render.SetStencilWriteMask( 255 ) ------ If a pixel fully passes, or if it fails the depth test, don't modify its Stencil Buffer value ------ (KEEP the current value) render.SetStencilPassOperation( STENCILOPERATION_KEEP ) render.SetStencilZFailOperation( STENCILOPERATION_KEEP ) -- Now, let's confiure the parts of the Stencil system we are going to use ------ We're creating a mask, so we don't want anything we do right now to draw onto the screen ------ All pixels should fail the Compare Function (They should NEVER pass) render.SetStencilCompareFunction( STENCILCOMPARISONFUNCTION_NEVER ) ------ When a pixel fails, which they all should, we want to REPLACE their current Stencil value with ------ whatever the Reference Value is ------ We can use whatever Reference Value we want for this; They have no special meaning. render.SetStencilReferenceValue( 9 ) render.SetStencilFailOperation( STENCILOPERATION_REPLACE ) -- At this point, we're ready to perform draw operations to create our mask DrawMask() -- Now, we need to re-configure the Stencil system so we can use the mask we just created ------ Like the Pass and Z Failure operations, we don't want to change the Stencil Buffer if a pixels ------ fails the Compare Function because that would change the mask render.SetStencilFailOperation( STENCILOPERATION_KEEP ) ------ We want to pass (and therefore draw on) pixels that match (Are EQUAL to) the Reference Value render.SetStencilCompareFunction( STENCILCOMPARISONFUNCTION_EQUAL ) -- We're finally ready to draw the content we want to have masked DrawContent() render.SetStencilEnable( false ) end ) </code> <output> <upload src="b2b4c/8dc48274ea360ef.mp4" size="3544394" name="SimpleStencilExampleOutput.mp4" /> To help us visualize what's happening in the clip above, I've commented out the line that enables the Stencil system so we can see both the mask (in black) and the content (in blue) at the same time. <upload src="b2b4c/8dc48271dcf0b3c.mp4" size="5533749" name="SimpleStencilExampleVisibleMaskOutput.mp4" /> </output>⤶ </example>⤶ ⤶ ## Making an Impossible Space⤶ <example>⤶ <description>⤶ This example showcases one of the things Stencils are commonly used for: Impossible spaces (Also called being "bigger on the inside" or "non-Euclidean geometry")⤶ ⤶ The basic concept is this: ⤶ * Draw the impossible space's exterior. ⤶ * If you're making a door to another dimension, this is the doorframe that you'll be placing the effect into/onto. ⤶ * Create a mask that is the size and shape of the opening you want to look into to see the effect. ⤶ * In the case of a doorway, this would be something the size and shape of the door. ⤶ * Quads are an easy choice for this role because they have backface culling as a standard feature. ⤶ * Draw the interior space (The larger on the inside part) using the mask we set up before. ⤶ * In the example below, the interior space is a short tunnel, but this could anything. ⤶ * Fun fact: In the case of the portals from the game Portal, this was not an "interior" space, but instead the view pointing out of the opposite portal.⤶ </description>⤶ <code>⤶ AddCSLuaFile()⤶ ⤶ DEFINE_BASECLASS( "base_anim" )⤶ ⤶ ENT.PrintName = "Impossible Geometry"⤶ ENT.Category = "Fun + Games"⤶ ⤶ -- It's important that we use the translucent rendergroup because it renders last⤶ -- If we used opaque instead, translucent objects would draw on top of the impossible geometry⤶ -- and the effect would be ruined.⤶ ENT.RenderGroup = RENDERGROUP_TRANSLUCENT⤶ ⤶ ENT.Spawnable = true⤶ ENT.AdminOnly = false⤶ ⤶ if SERVER then⤶ function ENT:Initialize()⤶ self:SetModel( "models/props_phx/construct/metal_plate1.mdl" )⤶ self:PhysicsInit( SOLID_VPHYSICS )⤶ self:SetMoveType( MOVETYPE_VPHYSICS )⤶ self:SetSolid( SOLID_VPHYSICS )⤶ local phys = self:GetPhysicsObject()⤶ if phys:IsValid() then⤶ phys:Wake()⤶ end⤶ end⤶ end⤶ if CLIENT then⤶ ⤶ local COLOR_BLACK = Color( 0, 0, 0 )⤶ ⤶ -- The size of the opening mask⤶ local OPENING_WIDTH = 20⤶ local OPENING_HEIGHT = 20⤶ ⤶ -- How far from the center of the entity to place the opening mask⤶ local OPENING_DEPTH = 3.2⤶ ⤶ function ENT:OnRemove( isFullUpdate )⤶ -- Clientside models need to be removed manually⤶ if self.clientsideModel and IsValid( self.clientsideModel ) then⤶ self.clientsideModel:Remove()⤶ end⤶ end⤶ ⤶ function ENT:Initialize()⤶ -- We'll be using this Clientside Model to draw the interior of the entity⤶ self.clientsideModel = ClientsideModel( "models/props_phx/construct/metal_plate1.mdl", RENDERGROUP_OPAQUE )⤶ self.clientsideModel:SetNoDraw( true )⤶ end⤶ ⤶ -- Just a convenience function. The main difference between this and ⤶ -- render.DrawModel() is that this also renders flashlights.⤶ function ENT:DrawClientsideModel( model, pos, angles )⤶ self.clientsideModel:SetModel( model )⤶ self.clientsideModel:SetPos( pos )⤶ self.clientsideModel:SetAngles( angles )⤶ self.clientsideModel:DrawModel()⤶ render.RenderFlashlights( function() self.clientsideModel:DrawModel() end )⤶ end⤶ ⤶ function ENT:DrawInterior()⤶ local interiorAngles = self:GetAngles()⤶ local interiorPos = self:LocalToWorld( Vector( 0, 0, -44.3 ) )⤶ ⤶ -- By coincidence, both of these models have compatible origins so we can draw them at the same position and angle⤶ -- without problems. This would not necessarily be true with other models.⤶ self:DrawClientsideModel( "models/props_phx/construct/metal_tube.mdl", interiorPos, interiorAngles )⤶ self:DrawClientsideModel( "models/props_phx/construct/metal_plate1.mdl", interiorPos, interiorAngles )⤶ end⤶ ⤶ -- The opening mask is a single quad that faces away from the entity⤶ function ENT:DrawOpeningMask()⤶ render.SetColorMaterial()⤶ render.DrawQuad(⤶ self:LocalToWorld( Vector( OPENING_WIDTH, OPENING_HEIGHT, OPENING_DEPTH ) ),⤶ self:LocalToWorld( Vector( OPENING_WIDTH, -OPENING_HEIGHT, OPENING_DEPTH ) ), ⤶ self:LocalToWorld( Vector( -OPENING_WIDTH, -OPENING_HEIGHT, OPENING_DEPTH ) ),⤶ self:LocalToWorld( Vector( -OPENING_WIDTH, OPENING_HEIGHT, OPENING_DEPTH ) ),⤶ COLOR_BLACK⤶ )⤶ end⤶ ⤶ function ENT:Draw()⤶ ⤶ self:DrawModel()⤶ ⤶ -- The Halo system also uses Stencils, so we'll be polite and avoid messing with them while it's drawing⤶ -- If we didn't do this, the Halo system would break and the entire screen would flash very unpleasantly ⤶ -- when the entity is grabbed with the physgun.⤶ local isDrawingHalo = halo.RenderedEntity() == self⤶ if isDrawingHalo then⤶ return⤶ end⤶ ⤶ -- Reset the Stencil system to values we know aren't going to cause problems⤶ render.ClearStencil()⤶ render.SetStencilWriteMask( 255 )⤶ render.SetStencilTestMask( 255 )⤶ render.SetStencilPassOperation( STENCILOPERATION_KEEP )⤶ render.SetStencilZFailOperation( STENCILOPERATION_KEEP )⤶ ⤶ render.SetStencilEnable( true )⤶ ⤶ -- We'll use Stencil Buffer value 1 as the value representing the opening mask⤶ render.SetStencilReferenceValue( 1 )⤶ ⤶ -- We only care about the Tepth Test right now, so don't bother with the compare function⤶ render.SetStencilCompareFunction( STENCILCOMPARISONFUNCTION_ALWAYS )⤶ ⤶ -- Start creating the opening mask⤶ render.SetStencilPassOperation( STENCILOPERATION_REPLACE )⤶ ⤶ -- Here we rely on the fact that a quad only draws when viewed from the front.⤶ -- From the back, it doesn't draw at all. This is called "backface culling" and⤶ -- without it the interior would draw on top of the entity even when viewed from the back.⤶ self:DrawOpeningMask()⤶ ⤶ -- Don't modify the mask now that it's created⤶ render.SetStencilFailOperation( STENCILOPERATION_KEEP )⤶ ⤶ -- We now want to only draw where the mask is set (Stencil Buffer values match the Reference Value)⤶ render.SetStencilCompareFunction( STENCILCOMPARISONFUNCTION_EQUAL )⤶ ⤶ -- Clear the Depth Buffer on the masked pixels⤶ -- Otherwise, the pixels from the entity and any surface behind it will always be ⤶ -- in front of the interior and thus the interior will never draw.⤶ render.ClearBuffersObeyStencil( 0, 0, 0, 255, true )⤶ ⤶ self:DrawInterior()⤶ ⤶ render.SetStencilEnable( false )⤶ end⤶ end⤶ </code>⤶ <output>⤶ ⤶ </output> </example>