Fix write stencil ops when writeMask == 0

We were checking whether we need to apply the write mask by erroneously looking at whether the writeMask was non-zero. This meant that when the write mask was zero, we would compute the stencil value, and store it, rather than store 0. Instead, what we mean is to apply the write mask if masking does something: when it's not FF (or 2^n-1, with n = number of stencil bits). Bug: b/141484021 Change-Id: I6134289ac8bb13a3502884e4e408f71961814adf Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/36848Reviewed-by: Nicolas Capens <nicolascapens@google.com> Kokoro-Presubmit: kokoro <noreply+kokoro@google.com> Tested-by: Antonio Maiorano <amaiorano@google.com>

Fix write stencil ops when writeMask == 0
caf67c37 · Antonio Maiorano · 8edf82c7 · caf67c37
Commit caf67c37 authored Oct 01, 2019 by Antonio Maiorano
Hide whitespace changes
Inline Side-by-side

Showing with 2 additions and 2 deletions

PixelRoutine.cpp src/Pipeline/PixelRoutine.cpp +2 -2

No files found.
--- a/src/Pipeline/PixelRoutine.cpp
+++ b/src/Pipeline/PixelRoutine.cpp
@@ -718,7 +718,7 @@ namespace sw
 		Byte8 newValue;
 		stencilOperation(newValue, bufferValue, state.frontStencil, false, zMask, sMask);
-		if(state.frontStencil.writeMask != 0)
+		if((state.frontStencil.writeMask & 0xFF) != 0xFF) // Assume 8-bit stencil buffer
 		{
 			Byte8 maskedValue = bufferValue;
 			newValue &= *Pointer<Byte8>(data + OFFSET(DrawData,stencil[0].writeMaskQ));
@@ -730,7 +730,7 @@ namespace sw
 		stencilOperation(newValueBack, bufferValue, state.backStencil, true, zMask, sMask);
-		if(state.backStencil.writeMask != 0)
+		if((state.backStencil.writeMask & 0xFF) != 0xFF) // Assume 8-bit stencil buffer
 		{
 			Byte8 maskedValue = bufferValue;
 			newValueBack &= *Pointer<Byte8>(data + OFFSET(DrawData,stencil[1].writeMaskQ));