Fix resolvable depth difference calculation for triangle near 0

When all three vertices of a triangle have depth 0, the maximum exponent e is 0. We can't just use (e - n) << n as the bit pattern for 2^(e - n), because the exponent field underflows and creates a negative value. All triangles where all depth values have a bias-adjusted exponent less than 23 have this issue (note 2^(23 - 127) = 4.9e-32). The fix is to perform 2^(e - n) as 2^e * 2^(-n). The multiplication handles the underflow of the exponent either by creating a subnormal, or zero. Bug: b/139341727 Change-Id: I44b87feb55f61c5fa18ba235e9ec211926de3b3e Fixes: b/174051829 Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/51148 Kokoro-Result: kokoro <noreply+kokoro@google.com> Tested-by: Nicolas Capens <nicolascapens@google.com> Reviewed-by: Alexis Hétu <sugoi@google.com> Commit-Queue: Nicolas Capens <nicolascapens@google.com>

Fix resolvable depth difference calculation for triangle near 0
7a519d34 · Nicolas Capens · swiftshader-scoped@luci-project-accounts.iam.gserviceaccount.com · 0a64a974 · 7a519d34
Commit 7a519d34 authored Dec 10, 2020 by Nicolas Capens Committed by swiftshader-scoped@luci-project-accounts.iam.gserviceaccount.com Dec 11, 2020
Show whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

SetupRoutine.cpp src/Pipeline/SetupRoutine.cpp +1 -1

No files found.
--- a/src/Pipeline/SetupRoutine.cpp
+++ b/src/Pipeline/SetupRoutine.cpp
@@ -466,7 +466,7 @@ void SetupRoutine::generate()
 					Int e = Max(Max(e0, e1), e2);
-					r = As<Float>(e - (23 << 23));
+					r = As<Float>(e) * Float(1.0f / (1 << 23));
 				}
 				bias = r * *Pointer<Float>(data + OFFSET(DrawData, constantDepthBias));