Fix float to r11g11b10 conversion

Floating point negative values were first converted to half values, followed with a conversion to 11 bit or 10 bit mini floats by chopping the mantissa and removing the sign bit. Negative values were converted to the same values as positive values since removing the sign bit ended up doing the equivalent of an Abs() call. Clamped the value to 0 before the conversion to solve the issue. Bug: b/146223877 b/147900455 Change-Id: I97decae66dc57a68f175b06902eb6725fc7d5794 Reviewed-on: https://swiftshader-review.googlesource.com/c/SwiftShader/+/42548 Kokoro-Presubmit: kokoro <noreply+kokoro@google.com> Tested-by: Alexis Hétu <sugoi@google.com> Reviewed-by: Nicolas Capens <nicolascapens@google.com>

Fix float to r11g11b10 conversion
1c9f2426 · Alexis Hetu · Alexis Hétu · fb670f56 · 1c9f2426
Commit 1c9f2426 authored Mar 19, 2020 by Alexis Hetu Committed by Alexis Hétu Mar 19, 2020
Hide whitespace changes
Inline Side-by-side

Showing with 2 additions and 1 deletion

ShaderCore.cpp src/Pipeline/ShaderCore.cpp +2 -1

No files found.
--- a/src/Pipeline/ShaderCore.cpp
+++ b/src/Pipeline/ShaderCore.cpp
@@ -611,7 +611,8 @@ Float4 r11g11b10Unpack(UInt r11g11b10bits)
 UInt r11g11b10Pack(const Float4 &value)
 {
-	auto halfBits = floatToHalfBits(As<UInt4>(value), true);
+	// 10 and 11 bit floats are unsigned, so their minimal value is 0
+	auto halfBits = floatToHalfBits(As<UInt4>(Max(value, Float4(0.0f))), true);
 	// Truncates instead of rounding. See b/147900455
 	UInt4 truncBits = halfBits & UInt4(0x7FF00000, 0x7FF00000, 0x7FE00000, 0);
 	return (UInt(truncBits.x) >> 20) | (UInt(truncBits.y) >> 9) | (UInt(truncBits.z) << 1);