yukijoou/kwin
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

core/colorpipeline: allow rounding away more floating point errors

Browse source 
Black point compensation introduced a tiny bit increased error in the floating point
calculations, but the errors are still way too small to actually matter.
To avoid those unnecessary calculations causing performance issues and more rounding
errors, this commit changes the allowed error to 10^-5.
This commit is contained in:
Xaver Hugl 2024-08-13 17:42:33 +02:00
parent aa93f6ac55
commit e3953e7602
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
src/core/colorpipeline.cpp
View file

@ -167,7 +167,7 @@ static bool isFuzzyIdentity(const QMatrix4x4 &mat)
{
// matrix calculations with floating point numbers can result in very small errors
// -> ignore them, as that just causes inefficiencies and more rounding errors
constexpr float maxResolution = 0.0000001;
constexpr float maxResolution = 0.00001;
for (int i = 0; i < 4; i++) {
for (int j = 0; j < 4; j++) {
const float targetValue = i == j ? 1 : 0;
@ -183,7 +183,7 @@ static bool isFuzzyScalingOnly(const QMatrix4x4 &mat)
{
// matrix calculations with floating point numbers can result in very small errors
// -> ignore them, as that just causes inefficiencies and more rounding errors
constexpr float maxResolution = 0.0000001;
constexpr float maxResolution = 0.00001;
for (int i = 0; i < 4; i++) {
for (int j = 0; j < 4; j++) {
if (i < 3 && i == j) {