Re: [PATCH v11 08/15] iio: afe: rescale: fix accuracy for small fractional scales

From: Andy Shevchenko
Date: Wed Dec 22 2021 - 07:29:08 EST


On Wed, Dec 22, 2021 at 5:47 AM Liam Beguin <liambeguin@xxxxxxxxx> wrote:
>
> From: Liam Beguin <lvb@xxxxxxxxxx>
>
> The approximation caused by integer divisions can be costly on smaller
> scale values since the decimal part is significant compared to the
> integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
> cases to maintain accuracy.

...

> - tmp = div_s64(tmp, 1000000000LL);
> +
> + tmp = div_s64_rem(tmp, 1000000000LL, &rem);
> *val = tmp;

> - return scale_type;
> +

It seems you may add this blank line to one of the previous patches, may you?

> + if (!rem)
> + return scale_type;

--
With Best Regards,
Andy Shevchenko