USN-8745-1: KissFFT vulnerabilities

KissFFT versions prior to the fix commit 1b083165 contain an integer overflow in kiss_fft_alloc() in kiss_fft.c on platforms where size_t is 32-bit. The nfft parameter is not validated before being used in a size calculation (sizeof(kiss_fft_cpx) * (nfft - 1)), which can wrap to a small value when nfft is large. As a result, malloc() allocates an undersized buffer and the subsequent twiddle-factor initialization loop writes nfft elements, causing a heap buffer overflow. This vulnerability only affects 32-bit architectures.

KissFFT before commit 8a8e66e contains an integer overflow vulnerability in the kiss_fftndr_alloc() function in kiss_fftndr.c where the allocation size calculation dimOther*(dimReal+2)*sizeof(kiss_fft_scalar) overflows signed 32-bit integer arithmetic before being widened to size_t, causing malloc() to allocate an undersized buffer. Attackers can trigger heap buffer overflow by providing crafted dimensions that cause the multiplication to exceed INT_MAX, allowing writes beyond the allocated buffer region when kiss_fftndr() processes the data.