Preprint: Upsampling Artifacts in Neural Audio Synthesis1 min read

Upsamplers are a key element for developing computationally efficient and high-fidelity neural audio synthesizers. Given their importance, together with the fact that the audio literature only provides sparse and unorganized insights, our work is aimed at advancing and consolidating our current understanding of neural upsamplers.

arXiv paper


A number of recent advances in audio synthesis rely on neural upsamplers, which can introduce undesired artifacts. In computer vision, upsampling artifacts have been studied and are known as checkerboard artifacts (due to their characteristic visual pattern). However, their effect has been overlooked so far in audio processing. Here, we address this gap by studying this problem from the audio signal processing perspective. We first show that the main sources of upsampling artifacts are: (i) the tonal and filtering artifacts introduced by problematic upsampling operators, and (ii) the spectral replicas that emerge while upsampling. We then compare
different neural upsamplers, showing that nearest neighbor interpolation upsamplers can be an alternative to the problematic (but state-of-the-art) transposed and subpixel convolutions which are prone to introduce tonal artifacts