This is the accompanying page for the preprint article Audio Inpainting in Time-Frequency Domain with Phase-Aware Prior authored by Peter Balušík and Pavel Rajmic.
We address the problem of time-frequency audio inpainting, where the goal is to fill missing spectrogram portions with consistent information. Despite recent advances, existing approaches still face limitations in both reconstruction quality and computational efficiency. To bridge this gap, we propose a method that utilizes a phase-aware signal prior which exploits estimates of the instantaneous frequency. An optimization problem is formulated and solved using the generalized Chambolle–Pock algorithm. The proposed method is evaluated against other time-frequency inpainting methods, specifically a deep-prior audio inpainting neural network, the autoregression-based approach known as Janssen-TF, and a sparsity-driven baseline. For short gap durations, the proposed approach achieves superior SNR, while performing comparably to Janssen-TF on larger gaps. In terms of perceptual quality (both objective and subjective), the proposed method consistently outperforms existing methods across all gap lengths. In addition, the reconstructions are obtained with a substantially reduced computational cost compared to alternative methods.
The preprint is available at arXiv.
Audio examples from the listening test
You can listen to the audio excerpts used in the listening test. The denotation of the six examples is the same as in the article Deep Prior-Based Audio Inpainting Using Multi-Resolution Harmonic Convolutional Neural Networks.Example0 (piano)
| Gap size | 2 columns | 4 columns | 6 columns |
|---|---|---|---|
| Original audio | |||
| Corrupted audio | |||
| DPAI with context | |||
| JanssenTF ADMM | |||
| U-PHAIN-TF | |||
| L1-TF* |
* These files were not included in the listening test.
Example1 (piano)
| Gap size | 2 columns | 4 columns | 6 columns |
|---|---|---|---|
| Original audio | |||
| Corrupted audio | |||
| DPAI with context | |||
| JanssenTF ADMM | |||
| U-PHAIN-TF | |||
| L1-TF* |
* These files were not included in the listening test.
Example3 (voice)
| Gap size | 2 columns | 4 columns | 6 columns |
|---|---|---|---|
| Original audio | |||
| Corrupted audio | |||
| DPAI with context | |||
| JanssenTF ADMM | |||
| U-PHAIN-TF | |||
| L1-TF* |
* These files were not included in the listening test.
Example4 (music)
| Gap size | 2 columns | 4 columns | 6 columns |
|---|---|---|---|
| Original audio | |||
| Corrupted audio | |||
| DPAI with context | |||
| JanssenTF ADMM | |||
| U-PHAIN-TF | |||
| L1-TF* |
* These files were not included in the listening test.
Example5 (music)
| Gap size | 2 columns | 4 columns | 6 columns |
|---|---|---|---|
| Original audio | |||
| Corrupted audio | |||
| DPAI with context | |||
| JanssenTF ADMM | |||
| U-PHAIN-TF | |||
| L1-TF* |
* These files were not included in the listening test.
Example7 (voice)
| Gap size | 2 columns | 4 columns | 6 columns |
|---|---|---|---|
| Original audio | |||
| Corrupted audio | |||
| DPAI with context | |||
| JanssenTF ADMM | |||
| U-PHAIN-TF | |||
| L1-TF* |
* These files were not included in the listening test.
Additional supplementary audio examples
These examples were not included in the listening test. The denotation of the examples is the same as in the article Deep Prior-Based Audio Inpainting Using Multi-Resolution Harmonic Convolutional Neural Networks.Example2 (rock)
| Gap size | 2 columns | 4 columns | 6 columns |
|---|---|---|---|
| Original audio | |||
| Corrupted audio | |||
| DPAI with context | |||
| JanssenTF ADMM | |||
| U-PHAIN-TF | |||
| L1-TF |
Example6 (rock)
| Gap size | 2 columns | 4 columns | 6 columns |
|---|---|---|---|
| Original audio | |||
| Corrupted audio | |||
| DPAI with context | |||
| JanssenTF ADMM | |||
| U-PHAIN-TF | |||
| L1-TF |
Supplementary plots
The plot below is not presented in the paper due to the lack of space.
The box plot presents the same test scores as in the paper, but split according to the size of gaps. For shorter gaps the results of Janssen-TF and U-PHAIN-TF are comparable and near-perfect, for larger gaps U-PHAIN-TF is the clear winner.
Masks
Only three masks were shown in the paper. Here, each mask is shown.
