audio distance and PQMF in 2.3.1

[victor-shepardson]

fullband spectral distance is now computed between the original waveform and the model reconstruction:

RAVE/rave/model.py

Line 291 in 8b25031

x_raw = batch

RAVE/rave/model.py

Line 340 in 8b25031

fullband_distance = self.audio_distance(x_raw, y_raw)

compared to older versions, where it was computed between the model reconstruction and the PQMF reconstruction of the data:

RAVE/rave/model.py

Lines 239 to 249 in 7661926

	x = self.pqmf.inverse(x_multiband)
	y = self.pqmf.inverse(y_multiband)
	p.tick('recompose')
	for k, v in multiband_distance.items():
	distances[f'multiband_{k}'] = v
	else:
	x = x_multiband
	y = y_multiband
	fullband_distance = self.audio_distance(x, y)

iiuc this should mainly affect causal models, but it seems like a problem in that case, since a causal model would not be able to compensate for the delay induced by the PQMF, unless I misunderstand?

do you know if the change was deliberate or why it was made?

[domkirke]

Hello Victor,

I am finally running through your issues. I actually change the code to be more general as the older ones were a little tweaked, but the inner behavior did not change : a multiband loss for each PQMF band, and a fullband loss for the global signal. And, regarding causality : causality only change the internal padding of convolutional layers, such that a given latent position does not influence the future of the sequence. Furthermore, reconstruction is done on both slightly-delayed PQMF, and is then trained to compensate it with the fullband distance. Some improvements could be done here though, even if nothing comes into my mind right now.