nvidia/bigvgan_v2_44khz_128band_512x

audio to audioPyTorchPyTorchneural-vocoderaudio-generationaudio-to-audioarxiv:2206.04658license:mitmit
292.9K

instantiate the model. You can optionally set use_cuda_kernel=True for faster inference.

model = bigvgan.BigVGAN.from_pretrained('nvidia/bigvgan_v2_44khz_128band_512x', use_cuda_kernel=False)

remove weight norm in the model and set to eval mode

model.remove_weight_norm() model = model.eval().to(device)

load wav file and compute mel spectrogram

wav_path = '/path/to/your/audio.wav' wav, sr = librosa.load(wav_path, sr=model.h.sampling_rate, mono=True) # wav is np.ndarray with shape [T_time] and values in [-1, 1] wav = torch.FloatTensor(wav).unsqueeze(0) # wav is FloatTensor with shape [B(1), T_time]

compute mel spectrogram from the ground truth audio

mel = get_mel_spectrogram(wav, model.h).to(device) # mel is FloatTensor with shape [B(1), C_mel, T_frame]

generate waveform from mel

with torch.inference_mode(): wav_gen = model(mel) # wav_gen is FloatTensor with shape [B(1), 1, T_time] and values in [-1, 1] wav_gen_float = wav_gen.squeeze(0).cpu() # wav_gen is FloatTensor with shape [1, T_time]

you can convert the generated waveform to 16 bit linear PCM

wav_gen_int16 = (wav_gen_float * 32767.0).numpy().astype('int16') # wav_gen is now np.ndarray with shape [1, T_time] and int16 dtype


## Using Custom CUDA Kernel for Synthesis
You can apply the fast CUDA inference kernel by using a parameter `use_cuda_kernel` when instantiating BigVGAN:

```python
import bigvgan
model = bigvgan.BigVGAN.from_pretrained('nvidia/bigvgan_v2_44khz_128band_512x', use_cuda_kernel=True)

When applied for the first time, it builds the kernel using nvcc and ninja. If the build succeeds, the kernel is saved to alias_free_activation/cuda/build and the model automatically loads the kernel. The codebase has been tested using CUDA 12.1.

Please make sure that both are installed in your system and nvcc installed in your system matches the version your PyTorch build is using.

For detail, see the official GitHub repository: https://github.com/NVIDIA/BigVGAN?tab=readme-ov-file#using-custom-cuda-kernel-for-synthesis

Pretrained Models

We provide the pretrained models on Hugging Face Collections. One can download the checkpoints of the generator weight (named bigvgan_generator.pt) and its discriminator/optimizer states (named bigvgan_discriminator_optimizer.pt) within the listed model repositories.

Model NameSampling RateMel bandfmaxUpsampling RatioParamsDatasetStepsFine-Tuned
bigvgan_v2_44khz_128band_512x44 kHz12822050512122MLarge-scale Compilation5MNo
bigvgan_v2_44khz_128band_256x44 kHz12822050256112MLarge-scale Compilation5MNo
bigvgan_v2_24khz_100band_256x24 kHz10012000256112MLarge-scale Compilation5MNo
bigvgan_v2_22khz_80band_256x22 kHz8011025256112MLarge-scale Compilation5MNo
bigvgan_v2_22khz_80band_fmax8k_256x22 kHz808000256112MLarge-scale Compilation5MNo
bigvgan_24khz_100band24 kHz10012000256112MLibriTTS5MNo
bigvgan_base_24khz_100band24 kHz1001200025614MLibriTTS5MNo
bigvgan_22khz_80band22 kHz808000256112MLibriTTS + VCTK + LJSpeech5MNo
bigvgan_base_22khz_80band22 kHz80800025614MLibriTTS + VCTK + LJSpeech5MNo
DEPLOY IN 60 SECONDS

Run bigvgan_v2_44khz_128band_512x on Runcrate

Deploy on H100, A100, or RTX GPUs. Pay only for what you use. No setup required.