Audio Course

Unit 0. Welcome to the course!

Unit 1. Working with audio data

Unit 2. A gentle introduction to audio applications

Unit 3. Transformer architectures for audio

Refresher on transformer models CTC architectures Seq2Seq architectures Audio classification architectures Quiz Supplemental reading and resources

Unit 4. Build a music genre classifier

Unit 5. Automatic Speech Recognition

Unit 6. From text to speech

Unit 7. Putting it all together

Unit 8. Finish line

Course Events

Join the Hugging Face community

and get access to the augmented documentation experience

Collaborate on models, datasets and Spaces

Faster examples with accelerated inference

Switch between documentation themes

to get started

Check your understanding of the course material

1. What is a vocoder?

An additional neural network that turns the spectrogram output of a transformer into a waveform. A type of transformer layer that is responsible for creating audio embeddings. An additional neural network that preprocesses speech audio to remove background noise

2. Wav2Vec2 is an example of

Seq2Seq architecture CNN architecture CTC architecture

3. What does a blank token in CTC algorithm do?

Blank token indicates breaks between the individual words in the sentence. Blank token is a predicted token that serves as a hard boundary between groups of characters. It makes it possible to filter out the duplicate characters Blank token is used for sounds that do not match any tokens in the vocabulary, similar to token for 'unknown'.

4. Which of the following statements about CTC models is FALSE?

CTC models use only the encoder part of the transformer architecture. Wav2Vec2 & HuBERT use the exact same architecture but are trained differently. CTC models tend to perform best for speech recognition compared to other architectures.

5. Whisper is an example of

Seq2Seq architecture CNN architecture CTC architecture

6. What is the easiest way to perform audio classification?

Use encoder-decoder transformers on the audio waveform. Use spectrograms and treat the task as an image classification problem. Turn a CTC model into a general-purpose audio classifier by changing the labels and training it with a regular cross-entropy loss function.

7. True or false? When treating spectrograms as images for classification, you will always benefit from image data augmentation techniques, such as shifting an image, cropping it, or resizing.

True False

Update on GitHub

←Audio classification architectures

Next chapter

Check your understanding of the course material 1. What is a vocoder?2. Wav2Vec2 is an example of 3. What does a blank token in CTC algorithm do?4. Which of the following statements about CTC models is FALSE?5. Whisper is an example of 6. What is the easiest way to perform audio classification?7. True or false? When treating spectrograms as images for classification, you will always benefit from image data augmentation techniques, such as shifting an image, cropping it, or resizing.