翻译教程3:播放音频。(1000行以内的代码,使用SDL+FFMPEG,播放音频视频)

翻译教程3:播放音频。(1000行以内的代码,使用SDL+FFMPEG,播放音频视频)
2010年12月13日
　　  翻译教程3：播放音频。（1000行以内的代码，使用SDL+FFMPEG，播放音频视频） 收藏
　　So now we want to play sound. SDL also gives us methods for outputting sound. The SDL_OpenAudio§()打开音频设备function is used to open the audio device itself. It takes as arguments an SDL_AudioSpec§ struct, which contains all the information about the audio we are going to output.
　　下面解释一下计算机是如何处理音频的：
　　Before we show how you set this up, let's explain first about how audio is handled by computers. Digital audio consists of a long stream of samples. Each sample represents a value of the audio waveform§. Sounds are recorded at a certain sample rate, which simply says how fast to play each sample, and is measured in number of samples per second. Example sample rates are 22,050 and 44,100 samples per second, which are the rates used for radio and CD respectively. In addition, most audio can have more than one channel for stereo or surround, so for example, if the sample is in stereo, the samples will come 2 at a time. When we get data from a movie file, we don't know how many samples we will get, but ffmpeg will not give us partial samples - that also means that it will not split a stereo sample up, either.
　　数字音频包含有一个长的samples流。每个流代表一个音频的wavefrom值。
　　音频以特定的sample率记录，表示以多快去播放每个sample，并且以每秒一组samples测量。Example的sample率是每秒22，050和44,100个samples，分别用于radio和CD。
　　另外，大多数的audio对于stereo和surround都有超过一个的channel。
　　因此，比如说，如果sample是stereo的，sample每次就会到来2个。
　　如果从一个电影文件中取数据，我们不知道可以得到多少samples，但是ffmpeg并不给我们
　　Partial的samples，这意味着它也并不把一个stereo sample分隔开来。
　　SDL's method for playing audio is this: you set up your audio options: the sample rate (called "freq" forfrequency频率 in the SDL struct), number of channels, and so forth, and we also set a callback function 回调函数and userdata. When we begin playing audio, SDL will continually call this callback function and ask it to fill the audio buffer with a certain number of bytes. After we put this information in the SDL_AudioSpec§ struct, we call SDL_OpenAudio§(), which will open the audio device and give us back another AudioSpec struct. These are the specs we will actually be using - we are not guaranteed to get what we asked for!
　　SDL用于播放音频的 方法是： 
　　你设置你的音频的选项：
　　sample rate （叫做freq代表频率，在SDL结构体中）
　　Channel的数目
　　我们也设置一个回调函数和用户数据。
　　当我们开始播放音频的时候，SDL将会不断的调用回调函数，并且让这个回调函数使用一些bytes填充音频缓冲区。
　　我们将这些信息放入SDL_AudioSpec§ struct中，我们调用 SDL_OpenAudio，这个 函数将打开音频设备并且恢复、归还其他的AudioSpec struct.
　　这些是我们实际将会用到的眼睛---我们并不保证我们能得到什么。
　　Keep that all in your head for the moment, because we don't actually have any information yet about the audio streams yet! Let's go back to the place in our code where we found the video stream and find which stream is the audio stream. 这个时候，保留你的head种所有的信息。
　　因为我们实际上还没有任何有关音频流的信息呢。
　　我们回到代码中我们找到音频流的地方。并且找到哪个流在音频流中。
　　// Find the first video stream   找到第一个音频流
　　videoStream=-1;  
　　audioStream=-1; 
　　for(i=0; i nb_streams; i++) {  流的数目
　　if(pFormatCtx->streams[i]->codec->codec_type==CODEC_TYPE_VIDEO
　　&&   流的编码的编码类型
　　videoStream streams[i]->codec->codec_type==CODEC_T YPE_AUDIO &&
　　audioStream streams[audioStream]->codec;
　　Contained within this codec context is all the information we need to set up our audio:
　　这个编码上下文中，我们需要对音频设置的是这些信息：
　　wanted_spec.freq = aCodecCtx->sample_rate;  频率
　　wanted_spec.format = AUDIO_S16SYS;   格式
　　wanted_spec.channels = aCodecCtx->channels;  channels
　　wanted_spec.silence = 0;    silence 
　　wanted_spec.samples = SDL_AUDIO_BUFFER_SIZE;   sample等于音频缓冲区的大小
　　wanted_spec.callback = audio_callback;   回调
　　wanted_spec.userdata = aCodecCtx;    用户数据
　　if(SDL_OpenAudio§(&wanted_spec, &spec) codec_id); if(!aCodec) {   fprintf(stderr, "Unsupported codec!\n");   return -1; } There! Now we're ready to start pulling audio information from the stream. But what do we do with that information? We are going to be continuously getting packets from the movie file持续从电影文件中取包, but at the same time SDL is going to call the callback function! The solution is going to be to create some kind of global structure that we can stuff audio packets in so our audio_callback has something to get audio data from! So what we're going to do is to create a queue of packets 创建包队列. ffmpeg even comes with a structure to help us with this:AVPacketList§, which is just a linked list for packets包的链列表. Here's our queue structure:
　　typedef struct PacketQueue {
　　AVPacketList§ *first_pkt, *last_pkt;
　　int nb_packets;
　　int size;
　　SDL_mutex§ *mutex;
　　SDL_cond§ *cond;
　　} PacketQueue;
　　现在我们打算从流中取出音频信息了。但是我们对那些信息要做些什么呢？
　　我们要从电影文件中，持续取包，同时，SDL也会去调用回调函数。
　　这一方法是创建一些全局的数据结构，这样，我们的audio_callback 可以从中取出音频数据了。那么， 我们要做的是创建一个包队列。Ffmpeg甚至自带了一个结构，帮助我们：
　　AVPacketList§ 这仅仅是一个包的链表。这是队列结构：
　　typedef struct PacketQueue {
　　AVPacketList§ *first_pkt, *last_pkt;
　　int nb_packets;
　　int size;
　　SDL_mutex§ *mutex;
　　SDL_cond§ *cond;
　　} PacketQueue;
　　First, we should point out that nb_packets is not the same as size - size refers to a byte size that we get from packet->size. You'll notice that we have a mutex§ and a condtion variable§ in there. This is because SDL is running the audio process as a separate thread音频处理是作为一个独立的线程的. If we don't lock the queue properly, we could really mess up our data. We'll see how in the implementation of the queue. Every programmer should know how to make a queue如何创建一个队列, but we're including this so you can learn the SDL functions.
　　首先，我们应该指出nb_packets并不是和size一样大的，size代表是我们可以从packet->size中取出的字节数。你可能会注意到，我们有一个mutex和一个condition variable 。这是因为SDL是把音频处理作为一个独立的线程来做的。如果我们没有合理的锁定队列的话，我们就可能搞糟我们的数据。我们将会看到队列是如何实现的。每个程序员都应该知道如何创建一个队列，我们讲这些，这样，你就可以学习下SDL函数了。
　　First we make a function to initialize the queue:   首先，创建一个函数以初始化队列。
　　void packet_queue_init(PacketQueue *q) {
　　memset(q, 0, sizeof(PacketQueue));
　　q->mutex = SDL_CreateMutex§();
　　q->cond = SDL_CreateCond§();
　　}
　　Then we will make a function to put stuff in our queue:   然后我们创建一个函数填充队列：
　　int packet_queue_put(PacketQueue *q, AVPacket§ *pkt) {
　　AVPacketList§ *pkt1;
　　if(av_dup_packet§(pkt) 内存
　　if (!pkt1)
　　return -1;
　　pkt1->pkt = *pkt;
　　pkt1->next = NULL;
　　SDL_LockMutex§(q->mutex);  锁定互斥量。
　　if (!q->last_pkt)
　　q->first_pkt = pkt1;
　　else
　　q->last_pkt->next = pkt1;
　　q->last_pkt = pkt1;
　　q->nb_packets++;
　　q->size += pkt1->pkt.size;
　　SDL_CondSignal§(q->cond);
　　SDL_UnlockMutex§(q->mutex);
　　return 0;
　　}
　　SDL_LockMutex§() locks the mutex in the queue so we can add something to it, and thenSDL_CondSignal§() sends a signal to our get function (if it is waiting) through our condition variable to tell it that there is data and it can proceed, then unlocks the mutex to let it go.
　　SDL_LockMutex§() 锁定队列中的互斥量，这样我们可以向队列中添加一些东西。然后SDL_CondSignal§()通过条件变量发送一个信号给我们的获取函数（如果她正在等待中），告诉她有一个数据，并且可以处理。然后解锁mutex放获取函数走。 Here's the corresponding get function. Notice how SDL_CondWait§() makes the function block (i.e. pause until we get data) if we tell it to. int quit = 0; 这是相应的获取函数。注意SDL_CondWait§()是怎么阻塞的（例如暂停知道我们得到数据）。
　　static int packet_queue_get(PacketQueue *q, AVPacket§ *pkt, int block) {
　　AVPacketList§ *pkt1;
　　int ret;
　　SDL_LockMutex§(q->mutex);
　　for(;;) {
　　if(quit) {
　　ret = -1;
　　break;
　　}
　　pkt1 = q->first_pkt;
　　if (pkt1) {
　　q->first_pkt = pkt1->next;
　　if (!q->first_pkt)
　　q->last_pkt = NULL;
　　q->nb_packets--;
　　q->size -= pkt1->pkt.size;
　　*pkt = pkt1->pkt;
　　av_free§(pkt1);
　　ret = 1;
　　break;
　　} else if (!block) {
　　ret = 0;
　　break;
　　} else {
　　SDL_CondWait§(q->cond, q->mutex);
　　}
　　}
　　SDL_UnlockMutex§(q->mutex);
　　return ret;
　　}
　　As you can see, we've wrapped the function in a forever loop so we will be sure to get some data if we want to block. We avoid looping forever by making use of SDL's SDL_CondWait§() function. Basically, all CondWait does is wait for a signal from SDL_CondSignal§() (or SDL_CondBroadcast()) and then continue. However, it looks as though we've trapped it within our mutex - if we hold the lock, our put function can't put anything in the queue! However, what SDL_CondWait() also does for us is to unlock the mutex we give it and then attempt to lock it again once we get the signal.
　　正如你所看到的，我们隐藏、缠绕这个函数在一个永久不停止的循环里头，这样我们可以确信可以得到一些数据，如果我们想阻塞的话。我们可以使用SDL的SDL_CondWait§() 函数来避免无限的循环。基本上，所有的CondWait所作的，都是等待一个来自SDL_CondSignal§() 或者是SDL_CondBroadcast()的信号，然后继续。然而，好像我们在我们的mutex中，把她陷入了一样-----如果我们维持这个锁，我们的输入函数，就不能放入任何东西到这个队列中。然而，SDL_CondWait() 所能做的，是解锁我们给他的mutex，然后一旦我们再次得到信号，就再试图锁定她。 You'll also notice that we have a global quit variable that we check to make sure that we haven't set the program a quit signal (SDL automatically handles TERM signals and the like). Otherwise, the thread will continue forever and we'll have to kill -9 the program. ffmpeg also has a function for providing a callback to check and see if we need to quit some blocking function: url_set_interrupt_cb§. 你会注意到，我们有个全局的quit变量，用以检查是否我们没有给程序设定一个推出信号（SDL自动处理诸如TERM的信号），否则，线程将会一直持续下去，我们可能就得kill -9杀死程序。Ffmpeg也有个函数用以提供一个回调去检查，如果需要退出一些正在被阻塞的函数：url_set_interrupt_cb§.
　　int decode_interrupt_cb(void) {
　　return quit;
　　}
　　...
　　main() {
　　...
　　url_set_interrupt_cb§(decode_interrupt_cb);  
　　...    
　　SDL_PollEvent(&event);
　　switch(event.type) {
　　case SDL_QUIT:
　　quit = 1;
　　...
　　This only applies for ffmpeg functions that block, of course, not SDL ones. We make sure to set the quit flag to 1.
　　这仅仅使用于阻塞了的ffmpeg函数，当然，不是SDL的。我们可以设置quit标志为1。 The only thing left is to set up our queue:  下面要做的事情，就是设置我们的队列
　　PacketQueue audioq;  包队列 audioq
　　main() {
　　...
　　avcodec_open§(aCodecCtx, aCodec);  编码器打开
　　packet_queue_init(&audioq);  包队列初始化
　　SDL_PauseAudio§(0);    SDL暂停音频
　　SDL_PauseAudio§() finally starts the audio device. It plays silence if it doesn't get data; which it won't right away.    暂停音频最终启开了音频设备。如果没有得到数据，就安静地播放。
　　So, we've got our queue set up, now we're ready to start feeding it packets. We go to our packet-reading loop:  这样，我们就设置了队列，现在我们开始准备养大包。我们去读取包的循环中：
　　while(av_read_frame§(pFormatCtx, &packet)>=0) {
　　// Is this a packet from the video stream?  这是一个来自视频流的包么？
　　if(packet.stream_index==videoStream) {  如果是视频流
　　// Decode video frame  解码视频帧
　　....
　　}
　　} else if(packet.stream_index==audioStream) { 否则，如果是音频流
　　packet_queue_put(&audioq, &packet);  那么就向音频对列中放入包
　　} else {
　　av_free_packet§(&packet);  不是音频也不是视频，就释放包
　　}
　　Note that we don't free the packet after we put it in the queue. We'll free it later when we decode it.
　　注意，我们在放入队列数据之后，是不释放包的。在解码之后，我们就释放了。 Now let's finally make our audio_callback function to fetch the packets on the queue. The callback has to be of the form void callback(void *userdata, Uint8 *stream, int len), where userdata of course is the pointer we gave to SDL, stream is the buffer we will be writing audio data to, and len is the size of that buffer. Here's the code: 现在，让我们最终用audio_callback函数取回队列中的包。回调应该是void callback(void *userdata, Uint8 *stream, int len)的形式。Userdata是我们传给SDL的指针，stream是我们将要写入音频数据的缓冲区，len是缓冲区的大小。这是代码：
　　void audio_callback(void *userdata, Uint8 *stream, int len) {
　　AVCodecContext§ *aCodecCtx = (AVCodecContext *)userdata;
　　int len1, audio_size;
　　static uint8_t audio_buf[(AVCODEC_MAX_AUDIO_FRAME_SIZE * 3) / 2];
　　static unsigned int audio_buf_size = 0;
　　static unsigned int audio_buf_index = 0;
　　while(len > 0) {
　　if(audio_buf_index >= audio_buf_size) {
　　/* We have already sent all our data; get more */
　　audio_size = audio_decode_frame(aCodecCtx, audio_buf,
　　sizeof(audio_buf));
　　if(audio_size 内存填充
　　} else {
　　audio_buf_size = audio_size;
　　}
　　audio_buf_index = 0;
　　}
　　len1 = audio_buf_size - audio_buf_index; 缓冲区的长度
　　if(len1 > len)
　　len1 = len;
　　memcpy(stream, (uint8_t *)audio_buf + audio_buf_index, len1);  内存复制
　　len -= len1;
　　stream += len1;
　　audio_buf_index += len1;
　　}
　　}
　　This is basically a simple loop that will pull in data from another function we will write,audio_decode_frame(), store the result in an intermediary buffer, attempt to write len bytes to stream, and get more data if we don't have enough yet, or save it for later if we have some left over. The size ofaudio_buf is 1.5 times the size of the largest audio frame that ffmpeg will give us, which gives us a nice cushion.
　　这个基本上是一个简单的循环，将从其他我们要写入的函数中获取的数据放入。audio_decode_frame()，存储结果在中间人、调解人buffer里头，试图写入len长度的数据岛stream中，并且如果还没填满，就获取更多的数据，并且如果有一些剩下了，我们会随后保存起来。audio_buf的大小是ffmpeg所能给我们的最大音频帧大小的1.5倍。这是个很好的坐垫。
　　Finally Decoding the Audio  最后了，解码音频。
　　Let's get to the real meat of the decoder, audio_decode_frame: 让我们来读一下解码器的肉，audio_decode_frame:音频解码帧
　　int audio_decode_frame(AVCodecContext§ *aCodecCtx, uint8_t *audio_buf,
　　int buf_size) {
　　static AVPacket§ pkt;   音频视频包
　　static uint8_t *audio_pkt_data = NULL;  音频包数据指针
　　static int audio_pkt_size = 0;   音频包大小
　　int len1, data_size;  长度、数据大小
　　for(;;) {   无限循环
　　while(audio_pkt_size > 0) {
　　data_size = buf_size;
　　len1 = avcodec_decode_audio2§(aCodecCtx, (int16_t *)audio_buf, &data_size, 
　　audio_pkt_data, audio_pkt_size);
　　if(len1 read loop to the queue, which is then read by theaudio_callback function, which hands that data to SDL, which SDL beams to your sound card. Go ahead and compile:
　　当我们得到一些数据，我们立即返回查看是否我们依旧需要从队列中得到更多的数据，或者我们已经完成了。如果我们依旧有更多的包要处理，我们我们为接下来的工作保存这个包。如果我们完成up一个包，我们最重要释放那个包。
　　gcc -o tutorial03 tutorial03.c -lavutil -lavformat -lavcodec -lz -lm \
　　`sdl-config --cflags --libs`
　　Hooray! The video is still going as fast as possible, but the audio is playing in time. Why is this? That's because the audio information has a sample rate - we're pumping out audio information as fast as we can, but the audio simply plays from that stream at its leisure according to the sample rate.
　　视频可以尽可能的快速了，但是音频正在及时、最后的播放。为什么？这是因为音频信息由一个sample 率---我们尽可能快的打出音频信息，但是音频只是根据sample rate 在空闲时从流中播放。
　　We're almost ready to start syncing video and audio ourselves, but first we need to do a little program reorganization. The method of queueing up audio and playing it using a separate thread worked very well: it made the code more managable and more modular. Before we start syncing the video to the audio, we need to make our code easier to deal with. Next time: Spawning Threads!
　　我们准备开始同步音频和视频了。但是首先，我们需要做的是改组、改编。对音频进行入队的方法并使用这一音频播放一个单独的线程，工作的很好：这是的代码更有可组织性和模块化。在我们开始同步视频到音频的之前，我们需要使得我们的代码更容易处理。
　　>> Spawning Threads§