FFmpeg SPS/PPS剖析

場(chǎng)景說明
?? ?????????在解碼過程中，需要設(shè)置SPS/PPS等解碼信息，才能夠初始化×××。有兩種方式可以設(shè)置SPS/PPS，一種是手動(dòng)指定SPS/PPS內(nèi)容，指定AVCodecContext
結(jié)構(gòu)體中extradata的值；一種是讓FFmpeg通過讀取傳輸數(shù)據(jù)來分析SPS/PPS信息，一般情況下在每一個(gè)I幀之前都會(huì)發(fā)送一個(gè)SPS幀和PPS幀


方法1）一般的攝像機(jī)都會(huì)在RTSP指令中SDP攜帶的SPS/PPS內(nèi)容，例如?？礗PC發(fā)送的RTSP指令中攜帶信息如下：sprop-parameter-sets=Z0IAKpY1QPAET8s3AQEBQAABwgAAV+QB,aM48gA==
分號(hào)前面是SPS,后面是PPS,這是base64編碼的結(jié)果，需要解碼，才能夠傳遞給extradata,并且還需要在SPS/PPS字符串前面添加起始碼（0x00 0x00 0x00 0x01）。補(bǔ)充一點(diǎn)SPS的類型是
0x67, PPS的類型是0x68

char szBase64SPSBuffer[] = { "Z0IAKpY1QPAET8s3AQEBQAABwgAAV+QB" };
uint8_t szDecodedSPS[128] = { 0 };
int nSPSLen = av_base64_decode(szDecodedSPS, szBase64SPSBuffer, sizeof(szBase64SPSBuffer));//nSPSLen=24

char szBase64PPSBuffer[] = { "aM48gA==" };
uint8_t szDecodedPPS[128] = { 0 };
int nPPSLen = av_base64_decode(szDecodedPPS, szBase64PPSBuffer, sizeof(szBase64PPSBuffer));//nPPSLen=4

將SPS/PPS拼湊的結(jié)果如下：
unsigned char szSPSPPS[] = { 0x00 ,0x00 ,0x01,0x67,0x42,0x00 ,0x2a ,0x96 ,0x35 ,0x40 ,0xf0 ,0x04 ,0x4f ,0xcb ,0x37 ,0x01 ,0x01 ,0x01 ,0x40 ,0x00 ,0x01 ,0xc2 ,0x00 ,0x00 ,0x57 ,0xe4 ,0x01 ,0x00 ,0x00 ,0x00 ,0x01 ,0x68 ,0xce ,0x3c ,0x80, 0x00};


然后拷貝SPS/PPS數(shù)據(jù)到AVFormatContext的extradata
unsigned char szSPSPPS[] = { 0x00 ,0x00 ,0x01,0x67,0x42,0x00 ,0x2a ,0x96 ,0x35 ,0x40 ,0xf0 ,0x04 ,0x4f ,0xcb ,0x37 ,0x01 ,0x01 ,0x01 ,0x40 ,0x00 ,0x01 ,0xc2 ,0x00 ,0x00 ,0x57 ,0xe4 ,0x01 ,0x00 ,0x00 ,0x00 ,0x01 ,0x68 ,0xce ,0x3c ,0x80, 0x00};
pFormatContext->streams[0]->codecpar->extradata_size = sizeof(szSPSPPS);
pFormatContext->streams[0]->codecpar->extradata = (uint8_t*)av_mallocz(pFormatContext->streams[0]->codecpar->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE);
memcpy(pFormatContext->streams[0]->codecpar->extradata, szSPSPPS, sizeof(szSPSPPS));

通過avcodec_parameters_to_context將信息從pFormatContext->streams[0]->codecpar拷貝到m_pAVCodecContext
avcodec_open2函數(shù)在調(diào)用的時(shí)候，會(huì)解析extradata數(shù)據(jù)的內(nèi)容


出錯(cuò)異常

I:2018-01-08 14:23:00 ms:221:nal_unit_type: 7, nal_ref_idc: 3
I:2018-01-08 14:23:00 ms:221:nal_unit_type: 8, nal_ref_idc: 3
I:2018-01-08 14:23:00 ms:221:sps:0 profile:66/42 poc:0 ref:1 120x68 FRM 8B8 crop:0/0/0/8 VUI 420 1800/90000 b8 reo:-1
I:2018-01-08 14:23:00 ms:221:pps:0 sps:0 CAVLC slice_groups:1 ref:1/1? qp:26/26/0/0 LPAR? ?
I:2018-01-08 14:23:00 ms:222:deprecated pixel format used, make sure you did set range correctly
I:2018-01-08 14:23:00 ms:250:non-existing PPS 0 referenced

錯(cuò)誤分析
????????從av_log日志可以看出已經(jīng)正確的解析出來PPS,但是在進(jìn)行avcodec_read_frame的時(shí)候 還是打印出錯(cuò)non-existing PPS 0 referenced,這是什么問題？
這主要是沒有在緩沖區(qū)中讀取到包含SPS/PPS信息的幀,而不是說×××沒有正確初始化SPS/PPS參數(shù)，這一點(diǎn)需要注意

問題
SDP中sprop-parameter-sets=Z0IAKpY1QPAET8s3AQEBQAABwgAAV+QB,aM48gA==
SPS=Z0IAKpY1QPAET8s3AQEBQAABwgAAV+QB
PPS=aM48gA==
十六進(jìn)制表示：
5a 30 49 41 4b 70 59 31 51 50 41 45 54 38 73 33 41 51 45 42 51 41 41 42 77 67 41 41 56 2b 51 42
碼流中的SPS和PPS
unsigned char sps_pps[] = { 0x00 ,0x00 ,0x01,0x67,0x42,0x00 ,0x2a ,0x96 ,0x35 ,0x40 ,0xf0 ,0x04 ,0x4f ,0xcb ,0x37 ,0x01 ,0x01 ,0x01 ,0x40 ,0x00 ,0x01
,0xc2 ,0x00 ,0x00 ,0x57 ,0xe4 ,0x01 ,0x00 ,0x00 ,0x00 ,0x01 ,0x68 ,0xce ,0x3c ,0x80, 0x00};
兩者完全不一致，暫時(shí)沒有頭緒

展望

????????從這里就可以看出，實(shí)際上并沒有多大的必要手動(dòng)設(shè)置SPS/PPS，因?yàn)镕Fmpeg會(huì)從緩沖中分析出SPS/PPS，然后解析出碼流參數(shù)，估計(jì)唯一的一種極端情況是I幀之前沒有任何的SPS/PPS信息，或者需要很長(zhǎng)時(shí)間才會(huì)發(fā)送一幀包含SPS/PPS的信息

相關(guān)代碼定義
/**
?* This struct describes the properties of an encoded stream.
?
*
?* sizeof(AVCodecParameters) is not a part of the public ABI, this struct must
?
* be allocated with avcodec_parameters_alloc() and freed with
?
* avcodec_parameters_free().
?
*/
typedef struct AVCodecParameters {
/**
???? * Extra binary data needed for initializing the decoder, codec-dependent.
???? *
??? ?
* Must be allocated with av_malloc() and will be freed by
??? ?
* avcodec_parameters_free(). The allocated size of extradata must be at
??? ?
* least extradata_size + AV_INPUT_BUFFER_PADDING_SIZE, with the padding
???? * bytes zeroed.
???? */

??? uint8_t *extradata;
?? ?
/**
???? * Size of the extradata content in bytes. ?
*/
??? int????? extradata_size;
}

描述說明
額外的二進(jìn)制數(shù)據(jù)用來初始化×××，主要是初始化SPS/PPS，必須通過av_malloc進(jìn)行內(nèi)存分配，然后通過avcodec_parameters_free進(jìn)行釋放。內(nèi)存分配的大小必須是extradata_size的大小加上AV_INPUT_BUFFER_PADDING_SIZE二進(jìn)制數(shù)據(jù)的真實(shí)長(zhǎng)度大小保存在extradata_size