做网站 需要了解什么,上海网站设计开,关键词搜索排名,做网站怎样写标题技术背景
我们在对接GB28181设备接入模块的时候#xff0c;遇到这样的技术诉求#xff0c;好多开发者期望能提供编码后#xff08;H.264/H.265、AAC/PCMA#xff09;数据对接#xff0c;确保外部采集设备#xff0c;比如无人机类似回调过来的数据#xff0c;直接通过模… 技术背景
我们在对接GB28181设备接入模块的时候遇到这样的技术诉求好多开发者期望能提供编码后H.264/H.265、AAC/PCMA数据对接确保外部采集设备比如无人机类似回调过来的数据直接通过模块对接到GB28181平台侧此外还期望不支持或者内网没有外部网络权限的RTSP设备也能间接接入到国标平台。
技术实现
编码后音视频数据
本文以Android平台为例基于上诉诉求我们设计的接口如下简单来说GB28181交互流程不变只要提供数据接入接口即可
/*** 设置编码后视频数据(H.264)** param codec_id, H.264对应 1** param data 编码后的video数据** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0.** param timestamp video timestamp** param pts Presentation Time Stamp, 显示时间戳** return {0} if successful*/public native int SmartPublisherPostVideoEncodedData(long handle, int codec_id, ByteBuffer data, int size, int is_key_frame, long timestamp, long pts);/*** 设置编码后视频数据(H.264)** param codec_id, H.264对应 1** param data 编码后的video数据**param offset data的偏移** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0.** param timestamp video timestamp** param pts Presentation Time Stamp, 显示时间戳** return {0} if successful*/public native int SmartPublisherPostVideoEncodedDataV2(long handle, int codec_id,ByteBuffer data, int offset, int size,int is_key_frame, long timestamp, long pts,byte[] sps, int sps_len,byte[] pps, int pps_len);/*** 设置编码后视频数据(H.264)如需录制编码后的数据用此接口且设置实际宽高** param codec_id, H.264对应 1** param data 编码后的video数据**param offset data的偏移** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0.** param timestamp video timestamp** param pts Presentation Time Stamp, 显示时间戳** param width, height: 编码后视频宽高** return {0} if successful*/public native int SmartPublisherPostVideoEncodedDataV3(long handle, int codec_id,ByteBuffer data, int offset, int size,int is_key_frame, long timestamp, long pts,byte[] sps, int sps_len,byte[] pps, int pps_len,int width, int height);/*** 设置音频数据(AAC/PCMA/PCMU/SPEEX)** param codec_id:** NT_MEDIA_CODEC_ID_AUDIO_BASE 0x10000,* NT_MEDIA_CODEC_ID_PCMA NT_MEDIA_CODEC_ID_AUDIO_BASE,* NT_MEDIA_CODEC_ID_PCMU,* NT_MEDIA_CODEC_ID_AAC,* NT_MEDIA_CODEC_ID_SPEEX,* NT_MEDIA_CODEC_ID_SPEEX_NB,* NT_MEDIA_CODEC_ID_SPEEX_WB,* NT_MEDIA_CODEC_ID_SPEEX_UWB,** param data audio数据** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0, audio忽略** param timestamp video timestamp** param parameter_info 用于AAC special config信息填充** param parameter_info_size parameter info size** return {0} if successful*/public native int SmartPublisherPostAudioEncodedData(long handle, int codec_id, ByteBuffer data, int size, int is_key_frame, long timestamp,ByteBuffer parameter_info, int parameter_info_size);/*** 设置音频数据(AAC/PCMA/PCMU/SPEEX)** param codec_id:** NT_MEDIA_CODEC_ID_AUDIO_BASE 0x10000,* NT_MEDIA_CODEC_ID_PCMA NT_MEDIA_CODEC_ID_AUDIO_BASE,* NT_MEDIA_CODEC_ID_PCMU,* NT_MEDIA_CODEC_ID_AAC,* NT_MEDIA_CODEC_ID_SPEEX,* NT_MEDIA_CODEC_ID_SPEEX_NB,* NT_MEDIA_CODEC_ID_SPEEX_WB,* NT_MEDIA_CODEC_ID_SPEEX_UWB,** param data audio数据** param offset data的偏移** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0, audio忽略** param timestamp video timestamp** param parameter_info 用于AAC special config信息填充** param parameter_info_size parameter info size** return {0} if successful*/public native int SmartPublisherPostAudioEncodedDataV2(long handle, int codec_id,ByteBuffer data, int offset, int size,int is_key_frame, long timestamp,byte[] parameter_info, int parameter_info_size);/*** 设置音频数据(AAC/PCMA/PCMU/SPEEX)** param codec_id:** NT_MEDIA_CODEC_ID_AUDIO_BASE 0x10000,* NT_MEDIA_CODEC_ID_PCMA NT_MEDIA_CODEC_ID_AUDIO_BASE,* NT_MEDIA_CODEC_ID_PCMU,* NT_MEDIA_CODEC_ID_AAC,* NT_MEDIA_CODEC_ID_SPEEX,* NT_MEDIA_CODEC_ID_SPEEX_NB,* NT_MEDIA_CODEC_ID_SPEEX_WB,* NT_MEDIA_CODEC_ID_SPEEX_UWB,** param data audio数据** param offset data的偏移** param size data length** param is_key_frame 是否I帧, if with key frame, please set 1, otherwise, set 0, audio忽略** param timestamp video timestamp** param parameter_info 用于AAC special config信息填充** param parameter_info_size parameter info size** param sample_rate 采样率,如果需要录像的话必须传正确的值**param channels 通道数, 如果需要录像的话必须传正确的值, 一般是1或者2** return {0} if successful*/public native int SmartPublisherPostAudioEncodedDataV3(long handle, int codec_id,ByteBuffer data, int offset, int size,int is_key_frame, long timestamp,byte[] parameter_info, int parameter_info_size,int sample_rate, int channels);
拉取RTSP流接入到GB28181平台 简单那来说把摄像机的RTSP流数据拉下来然后回调编码后的数据到上层上层根据GB28181数据格式要求实现PS打包然后通过对接GB28181平台信令和数据交互国标平台侧需要预览的时候信令交互后拉RTSP即可。
如何拉流
private boolean StartPull(){if ( isPulling )return false;if (!OpenPullHandle())return false;libPlayer.SmartPlayerSetAudioDataCallback(playerHandle, new PlayerAudioDataCallback());libPlayer.SmartPlayerSetVideoDataCallback(playerHandle, new PlayerVideoDataCallback());int is_pull_trans_code 1;libPlayer.SmartPlayerSetPullStreamAudioTranscodeAAC(playerHandle, is_pull_trans_code);int startRet libPlayer.SmartPlayerStartPullStream(playerHandle);if (startRet ! 0) {Log.e(TAG, Failed to start pull stream!);if(!isPlaying !isRecording isPushing !isRTSPPublisherRunning){libPlayer.SmartPlayerClose(playerHandle);playerHandle 0;}return false;}isPulling true;return true;}private void StopPull(){if ( !isPulling )return;libPlayer.SmartPlayerStopPullStream(playerHandle);if ( !isPlaying !isRecording !isPushing !isRTSPPublisherRunning){libPlayer.SmartPlayerClose(playerHandle);playerHandle 0;}isPulling false;}
拉到的音视频数据投递到GB28181接入模块
class PlayerAudioDataCallback implements NTAudioDataCallback{private int audio_buffer_size 0;private int param_info_size 0;private ByteBuffer audio_buffer_ null;private ByteBuffer parameter_info_ null;Overridepublic ByteBuffer getAudioByteBuffer(int size){//Log.i(getAudioByteBuffer, size: size);if( size 1 ){return null;}if ( size audio_buffer_size audio_buffer_ ! null ){return audio_buffer_;}audio_buffer_size size 512;audio_buffer_size (audio_buffer_size0xf) (~0xf);audio_buffer_ ByteBuffer.allocateDirect(audio_buffer_size);// Log.i(getAudioByteBuffer, size: size buffer_size: audio_buffer_size);return audio_buffer_;}Overridepublic ByteBuffer getAudioParameterInfo(int size){//Log.i(getAudioParameterInfo, size: size);if(size 1){return null;}if ( size param_info_size parameter_info_ ! null ){return parameter_info_;}param_info_size size 32;param_info_size (param_info_size0xf) (~0xf);parameter_info_ ByteBuffer.allocateDirect(param_info_size);//Log.i(getAudioParameterInfo, size: size buffer_size: param_info_size);return parameter_info_;}public void onAudioDataCallback(int ret, int audio_codec_id, int sample_size, int is_key_frame, long timestamp, int sample_rate, int channel, int parameter_info_size, long reserve){if ( audio_buffer_ null)return;audio_buffer_.rewind();if ( ret 0 (isPushing || isRTSPPublisherRunning || isGB28181StreamRunning)) {libPublisher.SmartPublisherPostAudioEncodedData(publisherHandle, audio_codec_id, audio_buffer_, sample_size, is_key_frame, timestamp, parameter_info_, parameter_info_size);}}}class PlayerVideoDataCallback implements NTVideoDataCallback{private int video_buffer_size 0;private ByteBuffer video_buffer_ null;Overridepublic ByteBuffer getVideoByteBuffer(int size){if( size 1 ){return null;}if ( size video_buffer_size video_buffer_ ! null ){return video_buffer_;}video_buffer_size size 1024;video_buffer_size (video_buffer_size0xf) (~0xf);video_buffer_ ByteBuffer.allocateDirect(video_buffer_size);return video_buffer_;}public void onVideoDataCallback(int ret, int video_codec_id, int sample_size, int is_key_frame, long timestamp, int width, int height, long presentation_timestamp){if ( video_buffer_ null)return;video_buffer_.rewind();if ( ret 0 (isPushing || isRTSPPublisherRunning || isGB28181StreamRunning) ) {libPublisher.SmartPublisherPostVideoEncodedData(publisherHandle, video_codec_id, video_buffer_, sample_size, is_key_frame, timestamp, presentation_timestamp);}}}
如何预览播放外部音视频数据
除了想把编码后的音视频数据转至GB28181外有些场景下还需要本地预览甚至对数据做二次处理视频分析、实时水印字符叠加等然后二次编码基于这样的场景诉求我们实现了Android平台外部编码数据实时预览播放模块。 外部H.264/H.265投递接口设计如下 // SmartPlayerJniV2.java// Author: daniusdk.com/*** 投递视频包给外部Live Source** param codec_id: 编码id, 当前仅支持H264和H265, 1:H264, 2:H265** param packet: 视频数据, ByteBuffer必须是DirectBuffer, 包格式请参考H264/H265 Annex B Byte stream format, 例如:* 0x00000001 nal_unit 0x00000001 ...* H264 IDR: 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....* H265 IDR: 0x00000001 vps 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit ....** param offset: 偏移量* param size: packet size* param timestamp_ms: 时间戳, 单位毫秒* param is_timestamp_discontinuity: 是否时间戳间断0:未间断1:间断* param is_key: 是否是关键帧, 0:非关键帧, 1:关键帧* param extra_data: 可选参数可传null, 对于H264关键帧包, 如果packet不含sps和pps, 可传0x00000001 sps 0x00000001 pps* ,对于H265关键帧包, 如果packet不含vps,sps和pps, 可传0x00000001 vps 0x00000001 sps 0x00000001 pps* param extra_data_size: extra_data size* param width: 图像宽, 可传0* param height: 图像高, 可传0** return {0} if successful*/public native int PostVideoPacketByteBuffer(long handle, int codec_id,java.nio.ByteBuffer packet, int offset, int size, long timestamp_ms, int is_timestamp_discontinuity, int is_key,byte[] extra_data, int extra_data_size, int width, int height);/** 请参考 PostVideoPacketByteBuffer说明*/public native int PostVideoPacketByteArray(long handle, int codec_id,byte[] packet, int offset, int size, long timestamp_ms, int is_timestamp_discontinuity, int is_key,byte[] extra_data, int extra_data_size, int width, int height);
PostVideoPacketByteBuffer()和PostVideoPacketByteArray()接口设计基本类似唯一的区别在于一个数据类型是ByteBuffer一个是byte数组。
其中codec_id系编码id目前仅支持H.264和H.265类型。 packet视频数据需要注意的是ByteBuffer必须是DirectBuffer, 包格式请参考H264/H265 Annex B Byte stream format, 例如: 0x00000001 nal_unit 0x00000001 ... H264 IDR: 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit .... H265 IDR: 0x00000001 vps 0x00000001 sps 0x00000001 pps 0x00000001 IDR_nal_unit .... 或 0x00000001 IDR_nal_unit .... extra_data: 可选参数可传null, 对于H264关键帧包如果packet不含sps和pps可传0x00000001 sps 0x00000001 pps对于H265关键帧包如果packet不含vps,sps和pps, 可传0x00000001 vps 0x00000001 sps 0x00000001 pps
音频AAC/PCMA/PCMU投递接口设计如下 /*** 投递音频包给外部Live source, 注意ByteBuffer对象必须是DirectBuffer** param handle: return value from SmartPlayerOpen()** param codec_id: 编码id, 当前支持PCMA、PCMU和AAC, 65536:PCMA, 65537:PCMU, 65538:AAC* param packet: 音频数据* param offsetpacket偏移量* param size: packet size* param pts_ms: 时间戳, 单位毫秒* param is_pts_discontinuity: 是否时间戳间断false:未间断true:间断* param extra_data: 如果是AAC的话需要传 Audio Specific Configuration* param extra_data_offset: extra_data 偏移量* param extra_data_size: extra_data size* param sample_rate: 采样率* param channels: 通道数** return {0} if successful*/public native int PostAudioPacket(long handle, int codec_id,java.nio.ByteBuffer packet, int offset, int size, long pts_ms, boolean is_pts_discontinuity,java.nio.ByteBuffer extra_data, int extra_data_offset, int extra_data_size, int sample_rate, int channels);/** 投递音频包给外部Live source, byte数组版本, 具体请参考PostAudioPacket** param is_pts_discontinuity: 是否时间戳间断0:未间断1:间断* return {0} if successful*/public native int PostAudioPacketByteArray(long handle, int codec_id,byte[] packet, int offset, int size, long pts_ms, int is_pts_discontinuity,byte[] extra_data, int extra_data_size, int sample_rate, int channels);
总结
通过以上描述大家可以看到GB/T 28181音视频数据源接入无论是编码前还是编码后数据或外部RTSP流数据包括数据预览如果有技术积累的话实现起来也没那么麻烦感兴趣的开发者可以尝试看。
