Implementing NVIDIA Video Codec SDK in Streaming Workflows

> vPacket;encoder->EncodeFrame(vPacket);for (const auto& packet : vPacket) { // send packet over network or write to file} The CopyToDeviceFrame() function copies raw NV12/YUV data to the registered input surface. EncodeFrame() encodes a single frame and returns one or more encoded packets. The output is a vector of std::vector buffers that hold the H.264/HEVC bitstream. If using asynchronous CUDA streams, ensure proper synchronization before reuse of memory. NVDEC Workflow Overview NVDEC handles the reverse: decoding compressed H.264/H.265 bitstreams into raw YUV frames. This is essential for real-time playback, editing, or AI inference pipelines. The decoding sequence involves: Parser Setup : CUvideoparser is created using a set of parameters that specify codec type, resolution, decode surfaces, and callback functions. Feeding Compressed Data : The application pushes H.264 or H.265 packets to the parser using cuvidParseVideoData(). Handling Decoded Frames : Upon decoding, frames are made available in GPU memory via a decode callback. These can be used for inference, rendering, or re-encoding. Frame Memory Management : Decoded frames are returned as CUdeviceptr. For integration with CUDA or OpenGL, they must be mapped or copied depending on use. NVDEC decodes directly to GPU memory in NV12 or P016 format. NVDEC Initialization and Decode The decoder is initialized by creating a video parser configured for the codec type and maximum decode surfaces. The parser invokes callbacks when frames are decoded or ready for display. Compressed packets are fed to the parser in sequence, which manages frame reconstruction and delivers decoded frames in GPU memory for further use. CUVIDPARSERPARAMS parserParams = {};parserParams.CodecType = cudaVideoCodec_H264;parserParams.ulMaxNumDecodeSurfaces = 20;parserParams.pfnDecodePicture = DecodeCallback;parserParams.pUserData = this;cuvidCreateVideoParser(&parser, &parserParams); The parser callback ( pfnDecodePicture, pfnDisplayPicture ) handles decode and display notifications. When a frame is available, cuvidMapVideoFrame() is used to access the frame buffer. Compressed packets are then pushed: CUVIDSOURCEDATAPACKET packet = { 0 };packet.payload = compressedData;packet.payload_size = dataSize;packet.flags = CUVID_PKT_ENDOFPICTURE;cuvidParseVideoData(parser, &packet); Frames are decoded in order and ready for processing. Memory Management and Buffering Efficient memory allocation and transfer are critical for performance. Device memory for frames is allocated with alignment using cudaMallocPitch() . Asynchronous memory copies help overlap data transfer with computation, reducing latency. Pinned host memory and zero-copy techniques improve data transfer speed, especially when integrating with external capture devices. uint8_t* d_frame;size_t pitch;cudaMallocPitch(&d_frame, &pitch, width, height * 3 / 2); // For NV12// Copy data for encodingcudaMemcpyAsync(d_frame, src_ptr, size, cudaMemcpyHostToDevice, stream); Pinned host memory and zero-copy access via cudaHostRegister are recommended for integrating with frame grabbers or DMA sources. Bitrate Control and Latency Optimization The encoder supports various rate control modes: CBR (Constant Bitrate) for streaming. VBR (Variable Bitrate) for file recording. ConstQP for lossless or controlled compression. Low-latency presets to minimize B-frame buffering and enable faster delivery. Set via: encConfig.rcParams.rateControlMode = NV_ENC_PARAMS_RC_CBR;encConfig.rcParams.averageBitRate = 5000000;encConfig.rcParams.vbvBufferSize = 5000000; For ultra-low-latency pipelines, use: NV_ENC_PRESET_LOW_LATENCY_HP_GUID NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ

Boost Campaign Performance Through Video