Stream Images From Python Opencv With Ffmpeg

In order to try an embedded AI, I want to stream an image dataset through a rtsp stream. What I tried to do is to read one image every X seconds and send it to the stream and infer

Solution 1:

Here is a reproducible sample - hoping you can copy paste and execute, but nothing is promised...

The example applies the following stages:

• Create 10 synthetic JPEG images in ./test_dataset folder, to be used as input.
• Execute FFplay sub-process as RTSP listener. When using TCP protocol we should start the TCP server first (FFplay is used as a TCP server in out case). We also need the receiver process, because without it, FFmpeg streamer process halts after the first frame.
• Execute FFmpeg sub-process for RTSP streaming. Cyclically read JPEG image to NumPy array (in BGR color format), and write the array as raw video frame to stdin pipe. Note: It is more efficient to write raw video frames, than encoding each frame to PNG (as used by your reference sample).

---

Here is the code:

import cv2
#import timeimport subprocess as sp 
import glob
import os

img_width = 1280
img_height = 720


test_path = './test_dataset'# Folder with synthetic sample images.

os.makedirs(test_path, exist_ok=True)  # Create folder for input images.

os.chdir(test_path)

ffmpeg_cmd = 'ffmpeg'# May use full path like: 'c:\\FFmpeg\\bin\\ffmpeg.exe'
ffplay_cmd = 'ffplay'# May use full path like: 'c:\\FFmpeg\\bin\\ffplay.exe'# Create 10 synthetic JPEG images for testing (image0001.jpg, image0002.jpg, ..., image0010.jpg).
sp.run([ffmpeg_cmd, '-y', '-f', 'lavfi', '-i', f'testsrc=size={img_width}x{img_height}:rate=1:duration=10', 'image%04d.jpg'])


img_list = glob.glob("*.jpg")
img_list_len = len(img_list)
img_index = 0

fps = 5

rtsp_server = 'rtsp://localhost:31415/live.stream'# You will need to start the server up first, before the sending client (when using TCP). See: https://trac.ffmpeg.org/wiki/StreamingGuide#Pointtopointstreaming
ffplay_process = sp.Popen([ffplay_cmd, '-rtsp_flags', 'listen', rtsp_server])  # Use FFplay sub-process for receiving the RTSP video.


command = [ffmpeg_cmd,
           '-re',
           '-f', 'rawvideo',  # Apply raw video as input - it's more efficient than encoding each frame to PNG'-s', f'{img_width}x{img_height}',
           '-pixel_format', 'bgr24',
           '-r', f'{fps}',
           '-i', '-',
           '-pix_fmt', 'yuv420p',
           '-c:v', 'libx264',
           '-bufsize', '64M',
           '-maxrate', '4M',
           '-rtsp_transport', 'tcp',
           '-f', 'rtsp',
           #'-muxdelay', '0.1',
           rtsp_server]

process = sp.Popen(command, stdin=sp.PIPE)  # Execute FFmpeg sub-process for RTSP streamingwhileTrue:
    current_img = cv2.imread(img_list[img_index])  # Read a JPEG image to NumPy array (in BGR color format) - assume the resolution is correct.
    img_index = (img_index+1) % img_list_len  # Cyclically repeat images

    process.stdin.write(current_img.tobytes())  # Write raw frame to stdin pipe.

    cv2.imshow('current_img', current_img)  # Show image for testing# time.sleep(1/FPS)
    key = cv2.waitKey(int(round(1000/fps)))  # We need to call cv2.waitKey after cv2.imshowif key == 27:  # Press Esc for exitbreak


process.stdin.close()  # Close stdin pipe
process.wait()  # Wait for FFmpeg sub-process to finish
ffplay_process.kill()  # Forcefully close FFplay sub-process
cv2.destroyAllWindows()  # Close OpenCV window